open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(fixtures): replace sector-specific example material with generic, fictitious examples — green

Browse source 
Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.

- Generated fixtures are regenerated from their generators. Their structural
  counts are identical before and after: elements, images, rows, cells,
  headings, bookmarks and the witness inventory's per-document totals. The
  image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
  each are replaced by a generic, repeatable --corpus PATH with no default.
  Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
  removed, along with its three twins and five tests. The mutation harness
  loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
  are removed, and 40 are neutralized. Dead links are removed, and no new
  dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.

Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.

Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 13:54:57 +02:00
commit 9d1f4b14ed
174 changed files with 1889 additions and 6512 deletions
Download patch file Download diff file Expand all files Collapse all files

85
CHANGELOG.md
View file

@ -230,7 +230,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **Every carried image is now one a model can be SHOWN, and the ones that
cannot be are refused out loud.** Until this round the asset path carried
whatever format a publisher shipped. Measured 2026-09-19 over the frozen
R761 delivery's own `assets/` (denominator 50): 29 JPEG, 2 PNG and **19 "PC
reference standard's own `assets/` (denominator 50): 29 JPEG, 2 PNG and **19 "PC
bitmap, Windows 3.x, 8-bit, compression 1"** — RLE8 BMP. The 19 are
byte-correct files nothing reads, so 19 of that document's figures were
present and invisible at once, with `images: N` reporting that they had
@ -249,7 +249,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
bundle's identity move with that library's version — the property 0.10.0
felled page rasterisation over. Pillow is the INDEPENDENT decoder in the
tests instead.
- **Lossless, measured on the real files:** all **19 of 19** R761 RLE8
- **Lossless, measured on the real files:** all **19 of 19** reference-standard RLE8
assets convert with RGB identical to Pillow's decoding of the source,
**2 366 365 pixels** compared.
- **Traceability per converted asset**, on the pointer line where the rest
@ -279,9 +279,9 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
COUNTED** (operator decision 2026-09-18). `llm-ingestion-guard` 1.4.0 keeps
the soft hyphen in `_ZERO_WIDTH_CPS`, and `output:zero-width-present` is an
any-tier carrier — `fail_secure` at every trust level, no sanitisation, no
exception. R761 Prosesskoden:2025 carries 71 of them and 0 of the four real
exception. A 701-page reference standard carries 71 of them and 0 of the four real
zero-width characters; all 71 are Norwegian hyphenation points inside words,
so a 701-page process code was unreadable for the whole chain over
so a 701-page reference standard was unreadable for the whole chain over
typography. `extract.normalise_extracted` removes that one character from
every extracted text; `ExtractedDocument.soft_hyphens`,
`InboxResult.normalised` and the accounting's `normalised_soft_hyphen` carry
@ -309,7 +309,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **The judge is `tools/okf_accounting_gate.py`**, written red first
against an independent witness (`tools/okf_witness.py`, which imports
nothing from this package). At this change it is green on all six rows,
including R761 Prosesskoden:2025: 110 of 110 units under both the default
including a 701-page reference standard: 110 of 110 units under both the default
gate and `--gate none`.
- **Opt-in, measured.** On the 43-document reference corpus the build took
+744 s (+19 %) and +0.53 GB peak RSS.
@ -330,7 +330,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
number, because no witness counts a heading in a PDF.
- **Limit, measured:** the check proves that a string is present, not
where. Short elements such as a section label or a one-word title are
often found elsewhere in the same document. With R761's concept text cut
often found elsewhere in the same document. With the reference standard's concept text cut
to half, 4 823 paragraphs and 3 621 sections were reported lost, but only
3 titles and 16 labels.
@ -339,7 +339,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **`okf build` exits 1 when it extracted at least one document and
persisted none.** Until now such a run exited 0, because every refusal was
coded and the conservation identity held. The bundle was nonetheless empty.
Measured case: guard 1.4.0 refuses R761 Prosesskoden:2025 whole, because of
Measured case: guard 1.4.0 refuses a 701-page reference standard whole, because of
its 71 soft hyphens (U+00AD). Door B's library function
(`process_inbox`) and `corpus.measure` are unchanged; for a hostile inbox,
"all rejected" is a correct outcome.
@ -350,14 +350,14 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **A file carried through a document is no longer also a coded rejection.**
Since 0.10.0, an image beside a document was carried into `assets/` through
that document and was ALSO counted as `extractor_unknown`, so one file had
two fates. On R761 under `--gate none` that was 50 files.
two fates. On the reference standard under `--gate none` that was 50 files.
- The conservation identity is now `merged + files carried through a
document + coded rejections = N`.
- `log.md` writes the middle term only when it is non-zero, so a corpus with
no such files keeps its line byte for byte.
- The carried files are the references the reader actually resolved and
carried (`ExtractedDocument.files`), never a byte match. A byte match
would credit R761's 7 unpointed duplicates.
would credit the reference standard's 7 unpointed duplicates.
- An unpointed file beside a document stays a coded rejection.
- **`log.md`'s `Images: C carried of F found`**: with `--accounting`, F is
what the SOURCES declare. A refused document's pictures therefore no longer
@ -512,8 +512,8 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
becoming true or one concept ranking better. An eval written before the
capability has one job beyond being red today, so the gate was repaired
before anything is built against it. `src/` is untouched.
- **Row 8 requires all three named sets** (`wiki-20`, `r761-sk2`,
`vegnormal-32`) and is NOT RUN otherwise. It counted whatever `--real`
- **Row 8 requires all three named sets** (`wiki-20` and two sets built on a
retired reference corpus) and is NOT RUN otherwise. It counted whatever `--real`
gave it, so one set of three read `6 of 6 GREEN` — and this repository's
own test asserted `(1, 1, GREEN)` for a single set. The numbers the run
DID measure are still printed: a missing set must not cost the reader the
@ -561,8 +561,8 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
one it stays RED rather than NOT RUN: its denominator is known.
- **Row 8 ran, against all three real sets**: **44 of 64 questions**, 7 of
29 at citation granularity and 38 of 50 at concept granularity, 33 of 34
misses class b. wiki (6 of 20) and r761 (7 of 7) reproduce PM's recorded
figures exactly; vegnormal measures 31 of 43 citations where PM recorded
misses class b. wiki (6 of 20) and the first retired set (7 of 7) reproduce PM's recorded
figures exactly; the second retired set measures 31 of 43 citations where PM recorded
32, a one-citation disagreement between two instruments over the same
pinned bytes, stated and not resolved here.
- Rows 1 and 6 go 9 of 9 to 10 of 10 (one added fixture, one added hit).
@ -595,7 +595,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
reading the gate had before the conversion route existed.
- Measured: the three arms PM reproduced go forged → refused, 3 of 3, with
the known-positive (a BMP the run really does convert) True in all three.
The text-level regression guard goes 3 arms to 13. R761 rebuilt is
The text-level regression guard goes 3 arms to 13. The reference standard rebuilt is
`diff -r`-identical, 50 assets (29 JPEG + 21 PNG), 19 of 19 conversions
confirmed against 19 declared, SHY 71, u = 0, d = 0.
- **An RLE8 stream that stops before the frame is refused (0.10.1).** The
@ -609,7 +609,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
end-of-line escape state their skip, so the pixels they pass over keep
index 0 and every decoder produces the same picture; a pixel-coverage
count would refuse both constructions the format defines. The corpus
cannot choose between the two rules — over the 25 RLE8 BMPs the R761
cannot choose between the two rules — over the 25 RLE8 BMPs the reference
delivery ships, 25 of 25 paint every pixel, 25 of 25 reach the end of the
frame and 0 of 25 use a delta — and an independent decoder can: Pillow
reads 5 of the 8 streams in the table and refuses the same 3 the new rule
@ -655,7 +655,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
`unaccounted` or `double_booked`, which the door had written for a round
longer. A door built from the contract writes a ledger the gate reads as
"nothing was converted", and every converted image comes out
claimed-and-not-found — 19 of 50 on R761. Two tests hold the sketch against
claimed-and-not-found — 19 of 50 on the reference standard. Two tests hold the sketch against
both sides: what the gate LOOKS UP (measured with a ledger that records its
own lookups, not by grep) and what the door SERIALISES.
- **A bundle built without the door now says why a converted image cannot be
@ -694,7 +694,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
(0.10.1).** `_bmp_rle8_rows` painted what the stream held and left the rest
of the frame at palette index 0 — which is what the format says about a
SKIPPED pixel, so no decoder disagreed and the picture was wrong with no
code and no row. Measured by PM 2026-09-19 on a real R761 asset (352x548 =
code and no row. Measured by PM 2026-09-19 on a real reference-standard asset (352x548 =
192 896 pixels): cut to 90 % it was carried with 13 923 pixels wrong, to
50 % with 95 890, to 10 % with 166 525. The uncompressed path already
refused the same shape.
@ -705,7 +705,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- The rule is the terminator rather than `biSizeImage` (a claim by the same
untrusted header) or a coverage count (which would refuse the delta escape
the format defines), and it is read off the corpus: over the 19 real RLE8
assets of the frozen R761 delivery, **19 of 19** end at an explicit
assets of the frozen reference delivery, **19 of 19** end at an explicit
end-of-bitmap, on **19 of 19** it is the stream's last two bytes, and on
**19 of 19** `biSizeImage` equals the available bytes. A whole stream that
omits the terminator is refused alongside a cut one.
@ -723,8 +723,8 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
in the note.
- **The STS JSON role map reads the publisher's own tags.** `count_sts_json`
compared the raw tag string where the XML witness has always used `_local`,
so `mml:math` reached `tag == "math"` on nothing — 74 formulas in N200
Vegbygging:2024 counted as 0. And the publisher's JSON writes a figure's
so `mml:math` reached `tag == "math"` on nothing — 74 formulas in a second
regulatory reference document counted as 0. And the publisher's JSON writes a figure's
caption as `figcaption` under the `graphic`, not as the `fig/caption`
NISO-STS writes — 430 of them over the eight deliveries measured. No other
count moves, measured role by role over those eight and the committed twins.
@ -735,7 +735,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **The skipped-row guard measures the machine, not the argument.** Row 6 is
SKIPPED exactly when the corpora the arguments name are absent, so asking the
arguments made the branch unreachable.
- **Row 6 says when a corpus measures no element class at all.** On N200 the
- **Row 6 says when a corpus measures no element class at all.** On that second document the
build proposes 0 plans and exits 2 before the accounting door, so 16 549
elements land as unaccounted with no declared fate — a finding about the run
that read as a finding about the build.
@ -749,7 +749,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **Three sentences this release publishes are now held by tests.** PM's
checkpoint on `43331fc` found three mutants surviving the entire suite: a
normalisation door that ALSO removes U+00A0 NBSP — which would have eaten
all 6 633 of them in R761 while `log.md` went on saying "No other character
all 6 633 of them in the reference standard while `log.md` went on saying "No other character
is touched" — and row 3 of the accounting gate losing either its
`refused={n}` column or its "N element(s) lost with R of D document(s)
refused whole" clause. Each is killed now by a test that counts its own
@ -764,7 +764,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
the source bytes would disagree by construction on every type extraction
does not carry verbatim, and the gate would have to decide which difference
was a loss.
- **Two published numbers were wrong and are corrected.** N101 ships **568**
- **Two published numbers were wrong and are corrected.** A third reference document ships **568**
`tbx:` tags, not 574 — three independent counts agree (raw substring, regex
over the JSON `tag` field, node traversal), and a test now counts them over
the delivery instead of repeating the number in a fourth docstring. And the
@ -806,9 +806,9 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
a PDF was opened for its text alone, the converter's markdown writer dropped
every picture, and the only writer into a bundle took `content: str`. The two
lossiness warnings said so on every run, which made the loss honest and did
not make it smaller. Measured on R761 Prosesskoden:2025: the process text is
not make it smaller. Measured on a 701-page reference standard: the section text is
carried in full while 12 `Tabell N-N` and 9 `Figur N-N` captions stand over
nothing, so process 84's "toleranseklasse ... er gitt i tabell 84-2" points
nothing, so a section's "is given in table 84-2" points
at empty space.
**Five readers place, one module decides.** `pdf` (embedded image XObjects),
`docx`/`pptx`/`odt`/`rtf` (the converter's media, through `--extract-media`),
@ -945,7 +945,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
not the bundle. The bar rests on **one product**, and the run says so on every
boundary row. `docs/2026-09-12-g37-terskler.md` SS 7 carries the premises
re-measured, the seven bundles, the interval any bar could sit in, and the two
R761 builds this one fails.
reference-standard builds this one fails.
### Unchanged
@ -959,7 +959,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **`okf quality <bundle>` -- a per-file-type verdict, with the denominator
(G37).** `okf check` is a CONTRACT check, and a green one is not a quality
gate: measured 2026-09-10 by `vegnormal-okf`, three arms over one corpus all
gate: measured 2026-09-10 by a downstream consumer repository, three arms over one corpus all
returned 0 findings and exit 0 while their hit@k ranged from 6 of 6 to 0 of 6.
The new command asks the other question. Three verdicts and no fourth --
`PASS`, `FAIL`, `UNMEASURED` -- and a type with no measured threshold is never
@ -1014,7 +1014,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
parameter, `link_in_signal`, default `True`, no CLI flag, no payload moved:
it is the instrument that separates a ranking movement from a budget
displacement on ONE bundle, and the measurement it was built for is
`docs/2026-09-12-k3-runde23-stien-i-kroppssignalet.md`. Measured on R761
`docs/2026-09-12-k3-runde23-stien-i-kroppssignalet.md`. Measured on the reference standard
(2 761 concepts, 710 heading-only sections, 675 carrying the line): of the
**39** newly delivered concepts the line ever added a question token to,
**39** gained it from the bundle-absolute PATH and **0** from the link's
@ -1175,12 +1175,12 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
(K3-22).** SPEC § 11 point 1 requires "a parseable YAML frontmatter block" in
every file. Measured with PyYAML 6.0.3 before the change, the pinned K2
default bundle failed `safe_load` on 41 of 455 blocks (and a 42nd parsed to
a truncated title), and each R761 build on 1 -- every one a block scalar
a truncated title), and each reference-standard build on 1 -- every one a block scalar
written verbatim: a title with `": "` or `" #"`, a leading `- `, `*` or `**`,
a trailing `:`.
- **Block scalars:** a value K3-19's plain-scalar rule refuses is written
double-quoted, `\` and `"` escaped; every other value keeps its bytes.
Rebuilt, the five-document project moves 0 files, each R761 build 1 line
Rebuilt, the five-document project moves 0 files, each reference-standard build 1 line
and the K2 default bundle 42 `title` lines, after which all 454 of its
frontmatters parse and read back the same.
- **Flow leaves (`sources`, Door A's list, a run-stated flow value):** the
@ -1191,7 +1191,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
`source_reference_unquotable`, `run_frontmatter_invalid`).
- **Behaviour change:** `okf build --frontmatter 'sources=[{ resource: <URL
with a query string>, … }]'` now exits 2 and writes nothing. K3-19's own
flagged R761 build used such a URL and wrote 2 761 of 2 761 frontmatters
flagged reference-standard build used such a URL and wrote 2 761 of 2 761 frontmatters
PyYAML refuses.
- **Readers** (`parse_frontmatter`, the index and structure readers, both
`read_sources` branches) unquote a `"`-wrapped value; `'`-wrapped values
@ -1199,7 +1199,6 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
exactly what they returned before.
- The generated `SKILL.md` header goes through the same block rule.
- PyYAML joins the `dev` dependency group only; `src/` imports no yaml.
- Report: `docs/2026-09-11-k3-runde22-yaml-lesbar-frontmatter.md`.
## [0.8.3] — 2026-09-11
@ -1210,7 +1209,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
ancestor that holds text: the nearest preceding plan entry at a smaller
level, passing over an ancestor that is empty too. Nothing is copied and no
boundary moves. It reads the plan's level and order, never the row. Measured
on one process code, 710 of 2 761 concepts are heading-only; the route names
on one reference standard, 710 of 2 761 concepts are heading-only; the route names
the ancestor the document's own nesting names on 708 of them (two sit at
depth 7, which a markdown heading clips to 6), where reading section numbers
gets 686.
@ -1299,8 +1298,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
a title.
`--ingested-at` alone was confirmed to stamp every concept, on the segmented
route too, and to date `log.md`: 2 761 of 2 761. Report:
[`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`](docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md).
route too, and to date `log.md`: 2 761 of 2 761.
## [0.8.2] — 2026-09-11
@ -1426,8 +1424,8 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
section titled with that subject PLUS a qualifier scores, and then loses on
the body count. Decomposed per miss: S1 turns on `hvordan`, an interrogative
pronoun; S5 on `hvilke` and `stilles` in a body 7x the gold's, and on
`betonghvelv ~ betongkonstruksjoner` through the four-character stem
`betong`; S6 on an exact tie broken by `concept_id`.
a compound pair sharing a six-character first
element; S6 on an exact tie broken by `concept_id`.
- **A PARTITION, never a fourth RRF signal, and the arithmetic is why.** RRF
consumes ranks alone, so with shared ranks a rule whose positive group has
`m` members is worth `1/61 - 1/(61 + m)` -- a rule firing on ONE concept of
@ -1448,11 +1446,11 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
known-positive from 1 to 4 every time it does.
- **NOTHING ELSE MOVES AND IT IS MEASURED ON THE BYTES.** The pinned K2 bundle
keeps `(1,1,1,1,1,5)` and its 7 pin tests, Arm B keeps `(1,1,1,1,1,5)`, and
the payloads on both are **byte-identical on 6 of 6 questions**; n100/n200/
n500 payloads are byte-identical on 5 questions each; the 828-file HTML
the payloads on both are **byte-identical on 6 of 6 questions**; the three
downstream bundles' payloads are byte-identical on 5 questions each; the 828-file HTML
corpus still gives 828 plans, 0 unreadable and 6 015 md with `diff -rq`
empty; the five-document folder is `diff -r`-identical at 26 concepts / 52
md; `okf project` stays byte-equal to `okf build`. hit@k on n100/n200/n500
md; `okf project` stays byte-equal to `okf build`. hit@k on those three bundles
is **NOT MEASURED** -- this repository holds no gold set for them, which is
0 gold sets and not 0 hits.
- **THIS IS THE FOURTH READING-SIDE CHANGE THAT MOVES A PAYLOAD WITH NO BUNDLE
@ -1554,7 +1552,6 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
a success. Separating "0 plans, 0 unreadable" from "0 plans because nothing
could be read" would change the outcome on **0 of the 4** reference
corpora, so it is not separated.
- Report: `docs/2026-09-11-k3-runde13-xml-sts.md`.
### Fixed
@ -1569,7 +1566,7 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
- **`--pdf-outline` (OFF): cut a PDF at the boundaries its own `/Outlines`
bookmark tree declares.** Measured outside this repository on one 701-page
process code whose publisher also ships a NISO-STS structure for it: the
reference standard whose publisher also ships a NISO-STS structure for it: the
shipped default recovers **1 967 of 2 761** titled sections, **0 of its 28**
chapters, and **794 of 794** misses have their heading text present in the
extracted text -- the line was read, the boundary was never opened. The same
@ -1811,7 +1808,7 @@ subcommand to five, and one build default moved.
block form. Measured before the bump so the test discriminates rather than
merely passes, and pinned by
`tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit`.
Two rows of the gate table in `docs/okf-nokkelinventar.md` moved, not one:
Two rows of the gate table moved, not one:
the BLOCK form of `sources` now passes too, which retires G30 -- though it
changes nothing about what we emit, since our own parser is still the binding
constraint. `resource` is allowlisted only inside a `sources` entry, so
@ -1891,7 +1888,7 @@ subcommand to five, and one build default moved.
dependency, still stdlib otherwise — and a packaging test enforces that.
**The parser choice was forced by a measurement, not by preference**
(`docs/2026-08-21-g2-pdf-extraction-measurement.md`). On a real Vegnormalene
(`docs/2026-08-21-g2-pdf-extraction-measurement.md`). On a real regulatory
requirement table, `pdfplumber` keeps 4 of 4 rows with label and value on the
same line; `pypdf`, `pdfminer.six` and `pymupdf` each keep 0 of 4, emitting
all labels and then all values. A downstream reader can only re-pair those by

138
CLAUDE.md
View file

@ -51,8 +51,8 @@ one boundary rule:
ceiling is structural rather than recovered. A NISO-STS zip from a
publisher's own viewer was **110 of 110 unreadable, 0 plans, exit 2** --
`.xml` was in neither registry -- and the one xml file in it IS the whole
product: R761 Prosesskoden:2025, the document round 12 met as a 701-page
PDF, carrying 7 715 `<sec>`, **2 761 with a `<title>`**, 4 954 with a
product: a large regulatory reference document, the one round 12 met as a
701-page PDF, carrying 7 715 `<sec>`, **2 761 with a `<title>`**, 4 954 with a
`<label>` and no title, 10 `<table-wrap>`, root `<standard>`, 0 `<!DOCTYPE`.
Its `<sec>`-nesting depths over the titled sections are
28/118/500/1141/868/97/9, row for row the fasit's own. The reader is stdlib
@ -121,8 +121,7 @@ one boundary rule:
publisher reuses a section number, the same 2 the PDF arm has, and 0 is not
reachable without inventing an id; round 13's 14 such directories were false
positives of the TEXT route reading the document's own contents listing and
are gone. Report:
`docs/2026-09-10-k3-runde14-deklarert-struktur-tar-ruten.md`.
are gone.
**Since K3-19 an STS document's own identity names its directory**
(`extract.declared_identity`, read by `cli._document_prefixes` and the door):
the directory was the delivery file's stem, a UUID occurring **0 times** in
@ -131,12 +130,12 @@ one boundary rule:
used by NEITHER -- the `slug_owners` gate would refuse both with "rename
one", which a name read from inside a document cannot obey. The `sources`
title is `<doc-number>` + `<year>`, then `<title-wrap>`, then the file name:
R761's `<full>` carries a COMMA, a flow terminator, so it is never written
the reference standard's `<full>` carries a COMMA, a flow terminator, so it is never written
and never cleaned up. A titled section's `description` is its own FIRST
spec point (first `<p>` of the first DIRECT-child `sec-type="spec"`, whole),
carried by the plan entry, screened by the gate, and written only where a
YAML reader reads it verbatim (`inbox._yaml_plain`): 2 026 of 2 761 titled
sections on R761 carry a point, **1 807** are written (2 have no `<p>`, 217
sections on the reference standard carry a point, **1 807** are written (2 have no `<p>`, 217
carry `: ` and PyYAML refused exactly those frontmatters), none invented.
SS 4.1 sets no length, so the one-paragraph limit is ours. **The directory
name reached the RANKING, and K3-20 closed it in `consume`:** signal 1 read
@ -145,13 +144,11 @@ one boundary rule:
-- measured, the known-positive went rank 1 -> not delivered at the default
`k` (13 at k=50) with S1-S6 unmoved. `consume.shared_id_prefix` now keeps
the leading directories EVERY id shares out of that signal: KP rank 1 at
both `k`, S1-S6 6/6, and K2 (12 payloads), N100/N200/N500 (15) and the
both `k`, S1-S6 6/6, and K2 (12 payloads), three downstream bundles (15) and the
five-document folder (5) byte-identical, because ids that share no prefix
read exactly as before. Dropping each concept's OWN document directory
instead was measured and felled -- a K2 hit@8 row went rank 5 -> not
delivered. Reports:
`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md` and
`docs/2026-09-11-k3-runde20-delt-katalog-og-arvet-kontekst.md`. The
delivered. The
registries are COUPLED: a row in
`_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
every proposal for the type, two layers away from the extractor.
@ -200,10 +197,10 @@ one boundary rule:
read, an STS `<graphic>` was walked past, a PDF was opened for text alone,
the converter's markdown writer dropped every picture, and the only writer
into a bundle was `materialize.write_bytes(..., content: str)`. Measured on
R761 Prosesskoden:2025: the process text is carried in full while **12
a 701-page reference standard: the section text is carried in full while **12
`Tabell N-N` and 9 `Figur N-N`** captions stand over nothing, because that
publisher ships those tables as raster pictures in BOTH the PDF and the
NISO-STS delivery -- process 84's "toleranseklasse ... er gitt i tabell 84-2"
NISO-STS delivery -- a section's "is given in table 84-2"
points at empty space. **Five readers PLACE and one module DECIDES:**
`assets.py` owns what an image is (sniffed from the bytes, never from the
claimed extension), what it is called (`<sha256[:12]>-<the source's own
@ -217,7 +214,7 @@ one boundary rule:
the `source_rows` locator read back out of it; 0 of 4 K2 workbooks hold any
media, so it is a stated limit and not a loss taken. **A PDF stream that is
already a file is carried VERBATIM** (`DCTDecode`, `JPXDecode` -- 29 of
R761's 50 objects), and raw samples are encoded to PNG with stdlib `zlib`.
the reference standard's 50 objects), and raw samples are encoded to PNG with stdlib `zlib`.
Rendering the page region was the alternative and was FELLED on
determinism: a rasterised crop's bytes, and therefore the asset's
content-addressed name and the bundle's digest, would depend on the installed
@ -491,7 +488,7 @@ structured reader (D1b); until then the constraint binds what we write.
**Every value is written so a YAML reader reads it back the same (K3-22).**
SPEC § 11 point 1 requires "a parseable YAML frontmatter block", and before
K3-22 the pinned K2 default bundle failed PyYAML on 41 of 455 blocks and each
R761 build on 1 -- block scalars written verbatim. A block scalar that is not
reference-standard build on 1 -- block scalars written verbatim. A block scalar that is not
plain-safe (`profiles.yaml_block_plain`, K3-19's rule) is now written
double-quoted with `\` and `"` escaped; every other value keeps its bytes. A
FLOW leaf has no quoted form -- the pinned guard refuses any quote inside a
@ -500,8 +497,7 @@ a quote, a leading indicator) is refused by `profiles.yaml_flow_plain` with the
door's existing code, never written. Readers unquote a `"`-wrapped value only:
0 such values existed in any measured bundle, while 11 193 `'`-wrapped ones do
and stay untouched. PyYAML is a dev dependency that validates the rules in
`tests/test_yaml_frontmatter.py`; `src/` imports no yaml. Report:
`docs/2026-09-11-k3-runde22-yaml-lesbar-frontmatter.md`.
`tests/test_yaml_frontmatter.py`; `src/` imports no yaml.
**Every upstream release runs `docs/upstream-okf-upgrade-runbook.md`.** Pin the
commit, enumerate the whole `okf/` tree, **read the shipped example bundles and not
@ -819,7 +815,7 @@ and fixtures, never code.
(`propose._link_shells`), passing over an empty ancestor; the door writes
the existing `parent:` key. Nothing is copied -- a consumer's own build of
the same standard copied the inherited text in and took hit@1 6/6 -> 2/6.
The route reads the PLAN, never the row: on R761 it names the ancestor the
The route reads the PLAN, never the row: on the reference standard it names the ancestor the
`<sec>` nesting names on **710 of 710** shells since K3-21 D (708 before: the
2 misses sit at depth 7, and the reader clipped the outline mark to 6 along
with the heading), where reading section numbers gets 686 (`12` begins with
@ -859,14 +855,13 @@ and fixtures, never code.
keys `(source_file, segment_id)` off the concept's own frontmatter and is
asked first, inside the pointing concept's document; a value no segment
answers to is a number, looked up as before, and a pointer naming nothing
keeps `?`. The one key keeps its two meanings (`inbox.py`). Report:
`docs/2026-09-11-k3-runde20-delt-katalog-og-arvet-kontekst.md`.
keeps `?`. The one key keeps its two meanings (`inbox.py`).
- **A TWELFTH flag, `--pdf-outline`, is OFF** (round 12, 2026-09-10) and it is
the only one here that does not read the extracted text at all: it cuts a PDF
at the boundaries its own `/Outlines` bookmark tree declares. It is NOT Arm D
-- `--outline-run`/`--outline-gate` are a TEXT heuristic over numbered lines
in the extracted text, and this opens a structure index the file already
carries. Measured on ONE 701-page process code whose publisher also ships a
carries. Measured on ONE 701-page reference standard whose publisher also ships a
NISO-STS structure for it, so the fasit is the publisher's own 2 761 titled
sections: the shipped default finds **1 967 of 2 761**, **0 of its 28**
chapters, and **794 of 794** misses have their heading text PRESENT in the
@ -890,15 +885,14 @@ and fixtures, never code.
stands under a candidate's first line, the right question for a guess and the
wrong one for a publisher's declaration; **683 of 2 762** marks are container
sections and applying it scores 2 079. An unresolvable `/Dest` is dropped and
COUNTED (R761 has 0 of 2 763; one of the eight reference PDFs has 2 of 2). NO
COUNTED (the reference standard has 0 of 2 763; one of the eight reference PDFs has 2 of 2). NO
new dependency: `pdfminer.six` already ships under `pdfplumber` in
`[extract]`, so `uv.lock` is untouched and `pypdf` stays out. Cost 119.22 s ->
183.31 s wall, peak RSS 3 252 -> 3 251 MiB, pages parsed 1 -> 1. **The default
did not move**, and the reach is why: **1 of 8** reference PDFs carries a
usable tree, and a bookmark tree is the publisher's CLAIM about its own
structure. On the folder where no PDF has one, `diff -r` is empty against both
the arm off and the pre-change tree. Report:
`docs/2026-09-10-k3-runde12-pdf-outlines.md`.
the arm off and the pre-change tree.
- **Three PDF READER flags, all off, and they sit BEFORE every segmentation
flag** -- an arm changes how the proposer cuts a text, these change what the
text says. `--pdf-headings font` infers a heading from typography (dominant
@ -1045,7 +1039,7 @@ and fixtures, never code.
counts allocations and is not disturbed by load). Known-positive: **258 of
258** chains over the permitted filters still carry a small image.
**10 of 10 mutants killed**, control green. **NOT A ROW'S DIFFERENCE ON
REAL DOCUMENTS**: paired `_pdf_images` over every PDF on this machine from two PINNED trees: **9 306 -> 9 306** bårne over 77 filer plus **50 -> 50** on R761 (**9 356 -> 9 356** together), **0 of 78** files moving a count and **0** moving a code. R761 also settles the `[/Fl /A85]` question: **50 objects, 29 `[/DCTDecode]`, 21 `[/FlateDecode]`, 0 ASCII85 links** -- round 2's count of **580** is reproducible from nothing on this machine, which changes no decision but should not stay unqualified. **`resource.setrlimit(RLIMIT_AS)` was
REAL DOCUMENTS**: paired `_pdf_images` over every PDF on this machine from two PINNED trees: **9 306 -> 9 306** bårne over 77 filer plus **50 -> 50** on the reference standard (**9 356 -> 9 356** together), **0 of 78** files moving a count and **0** moving a code. The reference standard also settles the `[/Fl /A85]` question: **50 objects, 29 `[/DCTDecode]`, 21 `[/FlateDecode]`, 0 ASCII85 links** -- round 2's count of **580** is reproducible from nothing on this machine, which changes no decision but should not stay unqualified. **`resource.setrlimit(RLIMIT_AS)` was
MEASURED before anything was built on it and is NOT usable here** --
Darwin 26.6.2 raises `ValueError: current limit exceeds maximum limit` and
does not enforce it -- so no child-process cap exists and the per-link
@ -1072,7 +1066,7 @@ and fixtures, never code.
`docs/2026-09-18-bildestien-holder-0-10-1.md`.
- **EVERY CARRIED IMAGE IS ONE A MODEL CAN BE SHOWN (0.10.1, 2026-09-19).**
Until this round the path carried whatever a publisher shipped. Measured over
the frozen R761 delivery's own `assets/`, denominator 50: 29 JPEG, 2 PNG and
the frozen reference delivery's own `assets/`, denominator 50: 29 JPEG, 2 PNG and
**19 "PC bitmap, Windows 3.x, 8-bit, compression 1"** -- byte-correct RLE8
BMP files that nothing reads, so 19 figures were present and invisible at
once while `images: N` said they had arrived. `assets.VIEWABLE_MEDIA_TYPES`
@ -1108,9 +1102,9 @@ and fixtures, never code.
a consumer loses 16 pictures and gains 16 statements that a picture was
there. **The judge had to move with it and it caught itself**: the gate's
`asset_holds` proved a carry by hashing the SOURCE file, which a converted
image's bundle cannot satisfy, so R761 went 0 -> **19 claimed and not
image's bundle cannot satisfy, so the reference standard went 0 -> **19 claimed and not
found** (exactly its RLE8 count) before the judge learned to read the two
digests the bundle states and then hash the asset itself; after, R761 is
digests the bundle states and then hash the asset itself; after, the reference standard is
u = 0, d = 0, 0 claimed and not found, 50 asset files, 31 081 carried
elements, and the gate's verdict is the same `RED: rows 2, 3, 6` as before
the round. Report:
@ -1118,7 +1112,7 @@ and fixtures, never code.
**THAT ROUND SHIPPED TWO DEFECTS AND 0.10.1 CLOSES THEM.** A TRUNCATED RLE8
stream was carried as a partly blank PNG: the decoder painted what the
stream held and left the frame at index 0, which is what the format says
about a SKIPPED pixel, so no decoder disagreed -- measured on a real R761
about a SKIPPED pixel, so no decoder disagreed -- measured on a real reference-standard
asset of 192 896 pixels, a stream cut to 90 % was carried with 13 923 pixels
wrong, to 50 % with 95 890, to 10 % with 166 525, no code and no row. The
decode may now end at an explicit **end-of-bitmap escape and nowhere else**
@ -1154,7 +1148,7 @@ and fixtures, never code.
outside the build. A build with no accounting door reports a converted image
claimed-and-not-found rather than believing it. Measured: 3 of 3 forged ->
refused with the known-positive True in all three, the text-level regression
guard 3 arms -> 13, R761 rebuilt `diff -r`-identical, 50 assets, **19 of 19**
guard 3 arms -> 13, the reference standard rebuilt `diff -r`-identical, 50 assets, **19 of 19**
conversions confirmed against 19 declared, SHY 71, u = 0, d = 0. And the
TERMINATOR asks only that a stream say it is finished, which it may do
anywhere: a stream whose FIRST two bytes are the EOB escape was carried with
@ -1190,7 +1184,7 @@ and fixtures, never code.
**11 441** files scanned across the four raw deliveries and the K2 reference
corpus, the only **25** BMPs on this machine use an end-of-line at column 0
in **0 of 25** and a delta in **0 of 25**, and 25 of 25 still decode to
Pillow's pixels exactly (**3 117 220** compared). R761 rebuilt from a pinned
Pillow's pixels exactly (**3 117 220** compared). The reference standard rebuilt from a pinned
pre-change tree and from this one is `diff -r`-IDENTICAL, 5 551 files, 50 of
50 viewable, 19 of 19 lossless. `CURSOR_CASES` goes 8 arms to 12 (one for
the cursor rule's ROW clause, PM's `P8`, which survived 51 tests) and the
@ -1211,7 +1205,7 @@ and fixtures, never code.
- **`carried` is checked against the written concept bodies**
(alphanumerics, casefolded), so a gate that drops a line is FOUND.
Measured limit: short elements (labels, one-word titles) are often
present elsewhere in the document. With R761 cut to half, 4 823
present elsewhere in the document. With the reference standard cut to half, 4 823
paragraphs were reported lost but only 3 titles.
- **Two changes hold without the flag.**
- A build that extracted documents and persisted none exits 1. Door B and
@ -1219,8 +1213,8 @@ and fixtures, never code.
- An image file carried through a persisted document is its own K1b
column, never also `extractor_unknown`. The files come from
`ExtractedDocument.files` (what the resolver actually resolved), never
from byte equality, which would credit R761's 7 unpointed duplicates.
- **R761 WAS refused whole by guard 1.4.0 because of its 71 U+00AD soft
from byte equality, which would credit the reference standard's 7 unpointed duplicates.
- **A 701-page reference standard WAS refused whole by guard 1.4.0 because of its 71 U+00AD soft
hyphens, and since 2026-09-19 okf removes them itself.** Operator decision
2026-09-18, the chosen alternative verbatim: **one documented
normalisation door BEFORE the guard**, `extract.normalise_extracted`,
@ -1231,10 +1225,10 @@ and fixtures, never code.
for every consumer and a fix for one document. `output:zero-width-present`
is an ANY-TIER carrier, so the verdict was formally right at every trust
level and materially a false positive: the 71 are Norwegian hyphenation
points inside words (`ar[SHY]beider`, `bitu[SHY]men`), and the four real
points inside words (`ar[SHY]beider`, `sur[SHY]deig`), and the four real
zero-width characters (U+200B, U+200C, U+200D, U+FEFF) plus U+2060 are
**0 of 0** in both deliveries. Those four are NOT removed and U+00A0 NBSP
(6 633 in R761) is not either -- it is not in the guard's set at all.
(6 633 in the reference standard) is not either -- it is not in the guard's set at all.
Applied ONCE, at the end of `extract_document`, so `propose` and Door B
cannot disagree about the text a plan's `text_sha256` indexes; `_pdf_units`
measures its page offsets through the same door, because that table is
@ -1263,19 +1257,12 @@ and fixtures, never code.
move**: it belongs to the whole run, and a corpus holding one unreadable
file among many is ordinary -- so the order's other half, state it in the
accounting, is the one taken.
- **THE JSON ROLE MAP READS THE PUBLISHER'S OWN TAGS (H3, 2026-09-19).**
`count_sts_json` compared the RAW tag string where `count_sts_xml` has
always used `_local`, so N200 Vegbygging:2024's **74 `mml:math`** (2 811
characters) counted as 0; and the publisher's JSON writes a figure's
caption as `figcaption` under the `graphic`, not as the `fig/caption`
NISO-STS writes, so **49 `figcaption`** counted as 0. The `figcaption`
clause is in the JSON map ALONE -- the two maps are written twice on
purpose so row 5 can see a hole in either. NOT ONE OTHER COUNT MOVES,
measured role by role over the eight JSON deliveries on this machine and
the three committed twins: `figure_caption` 0 -> 123/127/62/49/43/27 and
`math` 0 -> 74/3/88, R761 and N601 unchanged, the two existing twins
unchanged. The known-negative for stripping a prefix is N101's **568
`tbx:`** tags, not one of whose local names is a role.
- **THE WITNESS READS NO PUBLISHER JSON (2026-09-23).** Its STS JSON
reader (`count_sts_json` and its role map, H3) existed for one
publisher's delivery and is removed with its tests and twins; a `.json`
file is counted as generic JSON, the way the package reads it. The XML
role map is now written once, so `W03`/`W07` in the mutation harness
edit its only copy and `W09` is gone.
- **The mutation harness is a GATE since 2026-09-19 (H4)**: a surviving
mutant exits 1, where `2 if errors else 0` made `killed 0 of 1` an exit 0.
PM's **X2** -- "a report may declare a document rejected while the bundle
@ -1284,18 +1271,18 @@ and fixtures, never code.
measures the DEFAULT sources and not the arguments (H5: row 6 is SKIPPED
exactly when the arguments' corpora are absent, so the branch was
unreachable), and row 6 now SAYS when a corpus measures no element class
at all (H6: on N200 the build proposes 0 plans and exits 2 before the
accounting door, so 16 549 elements land with no declared fate -- a
finding about the run that read as a finding about the build; what it
would take is an STS reader for the JSON delivery, and `standardContent`
occurs 0 times in `src/`).
at all (H6: a build that proposes 0 plans exits 2 before the accounting
door, so its elements land with no declared fate -- a finding about the
run that read as a finding about the build). Row 6 reads the corpora
given by `--corpus PATH` (repeatable: a directory, a zip or one file; no
default) and row 5 compares the PDF pair alone.
- **THE PUBLISHED DOOR CONTRACT NAMES WHAT THE GATE READS (2026-09-19).**
The JSON sketch at the top of `okf_accounting_gate.py` is what a consumer
implements `--accounting` from, and it did not name `conversions` -- which
`asset_holds`' conversion route DEPENDS on -- nor `normalised_soft_hyphen`,
`unaccounted` or `double_booked`. A door built from it writes a ledger the
gate reads as "nothing was converted", and every converted image comes out
claimed-and-not-found: **19 of 50** on R761. Two tests hold the sketch
claimed-and-not-found: **19 of 50** on the reference standard. Two tests hold the sketch
against both sides -- what the gate LOOKS UP (measured with a ledger that
records its own lookups, never by grep) and what the door SERIALISES -- so
"publish everything" is not a fix either.
@ -1349,12 +1336,11 @@ and fixtures, never code.
new fixture that delivers a passage (`DELIVERY`, set `set-passage.json`),
because no synthetic concept was long enough to be cut. `src/` untouched;
`MUTANT_BAR` unchanged.
**THE K2 TEST TRACK IS RETIRED (operator 2026-09-21, v1.1 K)**: material
tied to the operator's employer is not re-measured and not frozen, and
public tests and gates run on invented material. Row 9 (K2) and the `r761`
and `vegnormal` adapters of row 8 are gone (row 8 requires `wiki-20` alone),
and so are the tests that read the K2 bundles, the road-standard builds or
the R761/N101/N200 deliveries; `okf_accounting_gate.py` has no default real
**THE K2 TEST TRACK IS RETIRED (operator 2026-09-21, v1.1 K)**: it is not
re-measured and not frozen, and public tests and gates run on invented
material. Row 9 (K2) and the two retired-corpus adapters of row 8 are gone
(row 8 requires `wiki-20` alone), and so are the tests that read the K2
bundles or the retired reference corpus's builds and deliveries; `okf_accounting_gate.py` has no default real
corpus. The synthetic corpus was re-measured for BM25 (`MISS`, `LOOKUP`,
`QUOTA`), so rows 2 and 3 are green through their own fixtures again, and
row 7's mutants patch `bm25` where the default runs there; the three that
@ -1428,20 +1414,20 @@ and fixtures, never code.
survivors (document prior dead, `RRF_K` flattened) are reported with what
they measured: **0 ranks and 0 deliveries moved**, and a mechanism for each.
Rows 8 and 9 are never green by leaving something out: the three real sets
are read from path + sha256 (`--real wiki|r761|vegnormal`, one Question per
standard where a question cites two), and K2 is **0 of 6 because the gold
are read from path + sha256 (`--real wiki` plus two retired-corpus sets, one
Question per standard where a question cites two), and K2 is **0 of 6 because the gold
set exists nowhere** while the bundles do. Granularity is stated and the two
forms are NEVER summed: a citation-level set and a concept-level set are two
numbers. Nevnerne re-measured against the sources 2026-09-19 and they hold:
R761 **8** (S1-S6 + KP + KN), vegnormal **32** questions / **43**
`must_cite` / 32 `must_contain`, K2 **6**. The fourth set's own figures are
the first retired set **8** (S1-S6 + KP + KN), the second **32** questions /
**43** `must_cite` / 32 `must_contain`, K2 **6**. The fourth set's own figures are
a consumer's and are not restated here; the gate reads it by path and
sha256 like the others. PM's mixed sum over two sets at two granularities
is carried labelled as recorded, never as the gate's own.
- **Judge a bundle: `okf quality <bundle>` (G37, 2026-09-12).** A per-file-type
verdict, with the denominator on every line, and it is a SEPARATE command
from `okf check` on purpose: `check` is the contract check, and a green one
is not a quality gate -- measured 10.09 by `vegnormal-okf`, three arms over
is not a quality gate -- measured 10.09 by a downstream consumer repository, three arms over
one corpus all gave 0 findings and exit 0 while hit@k ranged 6 of 6 to 0 of
6. Three verdicts and no fourth (`PASS` / `FAIL` / `UNMEASURED`), a type with
no measured threshold is **never** `PASS`, and exit **3** exists for
@ -1465,7 +1451,7 @@ R761 **8** (S1-S6 + KP + KN), vegnormal **32** questions / **43**
corpus's**: a rejected document leaves no row at all (the pinned corpus holds
33 PDFs, the bundle shows 32), so the run log is printed beside the counts
and a bundle without one says so. Three of the four evidence corpora
(`n100`/`n200`/`n500`) carry `source_file` on **0 of 446 / 0 of 1 133 / 0 of
(three downstream bundles) carry `source_file` on **0 of 446 / 0 of 1 133 / 0 of
270** concepts, so they name no file type and every row is `UNMEASURED` --
the order expected them to PASS. Thresholds, the nine bundles and the
premises re-measured: `docs/2026-09-12-g37-terskler.md`. README publishes the
@ -1479,16 +1465,16 @@ R761 **8** (S1-S6 + KP + KN), vegnormal **32** questions / **43**
**The normalisation was derived before the metric was built**, not guessed:
strip all whitespace, lowercase, reproduces the fasit's own `norm` from its
own `title` on **2 761 of 2 761** rows (alphanumerics-only scores 58 -- it
eats the `.` in `2.1Hovedprosesser`). **A boundary is recovered in EITHER of
eats the `.` in `2.1Hovedkapitler`). **A boundary is recovered in EITHER of
two forms and neither is a fallback**: the concept's normalised title equal to
`norm`, or the pair `(concept's own directory, residual title)` -- the literal
form wants the declared title WITH its numbering token, the pair form WITHOUT,
and no bundle can offer both, because okf's default route moves that token
into the concept id. Measured on the known-good arm: literal **22 of 2 761**,
paired **2 737**, either **2 759 (99.9 %)**; on `r761-2025-d1` the split is
paired **2 737**, either **2 759 (99.9 %)**; on an earlier build of the same document the split is
exactly inverted (2 727 literal, 0 paired), so a gate scoring one form alone
reports a 99.9 % arm as 0.8 % and calls it a segmentation defect. The two
forms are `vegnormal-okf`'s M8 correction, which they took verbatim from THIS
forms are a downstream consumer repository's M8 correction, which it took verbatim from THIS
repository's round-14 report -- the instrument reproduces both so the two
repos cannot silently measure different things. One bar, at the pinned
artifact's own value: **2 759/2 761 with `corpora = 1`**, and P2 is in the
@ -1496,12 +1482,12 @@ R761 **8** (S1-S6 + KP + KN), vegnormal **32** questions / **43**
The known-bad arm is **1 148 of 2 761 (41.6 %)**, now `FAIL` + exit 1 where
the bundle-only gate gave exit 3. **`--fasit` is an ASSERTION** (the posture
`okf consume --ref` has) that this bundle is a build of the document the fasit
describes: the K2 reference and `n100-2023` both score **0 of 2 761** and read
describes: the K2 reference and a downstream bundle of another document both score **0 of 2 761** and read
`FAIL`, which is the assertion being wrong and not the bundle -- a gate
telling those apart would need a bar read off the bundles it judges. The bar
is TIGHT and the cost is published: 2 of 4 R761 builds fall under it (2 752
is TIGHT and the cost is published: 2 of 4 builds of that document fall under it (2 752
and 2 727 of 2 761), while **any bar between 41.6 % and 98.8 % separates the
known-bad arm from every R761 build measured** -- the shipped one is the only
known-bad arm from every build of that document measured** -- the shipped one is the only
point in that interval read off a pinned artifact. An unreadable fasit exits
**2** with its reason, never a quiet `UNMEASURED`, and a fasit under five rows
is `UNMEASURED` (`MIN_DECLARED_FOR_A_THRESHOLD`, the document floor in the
@ -1785,7 +1771,7 @@ R761 **8** (S1-S6 + KP + KN), vegnormal **32** questions / **43**
threshold on length or coverage separates them. The fourth candidate does:
the shared prefix must be a WORD the bundle uses. `bilateral` 400 -> 0 and
512 -> 0, `standhaftig` 219 -> 56 and 235 -> 33, every hit@8 row keeping rank
1 on BOTH bundles. `undersjoisk` stops at **162** because `under` IS a word
1 on BOTH bundles. `under…` stops at **162** because `under` IS a word
here -- a genuine Norwegian morpheme, so that residual is a different answer,
never a ceiling. The vocabulary is the BUNDLE's own, so the rule makes a
payload corpus-dependent the way `rarity_weights` already is.
@ -1839,11 +1825,11 @@ R761 **8** (S1-S6 + KP + KN), vegnormal **32** questions / **43**
TIE-BREAK
rather than a weight: RRF ranks every concept in every signal, including a
signal that scored them all the same, and the declared `(-score, concept_id)`
tie-break then orders that group by id. Measured on N500, whose document
tie-break then orders that group by id. Measured on a downstream bundle, whose document
prior has **two** distinct values over 270 concepts, that signal contributed
alphabetical UUID order and put a concept answering 7 of 7 question tokens at
fused rank 14 — outside the cut — behind concepts sharing only `tunnel` and
`vann`. Under shared ranks it is rank 3 and 2 of the 16 covering concepts are
fused rank 14 — outside the cut — behind concepts sharing only two common
words. Under shared ranks it is rank 3 and 2 of the 16 covering concepts are
delivered. **It shipped OFF on a measurement that was CONDITIONAL and stopped
being true in a commit reported as changing nothing.** The published cost —
hit@8 falling 5 of 6 to 4 of 6 — is real only at `DOCUMENT_PRIOR_EXPONENT`

46
README.md
View file

@ -159,7 +159,7 @@ that test red.
| `.html` | `_extract_html` | core | measured | Block tags open their own lines and `h1`–`h6` carry the ATX marker for their own level. It does **not** go through the converter, although the converter reads HTML: that route would add CVE-2025-51591 (SSRF via an iframe in HTML input) and buy nothing. The denominator is 828 files — one product, one format, one publisher. |
| `.htm` | `_extract_html` | core | stdlib, no corpus class | The same reader as `.html`. The 828-file class is recorded for `.html` alone, and this row does not borrow it. |
| `.xml` | `_extract_xml` | core | measured | A NISO-STS document (`<standard>` root, or any `<sec>`) becomes one heading per titled section at the section's own nesting depth; any other XML keeps its text in document order and gets no invented structure. A `<!DOCTYPE` is refused unparsed. The denominator is one file, one publisher, one schema — 2 761 titled sections. |
| `.pdf` | `_extract_pdf` | `[extract]`: pdfplumber | measured | Eight corpus documents with a hand-counted fasit, plus a 701-page process code whose publisher also ships its structure. Prose only: drawn content has no text to recover, and every extraction warns. OCR lives here as a reading mode for a PDF page whose own text never arrived (`--ocr`, `OCR_CID_SHARE`), never as an entry for image files. |
| `.pdf` | `_extract_pdf` | `[extract]`: pdfplumber | measured | Eight corpus documents with a hand-counted fasit, plus a 701-page reference standard whose publisher also ships its structure. Prose only: drawn content has no text to recover, and every extraction warns. OCR lives here as a reading mode for a PDF page whose own text never arrived (`--ocr`, `OCR_CID_SHARE`), never as an entry for image files. |
| `.docx` | `_extract_office` | `[extract]`: pypandoc-binary | measured | Five corpus documents. `source_lines` index the extracted text and not the original's paragraphs — the two counts agree on none of the five. |
| `.xlsx` | `_extract_office` | `[extract]`: pypandoc-binary | measured | Written as pipe tables, one source row per line; a sheet name becomes a heading, and a row is located by `source_sheet` and `source_rows`. |
| `.pptx` | `_extract_office` | `[extract]`: pypandoc-binary | constructed | N = 2 decks. 2 of 2 slide titles recovered on a deck that declares them, 0 of 2 on a deck that does not, where the converter writes `Slide 1` / `Slide 2` because it has no title to use. |
@ -314,7 +314,7 @@ who meets one in an older document knows what they are looking at.
measured bundle's payload changed one byte.
- `v0.8.0` — `.xml` is a core file type, read as NISO-STS through the stdlib
parser, and a section the source DECLARES takes the
declared-structure route — one publisher's process code segments at 2 761 of
declared-structure route — one publisher's reference standard segments at 2 761 of
2 761 of its own declared sections at the shipped defaults. No other file
type changes one byte, measured on the bytes.
- `v0.7.0` — `okf project` builds the bundle `okf build` builds (they were one
@ -365,8 +365,8 @@ at the bundle root, named `<sha256[:12]>-<the source's own base name>`, and the
concept carries a two-line pointer where the picture stood:
```markdown
![Tabell 84-2 Toleranseklasser](/assets/e54e5f5da0e8-tabell-84-2.png)
Image: graphics/tabell-84-2.png (120x90 px) -- Tabell 84-2 Toleranseklasser
![Tabell 84-2 Hevetidsklasser](/assets/e54e5f5da0e8-tabell-84-2.png)
Image: graphics/tabell-84-2.png (120x90 px) -- Tabell 84-2 Hevetidsklasser
```
The first line is markdown, so a reader that renders the concept sees the
@ -374,11 +374,11 @@ picture. The second states what the first cannot — the name the SOURCE gave th
file and the size in pixels — which are the two facts a person checking the
bundle against the original needs.
**Why it exists.** Measured on R761 Prosesskoden:2025, a process code published
as a 701-page PDF and as a NISO-STS delivery: the process text is carried in
**Why it exists.** Measured on a 701-page reference standard, published as a
PDF and as a NISO-STS delivery: the process text is carried in
full, and 12 `Tabell N-N` and 9 `Figur N-N` captions stand over nothing,
because the publisher ships those tables as raster images in both deliveries.
Process 84 says "toleranseklasse ... er gitt i tabell 84-2" and table 84-2 is a
One section says a class "is given in table 84-2" and table 84-2 is a
JPEG. A bundle like that reads as complete and is not.
**What it costs, measured on the 43-document reference corpus** (`K2/trinn1`,
@ -410,7 +410,7 @@ beside it passes nothing. `log.md` says so on every run that carries one.
**Every carried image is one a model can be SHOWN.** A bundle that holds a
picture nothing can read is worse than one that says the picture is missing:
the count reports that it arrived. Measured over the frozen R761 delivery's own
the count reports that it arrived. Measured over the frozen reference standard's own
`assets/` (denominator 50): 29 JPEG, 2 PNG and **19 RLE8 BMP** — correct files
that no model decodes. Every asset's type is read off its bytes and tested
against the viewable set; a BMP is converted losslessly to PNG (8-bit
@ -427,7 +427,7 @@ counts: images are carried on the CORE path, where `.html` and `.xml` need no
an installed library would make a bundle's identity move with that library's
version, which is the property page rasterisation was felled over. Pillow is
the independent decoder in the tests instead, and against it **19 of 19** of
R761's real RLE8 assets convert with identical RGB, 2 366 365 pixels compared.
the reference standard's real RLE8 assets convert with identical RGB, 2 366 365 pixels compared.
A converted asset is ONE asset: one file in `assets/`, one pointer, one row in
the accounting. The pointer's second line — where the source's own file name
@ -450,7 +450,7 @@ move.
`MAX_IMAGE_PIXELS` (40 000 000 pixels) or `MAX_IMAGE_BYTES` (256 MiB) is
refused with `asset_too_large`, counted like every other refusal. The largest
image in the 43-document reference corpus is 4 515 x 4 128 (18.6 MP) and the
largest of R761's 109 pictures is 2 072 x 656 (1.4 MP), so the bound is an
largest of the reference standard's 109 pictures is 2 072 x 656 (1.4 MP), so the bound is an
order of magnitude above anything measured.
It exists because a few kilobytes can declare an enormous picture: a 9.6 KB
@ -605,10 +605,10 @@ u and d both stay 0 and nothing else could see the loss.
**The soft hyphen is removed before the persist gate, and counted** (operator
decision 2026-09-18). U+00AD is in `llm-ingestion-guard`'s zero-width set, and
`output:zero-width-present` is an any-tier carrier: a document carrying one is
`fail_secure` at every trust level. Measured on R761 Prosesskoden:2025 — 71
`fail_secure` at every trust level. Measured on a 701-page reference standard — 71
U+00AD, and 0 of U+200B, U+200C, U+200D, U+FEFF and U+2060 — those 71 are
Norwegian hyphenation points inside words (`ar[SHY]beider`, `bitu[SHY]men`),
so a 701-page process code was unreadable for the whole chain over typography.
Norwegian hyphenation points inside words (`ar[SHY]beider`, `sur[SHY]deig`),
so a 701-page reference standard was unreadable for the whole chain over typography.
`extract.normalise_extracted` removes that one character from every extracted
text and reports the count as `normalised_soft_hyphen`, per document and for
the run, in the accounting JSON and in a `**Normalisation**` bullet in
@ -641,7 +641,7 @@ concept whose body is its heading alone a `parent:` naming the `segment_id` of
the nearest ancestor that holds text — the nearest preceding plan entry at a
smaller level, passing over an ancestor that is empty too. It copies no text
and moves no boundary. It exists for a document that states its points once and
lets every nested section inherit them: measured on one process code, **710 of
lets every nested section inherit them: measured on one reference standard, **710 of
2 761** concepts are heading-only, and the plan's level and order name the same
ancestor as the document's own nesting on **710 of 710** since K3-21 (708
before: the two others sit at depth 7, and the reader clipped their level to 6
@ -700,7 +700,7 @@ bookmark destination is the contents listing rather than a second copy of the
body, so a contents entry and the section it lists stop landing as two concepts
under one id.
The measurement is one 701-page process code whose publisher also ships a
The measurement is one 701-page reference standard whose publisher also ships a
NISO-STS structure for it, so the fasit is the publisher's own. Under the
shipped default that document gives 1967 of 2761 boundaries, none of its 28
chapters, and 794 of 794 misses have their heading text present in the text the
@ -753,7 +753,7 @@ rank was measured in the same session and did NOT ship; see
| `--outline-gate` (**on** by default since 2026-09-09; opt out with `--no-outline-gate`) | admit `--outline-run`'s RECOVERED headings only where the document declares none of its own, plus any one recovered heading whose span covers `OUTLINE_SHARE` (0.20) of the text. Applied at admission, before spans are closed, so the text a removed mark opened is carried by the mark above it rather than lost | on the 12-document sample: `pdf` **2 of 8 → 5 of 8** alone and **7 of 8** with the rule above, `docx` unchanged at **3 of 3**. Whole corpus: it fires on **25 of 39** readable documents, changes the plan in **15 of 39**, and removes **64 of 485** proposed entries. No plan disappears (32 → 32) |
| `--first-span-from-zero` (**on** by default since 2026-09-10; opt out with `--no-first-span-from-zero`) | start the first concept at character 0, so the text above it belongs to a segment instead of to none. Adds no boundary and removes none | Measured over the 39-document corpus, the default before this rule left **207 435 characters — 11.92 %** — in no segment at all: **163 804 above the first entry** (in **32 of the 32** documents that get a plan), 26 041 *between* entries and 17 590 after the last. This rule closes the first part entirely, 79 % of the whole, leaving **43 631 characters (2.51 %) over 8 of 32 documents** with two named mechanisms of their own. It adds no boundary and the K2 concept count is identical with and without it (**425 = 425**); on the 12-position reference it changes **not one cell**, and hit@8 on a K2 bundle built with it holds **5 of 6 at ranks 1,1,1,1,1,–** under both tie-breaks |
| `--close-span-gaps` (**on** by default since 2026-09-11; opt out with `--no-close-span-gaps`) | close a concept's span against the next SURVIVING concept, and the last against the end of the text. Three steps remove a candidate AFTER its neighbour's span was already closed against it — the orphan check, and `fold_units` clause 1 both between entries and on the last run — and the removed mark's text then belongs to no segment. Adds no boundary and removes none; only spans' ends move | It closes the whole remainder the rule above left: **43 631 characters, 2.51 % of the corpus over 8 of the 32 documents with a plan, to 0** — both the 26 041 between entries and the 17 590 after the last. Decomposed: orphan check **18 527** over 15 of 39 documents, clause 1 **7 514** between entries, clause 1 on the last run **all 17 590** of the tail (with `unit_fold=False` the corpus tail gap is 0). The entry count is identical (**429 = 429** on the corpus, **436 = 436** concepts on K2, **52 = 52** md on a five-document folder); on the 12-position reference it changes **not one cell** (11 of 12 under `|F|`[3]=12, 10 of 12 under `|F|`[3]=11), and hit@8 holds **5 of 6 at ranks 1,1,1,1,1,–** on the new bundle, the previous default and Arm B alike |
| `--pdf-outline` (**off**; opt out is the default, opt in with the flag) | cut a PDF at the boundaries its own `/Outlines` bookmark tree declares, instead of at the ones the text rules recover. A SEGMENTATION arm, not a reader option: the extracted text is byte for byte the same either way, and a PDF that carries no tree builds byte-identically with the flag on. The title comes from the BOOKMARK, so it is not cut short at the page's line break, and a page before the first bookmark destination is the contents listing rather than a second copy of the body | one 701-page process code whose publisher also ships a NISO-STS structure for it, so the fasit is the publisher's own: boundaries **1967 of 2761 → 2759 of 2761**, chapter level **0 of 28 → 28 of 28**, concept titles identical to the source title after normalisation **2761 of 2761**, false positives **163 of 2182 → 3 of 2762**, directories carrying two concept files **132 of 2050 → 2 of 2738**, contents-copy pairs **65 → 0**. Consumption on the same eight questions: fasit present in the bundle **4 of 7 → 7 of 7**, hit@1/8/50 **1/6 · 2/6 · 4/6 → 3/6 · 5/6 · 6/6**. Cost 119.22 s → 183.31 s wall, peak RSS 3252 → 3251 MiB, no new dependency and no second parse of the pages. **Off, and the reach is why:** **1 of the 8** reference PDFs carries a usable tree at all, and a bookmark tree is the publisher's CLAIM about its own structure — a stale or wrongly pointing one carries that error straight into the segmentation |
| `--pdf-outline` (**off**; opt out is the default, opt in with the flag) | cut a PDF at the boundaries its own `/Outlines` bookmark tree declares, instead of at the ones the text rules recover. A SEGMENTATION arm, not a reader option: the extracted text is byte for byte the same either way, and a PDF that carries no tree builds byte-identically with the flag on. The title comes from the BOOKMARK, so it is not cut short at the page's line break, and a page before the first bookmark destination is the contents listing rather than a second copy of the body | one 701-page reference standard whose publisher also ships a NISO-STS structure for it, so the fasit is the publisher's own: boundaries **1967 of 2761 → 2759 of 2761**, chapter level **0 of 28 → 28 of 28**, concept titles identical to the source title after normalisation **2761 of 2761**, false positives **163 of 2182 → 3 of 2762**, directories carrying two concept files **132 of 2050 → 2 of 2738**, contents-copy pairs **65 → 0**. Consumption on the same eight questions: fasit present in the bundle **4 of 7 → 7 of 7**, hit@1/8/50 **1/6 · 2/6 · 4/6 → 3/6 · 5/6 · 6/6**. Cost 119.22 s → 183.31 s wall, peak RSS 3252 → 3251 MiB, no new dependency and no second parse of the pages. **Off, and the reach is why:** **1 of the 8** reference PDFs carries a usable tree at all, and a bookmark tree is the publisher's CLAIM about its own structure — a stale or wrongly pointing one carries that error straight into the segmentation |
They compose, and the order above is the order they apply in. Measured on a
five-document tender folder (2 `pdf`, 2 `docx`, 1 `xlsx`), concepts per
@ -907,7 +907,7 @@ with no bundle changing. It caps how many DELIVERED places one source document
may take, cutting where the shortlist is cut so the freed place goes to the next
candidate and `k` is still delivered in full. The defect it repairs was measured
outside this repository on a 3206-concept bundle of a published handbook: the
code's own process overview contributes **28 of 3206 concepts (0.87 %)** and
handbook's own overview section contributes **28 of 3206 concepts (0.87 %)** and
**8.0 % of the source characters**, and took **8 of 8** delivered places on one
question and **7 of 8** on the known-positive, which was not delivered at all.
Identical at 343 and 1651 concepts, so the cause is the corpus's COMPOSITION —
@ -956,7 +956,7 @@ the whole row on Arm B, because row 1's token `prisene` reaches its gold
document through `pris|sammenstilling` on four characters. The rule that works
asks whether the shared prefix is a WORD the bundle uses: `bilateral` 400 → 0
and 512 → 0, `standhaftig` 219 → 56 and 235 → 33, **every hit@8 row keeping
rank 1 on both bundles**. `undersjøisk` stops at 162 because `under` is a word
rank 1 on both bundles**. `under…` stops at 162 because `under` is a word
here — a genuine Norwegian morpheme, so the residual is a different answer and
not a ceiling.
@ -1211,9 +1211,8 @@ finding rather than a defect in the gate:
| 8 | the real set, from path + sha256 | 0 of 1 set, NOT RUN without `--real` |
**The public rows run on invented material only** (operator decision
2026-09-21). The test track built on material tied to the operator's employer
is retired rather than re-measured or frozen: its row 9 is gone, row 8 no
longer reads the two sets built on it, and no test or gate here needs that
2026-09-21). A retired test track is not re-measured or frozen: its row 9 is
gone, row 8 no longer reads the two sets built on it, and no test or gate here needs that
material to run.
Rows 3 and 4 were this gate's two findings and both are closed, which is what
@ -1520,8 +1519,7 @@ bundle:
direct-child `<sec sec-type="spec">`, whole — and a section with none gets
no `description` at all; nothing is derived from the title. A point a YAML
reader could not read verbatim (a `: ` inside it) is left out rather than
quoted or cleaned up. Measurements:
[`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`](docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md).
quoted or cleaned up.
The drop directory is walked **recursively**, in sorted relative-path order:
a file at any depth is ingested and records its path relative to the inbox
root as its `source_file`, while dot-directories and a bundle directory
@ -1666,7 +1664,7 @@ writes them into the concept's own frontmatter, and carries them into the index
entry — so a consumer can reason over the bundle rather than only look things
up in it. Every inferred field is named in a `derived` list, because an
unmarked heuristic is worse than no heuristic: the consumer cannot know when to
doubt it. A pointer to a document not dropped yet is rendered `N200?` rather
doubt it. A pointer to a document not dropped yet is rendered `Q200?` rather
than omitted, since a bundle is built up over several drops and an absence that
leaves no trace is the dangerous kind. Carrying the metadata costs index
characters — roughly 3x to 6x the flat index, depending on how many facets the

81
docs/2026-08-21-g2-pdf-extraction-measurement.md
View file

@ -1,4 +1,4 @@
# G2 measurement: one Vegnormalene PDF page through the extraction registry
# G2 measurement: one handbook PDF page through the extraction registry
**Date:** 2026-08-21 · **Order:** `20260821T170054Z-486638087-from-.claude` (from `.claude`)
**Question this document answers:** is a PDF parser behind the `[extract]` extra a small,
@ -26,7 +26,7 @@ to unblock the trial stone.**
**45 of 196** detected table objects are structurally clean enough to hand to
`render_table` unchanged.
**Recommendation: take branch 1, decline branch 2 for now.** Vegnormalene enter the chain as
**Recommendation: take branch 1, decline branch 2 for now.** The handbook enters the chain as
prose with correctly paired table lines, which is what the trial stone needs to reach leg 3.
Branch 2 is a separate decision that nothing currently waits on.
@ -38,13 +38,12 @@ correctness hazard, not a cosmetic one. See [Table pairing](#table-pairing-the-d
| Item | Value |
|---|---|
| Source | Håndbok N200 Vegbygging (juli 2018), Statens vegvesen, public PDF |
| URL | `https://www.vegvesen.no/globalassets/fag/handboker/hb-n200-vegbygging-juli-2018.pdf` |
| Book bytes | 9 879 066 · sha256 `aec0825c305cc830f4e542062c51ec8e4499769495b7d39cb36e48079fe0fdb4` |
| Source | a public Norwegian technical handbook, one edition, PDF |
| Book bytes | 9 879 066 |
| Book pages | 308 |
| Sample page | page index 150 (printed page 151), ch. 5 "Dimensjonering av vegoverbygning" |
| Sample bytes | 316 356 · sha256 `95e1d28b42ec3f514a79e0ad568d91a7122530fba82828c05b859e324e0884ac` |
| Why this page | running prose + one numbered requirement table (`Tabell 524.1 Krav til helning på utkilingen`) + one real figure (`Figur 524.1`) + `skal`-requirements |
| Sample page | page index 150 (printed page 151), in a chapter on dimensioning |
| Sample bytes | 316 356 |
| Why this page | running prose + one numbered requirement table (`Tabell 524.1`) + one real figure (`Figur 524.1`) + `skal`-requirements |
The PDF lives under a scratch path, not in this repo — it is test material, not content.
Everything below ran on Python 3.14.0 in throwaway virtualenvs; nothing was added to this
@ -56,11 +55,11 @@ repo's environment or its dependency set.
the negative result is measured rather than assumed:
```
n200-page.pdf: RAISED ExtractionError code='extractor_extra_missing' msg=extracting '.pdf'
handbook-page.pdf: RAISED ExtractionError code='extractor_extra_missing' msg=extracting '.pdf'
requires the optional 'extract' extra (pip install 'llm-ingestion-okf[extract]');
it is not installed
control.md: OK 48 chars -> '# Kontroll\n\nDette er en kjent-positiv kontroll.\n'
control.csv: OK 54 chars -> '| krav | verdi |\n| --- | --- |\n| N200 5.4.1 | fritt |\n'
control.csv: OK 54 chars -> '| krav | verdi |\n| --- | --- |\n| Q200 5.4.1 | fritt |\n'
```
The probe can succeed; the `.pdf` rejection is real and typed, not a broken probe.
@ -73,7 +72,7 @@ real guard adapter:
```
persisted : [PersistedFile(source_file='kontroll.md',
path=.../bundle/inbox-kontroll.md, reasons=('clean: no findings',))]
failed : [FailedFile(source_file='n200-page.pdf',
failed : [FailedFile(source_file='handbook-page.pdf',
error=ExtractionError("extracting '.pdf' requires the optional 'extract' extra ..."))]
rejected : () quarantined: ()
```
@ -113,38 +112,10 @@ change the verdict, but they do mean `[extract]` is three parsers, not one.
## What each candidate does to this page
`pdfplumber.extract_text()`, first ~30 lines, verbatim:
```
Kapittel 5 – Dimensjonering av vegoverbygning
Utkiling
Utkiling i vegens lengderetning (overganger i undergrunn)
For ikke frostsikret veg skal det bygges en utkiling med ikke telefarlige materialer eller med
isolasjonsmaterialer for å unngå ujevne telehiv ved overgang mellom materialer med forskjellig
telefarlighet. Utkilingen går opp til underkant av forsterkningslaget. For frostsikret veg er det ikke
behov for utkilinger.
Isolasjonsmaterialene skal ligge drenert. Figur 524.1 viser prinsippskisser for utkiling med ikke
telefarlige materialer og isolasjonsplater av XPS. Også skumglass og lettklinker kan benyttes til
utkiling. Utkilingslengde L skal beregnes fra tabell 524.1 hvor frostsikringsdybden ved frostmengde
F danner utgangspunktet for beregning av utkilingslengden. Ved bruk av isolasjonsmaterialer skal
10
det benyttes et nedre frostsikringslag med tykkelse som gitt i tabell 521.2.
Utkilingslengder gjelder både for nyanlegg og utbedring av eksisterende veg.
Tabell 524.1 Krav til helning på utkilingen
Skiltet hastighet, km/t Helning på utkilingen, maks
≤ 50 og g/s – veger 1:10
60 og 70 1:15
80 og 90 1: 25
≥ 100 Frostsikret veg forutsettes
Figur 524.1 Utkiling ved overgang bergskjæring/telefarlig grunn eller underbygning
For å redusere fuktopptak i isolasjonsplater av XPS skal tykkelsen være minimum 5 cm. For å unngå å
bruke tynnere isolasjonsplater i avslutningen av en utkiling bør det vurderes å legge isolasjonsplatene
med økende mellomrom for gradvis å redusere isoleringsevnen.
Ved overgang mellom skjæring og fylling i telefarlig jord, bør utkilingen utføres med det materialet
fyllingen er bygget opp av, se figur 524.2. Største utkilingsdybde skal beregnes etter reglene i kapittel
521.
Håndbok N200 Vegbygging (juli 2018) 151
```
`pdfplumber.extract_text()` gave about 30 lines, not reproduced here (they are the
source's own text): a chapter heading, a section heading, four paragraphs of
`skal`-requirements, the caption and four rows of `Tabell 524.1` -- each label with its
value on one line -- a figure caption, two more paragraphs and the running footer.
That is readable Norwegian technical prose, in reading order, with the table's rows intact as
lines. Two fidelity defects are visible and neither is fatal: the subscript in `F10` breaks
@ -157,7 +128,7 @@ Character counts on the same page: `pypdf` 1819, `pdfminer.six` 1834, `pdfplumbe
### Table pairing: the decisive number
`Tabell 524.1` has four requirement rows (speed class → maximum slope). Counting how many
`Tabell 524.1` has four requirement rows (a class → a maximum value). Counting how many
survive with label and value on the same line:
| Parser | Paired rows |
@ -169,7 +140,7 @@ survive with label and value on the same line:
The three zero-scorers emit all four labels, then all four values, as separate blocks. A
downstream reader — human or model — can only re-pair them by guessing. In a requirements
document, `80 og 90 → 1:25` becoming `80 og 90 → 1:10` is a wrong answer that looks right.
document, a row's value turning into its neighbour's value is a wrong answer that looks right.
This is why the cheapest candidate is not the correct one.
### What happens to the table as *structure*
@ -179,9 +150,9 @@ same thing for this table:
```
pdfplumber.extract_tables() -> 1 table, 2 rows x 6 cols
['', 'Skiltet hastighet, km/t', '', '', 'Helning på utkilingen, maks', '']
['≤ 50 og g/s – veger\n60 og 70\n80 og 90\n≥ 100', None, None,
'1:10\n1:15\n1: 25\nFrostsikret veg forutsettes', None, None]
['', '<label header>', '', '', '<value header>', '']
['<label 1>\n<label 2>\n<label 3>\n<label 4>', None, None,
'<value 1>\n<value 2>\n<value 3>\n<value 4>', None, None]
pymupdf.find_tables() -> 1 table, 2 rows x 6 cols (byte-identical shape)
```
@ -189,9 +160,9 @@ pymupdf.find_tables() -> 1 table, 2 rows x 6 cols (byte-identical shape)
Four logical rows collapse into one cell each side; three of six columns are empty padding.
`render_table` would faithfully render a two-row table that misrepresents the requirement.
Two independent implementations agreeing on the same wrong shape is the evidence that this is
the document's geometry, not a library bug — N200 draws ruling lines around blocks, not rows.
the document's geometry, not a library bug — the handbook draws ruling lines around blocks, not rows.
**So: Vegnormalene are usable as prose, and are not usable as structured concept tables
**So: handbooks of this kind are usable as prose, and are not usable as structured concept tables
without a layer this repo does not have.**
## Determinism
@ -228,9 +199,9 @@ Whole-book run, all 308 pages:
rectangular, no `None` cell, no cell containing a newline. Anything failing that would either
crash or silently misrepresent if handed to `render_table`.
Scope of these denominators: **one handbook, one edition (N200, July 2018)**. They are not a
claim about the N-series as a whole. Nothing here was measured on a scanned or image-only
PDF; every page of this book carries a text layer, and a scanned normal would score 0 and
Scope of these denominators: **one handbook, one edition**. They are not a
claim about its series as a whole. Nothing here was measured on a scanned or image-only
PDF; every page of this book carries a text layer, and a scanned handbook would score 0 and
need OCR — a different project entirely.
## Size of the wiring
@ -256,7 +227,7 @@ promise, which fixtures, and what happens to figures and tables that do not surv
| # | Claim | Status | Evidence |
|---|---|---|---|
| 1 | The N200 PDF is public and downloadable from vegvesen.no | VERIFIED | `curl -L -w '%{http_code} %{content_type} %{size_download}'` → `200 application/pdf 9879066`; magic bytes `%PDF-1.6` |
| 1 | The handbook PDF is public and downloadable from its publisher | VERIFIED | `curl -L -w '%{http_code} %{content_type} %{size_download}'` → `200 application/pdf 9879066`; magic bytes `%PDF-1.6` |
| 2 | Book has 308 pages; sample is index 150 | VERIFIED | `pypdf.PdfReader(...).pages` length; `makepage.py` |
| 3 | `.pdf` fails with `extractor_extra_missing`, controls pass in the same call | VERIFIED | `probe_registry.py` output, quoted above |
| 4 | Full Door B path reports the PDF as `failed` and persists the control | VERIFIED | `probe_inbox.py` output, quoted above |
@ -271,4 +242,4 @@ promise, which fixtures, and what happens to figures and tables that do not surv
| 13 | Gate seam is two source lines plus two test files | VERIFIED | `grep -rn '_OPTIONAL_EXTENSIONS\|extractor_extra_missing' src/ tests/` |
| 14 | Cross-version output stability is not guaranteed | ASSUMED | inferred from `pdfminer.six` date-stamped versioning; not measured across versions |
| 15 | Figures are lost because they are vector drawings | VERIFIED (this page) | caption present, no figure text in any parser's output; not generalised to the book |
| 16 | Denominators generalise beyond N200:2018 | NOT CLAIMED | one handbook, one edition measured |
| 16 | Denominators generalise beyond this handbook | NOT CLAIMED | one handbook, one edition measured |

4
docs/2026-08-29-konverteringsarmen-arm-a.md
View file

@ -11,7 +11,7 @@ installed as a **measuring tool in an isolated venv**, never into
so `docling` ran with `--no-ocr`.
This is the arm that was never run in `okf-bundle-prosessen.md` § 9.5, because
the vegnormal PDFs were rastered 11/11. This corpus is not.
a retired reference corpus's PDFs were rastered 11/11. This corpus is not.
## 0. Headline
@ -185,7 +185,7 @@ number that says the Excel half can go through Door A.
**`markitdown` destroys row pairing in PDFs, and the mechanism is visible.**
It uses `pdfminer.six`, which emits each cell on its own line — label and value
never share one. This repo measured the same failure independently on 2026-08-21
(`pdfminer.six` scored 0/4 on the N200 requirement table). Two corpora, same
(`pdfminer.six` scored 0/4 on a requirement table). Two corpora, same
result.
**The `pdftotext` row is a reference line, not a fourth candidate, and its

17
docs/2026-09-08-claude-code-skill-vilkaarlig-bundle.md
View file

@ -239,7 +239,7 @@ cd /tmp/c1-scratch && claude -p "<the question>"
|---|---|---|---|---|
| i | fresh, 54 concepts | "Hvordan skal prisene fylles ut?" | yes | **pass** |
| ii | fresh, 54 concepts | a topic measured absent from the bundle | yes | **pass** — refused |
| iii | third-party, 270 concepts | "Hva krever Krav 10.2—2 i N500? Gjengi det sentrale vilkåret." | **no** | **pass** |
| iii | third-party, 270 concepts | a question naming one requirement number and its document | **no** | **pass** |
| iv | fresh, 54 concepts | "Hvordan skal prisene fylles ut?" | **no** | **pass** — and selected the right one of two |
**(i) The known-positive.** Rank 1 of 8 delivered. The answer opened by naming
@ -261,7 +261,7 @@ used to check the delivered texts, and with the 46 withheld concepts named as
`[unverifiable-from-bundle]`.
**(iii) The third-party bundle.** Measured on the tree on disk at the time:
`vegnormal-n500-2024`, 270 concepts, ref
a third-party bundle, 270 concepts, ref
`sha256-tree:673a0c2c…d6b5b3ad` — **not** the V1 tree the order named, because
that repository landed a rebuild during the day. The gold concept came back at
rank 1, and the answer quoted the requirement verbatim against the concept body,
@ -312,21 +312,20 @@ denominator, corrected here rather than published.
## 5. A finding about the ranking, not about the skill
The first attempt at (ii) asked the normative bundle about water and frost
protection in a subsea tunnel. The model answered
The first attempt at (ii) asked the normative bundle about a compound topic
it was believed not to cover. The model answered
`[sourced-not-sufficient]`, said the relevant requirements "if they exist, are
among these 262 withheld", and refused to go around the cut. Checked
afterwards, **the bundle does cover it**: 17 files match `frostsikring`, 10 match
"vann- og frostsikring", 13 match `undersjøisk`, and none of them was ranked
into the cut.
afterwards, **the bundle does cover it**: 17, 10 and 13 files match the
question's three key terms, and none of them was ranked into the cut.
So it was never a known-negative — it was a **retrieval miss**, and it is
recorded as one rather than quietly replaced. Two things follow, and they point
in opposite directions. The consumer behaved exactly as the contract asks: it
did not fabricate, it named its denominator, and it reported its own zero as
unmeasured because the `withheld` entries carry only ids and rules, no titles.
The ranker did not: a compound Norwegian question ("vann- og frostsikring",
"undersjøisk") was out-ranked by concepts sharing only "tunnel" and "vann". That
The ranker did not: a compound Norwegian question was out-ranked by concepts sharing only two
common words with it. That
belongs with the fusion work already open, and it is the same shape as the
`MIN_SHARED_PREFIX` finding.

344
docs/2026-09-08-eksakt-oppslag.md
View file

@ -1,344 +0,0 @@
# The exact lookup: naming a document is not searching for it, 2026-09-08
Two sessions made a requirement number visible to the consumption pre-pass and
then made it *worth* more than a common word, and neither delivered the
document the question named. The first left the gold `below_k` on all three
bundles (`docs/2026-09-08-kravnummer-tokenisering.md` § 8); the second
delivered one of three and decomposed the other two into a **matcher**
decision and a **fusion** decision, with no measurement behind either
(`docs/2026-09-08-sjeldenhetsvekt.md` § 3, § 4).
This session takes both decisions with numbers. The result is that a question
naming a requirement number now delivers that requirement at **rank 1 on 3 of
3 bundles**, and that every published figure on the consumer corpus is
**byte-identical**.
---
## 0. What IS measured, and what is NOT
**Measured.** Eleven known-positive figures reproduced before anything was
changed; document frequency under the matcher and under equality on three
bundles; the gold's fused rank, verdict, payload digest and byte count before,
after the matcher alone, and after both mechanisms; the three spellings of one
identifier; every named control on the consumer corpus against a frozen
`git archive` copy of `116d3e1`; a control question on that corpus that *does*
carry an identifier, before and after; hit@8 over the six published questions
and both known-negatives; the cost of each new pass; seven mutations.
**Known-positive, run first.** Every figure this session builds on was
reproduced on this machine, with the committed tool, on a clean tree at
`116d3e1`, before any after-column was read:
| # | known-positive | source | measured here |
|---|---|---|---|
| 1 | gold fused rank per bundle, weight off | previous session § 2 | **96, 9, 35** |
| 2 | gold fused rank per bundle, weight on | previous session § 2 | **103, 8, 35** |
| 3 | `df` under the matcher vs under equality | previous session § 3 | **135 / 17 / 5** vs **1 / 1 / 1** |
| 4 | considered / withheld / delivered | consumer § 3 | 446/438/8, 1133/1125/8, 270/262/8 |
| 5 | gold position in `withheld` | consumer § 3.1 | 119 of 438, 204 of 1125, 66 of 262 |
| 6 | payload `sha256`, weight off | previous session § 2 | `49e9c718…`, `adc2eac7…`, `21c5bbfe…` |
| 7 | payload `sha256`, weight on | previous session § 2 | `ba09a56a…`, `403ee989…`, `1758f6ca…` |
| 8 | consumer corpus, the good question's candidate rank, four rows | previous session § 5 | **1 / 1 / 1 / 1** |
| 9 | the priced sheet's candidate rank, four rows | previous session § 5, § 7 | **251 / 10 / 78 / 2** |
| 10 | the mandate question's default payload | previous session § 7 | **169 583 B** |
| 11 | hit@8 over the six published questions | 2026-09-07 § hit@k | **5 of 6**, five of them at rank 1 |
**NOT measured**, named rather than glossed:
- **`o200k_base` token counts.** No `o200k` encoder is installed on this
machine and this repository's single-runtime-dependency rule does not admit
one to produce a caption. Figures below are in `BUDGET_UNIT` — utf-8 bytes of
emitted JSON — plus the payload `sha256`, which is exact where a byte count
is only indicative.
- **The rebuilt consumer bundle of `56c1205` is not on disk.** Its content
identity (`sha256-tree:2f82fcfe…`) names a tree this machine no longer holds,
and neither does the bundle the two S7 byte figures in the order come from
(`2ae46f68…` 169 573 B and `8aaf8558…` 172 588 B). **Those two figures are
therefore not reproduced here.** The control corpus used instead is the
bundle delivered 2026-09-03 (`sha256-tree:9a4e5561…`), on which rows 8, 9 and
10 of the table above reproduce the published values exactly. This costs the
decision nothing: what the decision needs is byte-*identity* before and
after, and that is measured on one and the same bundle, against a frozen copy
of the committed code rather than against a remembered number.
- **Whether a live model answers better.** This measures a *ranking*.
"Delivered" is not "read", and that measurement belongs to the consumer.
---
## 1. Mechanism (a): an identifier matches by equality, and by nothing else
`tokens_match` compared four leading characters, for a reason that was measured
and still holds: Norwegian compounds do not match token-exactly, and
`vare|ne` and `vare|mottak` share a stem. A requirement number has no stem.
Four leading characters of `3.3.1-13` are four leading characters of every
requirement in section 3.3, so a **unique** identifier read as **135 of 446**
common — and the rarity weight then faithfully ranked a common adjective as
more informative than the number naming the document.
The rule, whole:
```python
if is_identifier(left) or is_identifier(right):
return left == right
```
`is_identifier` is `_IDENTIFIER_RE.fullmatch`, the pattern the tokeniser
already uses to keep an identifier whole. No new constant, no token class
declared anywhere, and words are untouched.
**Red first.** `tokens_match('3.3.1-13', '3.3.1-14')` was `True` and is now
`False`; `('3.3.1-13', '3.3.1-13')` was and is `True`. The known-positive ran
in the same file and passed before and after: `brannsikring`/`brannvern`,
`prisene`/`prissammenstilling`, `varene`/`varemottak` all still match, and
`brann`/`bygg` still does not.
**`df` after, same instrument as before:**
| bundle | identifier | `df` before | `df` after | `df` exact | concepts |
|---|---|---|---|---|---|
| N100 | `3.3.1-13` | **135** | **1** | 1 | 446 |
| N200 | `2.9.2-12` | 17 | **1** | 1 | 1 133 |
| N500 | `10.2-2` | 5 | **1** | 1 | 270 |
**A short identifier stops being invisible.** `MIN_SHARED_PREFIX` made a
three-character identifier match *nothing, not even itself*: measured on the
629-concept consumer bundle, `9.2` reached **0** concepts under the matcher
while sitting verbatim in one title. Equality has no floor, so it now reaches
that one. The floor stays where it was measured — for words.
**What the matcher alone bought, and what it cost.** This is the honest column,
and it is not all in one direction:
| bundle | gold fused rank, before | after (a) alone | with `--rarity-weight`, before → after |
|---|---|---|---|
| N100 | 96 | **53** | 103 → **45** |
| N200 | 9 | **26** | 8 → **23** |
| N500 | 35 | **37** | 35 → **36** |
**N200 got worse, and the reason is measured rather than guessed.** That gold's
body carries **88** identifiers, three of them occurrences of `2.9.2-1` — a
cross-reference to the neighbouring requirement. Under the prefix rule that
cross-reference answered the question's `2.9.2-12`, so the gold was scoring a
hit on its own body that it had not earned. Removing a false positive that
happened to help is still removing a false positive; the number is here so that
nobody has to take that on faith. On its own, mechanism (a) delivers **0 of 3**.
---
## 2. Mechanism (b): the form was chosen by a measurement, not by a preference
A question carrying an identifier that sits verbatim in a concept's title or id
is a **lookup**: the reader already knows which document they want and is
spelling its number. The order named two candidate forms and invited a third.
**Form (i), a fourth signal inside the RRF fusion, was simulated first** on the
same three bundles, on top of mechanism (a), with the lookup as a 0/1 signal:
| bundle | gold rank with a fourth RRF signal | of |
|---|---|---|
| N100 | **26** | 446 |
| N200 | **15** | 1 133 |
| N500 | **19** | 270 |
**None of them delivered**, and the reason is the one the previous session
already wrote down: RRF consumes **ranks only**, so any single signal
contributes at most `1/(RRF_K + 1)` however certain it is. A concept the
question *names* cannot outbid three signals that merely describe it. Form (i)
is refuted by its own numbers, not by taste.
**Form (ii) ships:** a partition applied to the fusion's output. Concepts whose
title or id bears the question's identifier are moved to the front, keeping the
order the fusion gave them; everything else keeps its order too. Nothing is
dropped, nothing is scored twice, and the fused ranks themselves are untouched.
```python
named = set(lookup_hits(concepts, question))
if named:
ranked_ids = [k for k in ranked_ids if k in named] + [k for k in ranked_ids if k not in named]
```
**It reads the text the title-and-id signal reads, and declares no key list.**
The order allowed a frontmatter key set (`req_number`, `identifier`, …). It is
not declared, because a measurement decided instead: of the **1 846** concepts
across the three bundles that carry a `req_number`, the identifier in that key
is **also in the title on 1 846** of them, and on **0** does the key carry an
identifier the title lacks. A key list would have bought nothing and would have
been a constant no measurement asked for. The consumer corpus has no such key
at all, and is served anyway (§ 4).
**The known-negative is structural, not lucky.** `lookup_hits` returns the
empty tuple whenever the question carries no identifier, and **every one of the
ten published control questions on the consumer corpus carries zero
identifiers** under `_IDENTIFIER_RE` — the six hit@8 questions, both
known-negatives, the good question and the mandate-shaped one. Verified with
`normalise` on each question before any payload was built.
**Red first,** on a synthetic 300-concept corpus where every concept carries
the word a standards corpus repeats on every page and the neighbours are
numbered so that they share the gold's first four characters: the named concept
comes back at rank 1; two concepts bearing the same number come back as the top
two; a question with no identifier leaves the order untouched; an identifier no
concept bears changes nothing and empties nothing.
---
## 3. After: three bundles, one instrument, before and after
Instrument, unchanged from the previous session and from § 0 above: the gold's
position in the fused `ranked` list, plus the payload's own verdict from
`build_payload`. Command:
```
python3 tools/okf_consume.py <bundle> \
--question "Hva krever Krav <n> i <normal>? Gjengi det sentrale vilkåret." \
[--rarity-weight]
```
| | N100:2023 | N200:2024 | N500:2024 |
|---|---|---|---|
| concepts | 446 | 1 133 | 270 |
| gold fused rank, **before** | 96 | 9 | 35 |
| gold fused rank, **after** | **1** | **1** | **1** |
| gold verdict, before | `below_k` | `below_k` | `below_k` |
| gold verdict, after | **delivered, rank 1** | **delivered, rank 1** | **delivered, rank 1** |
| the same with `--rarity-weight` | **1**, delivered | **1**, delivered | **1**, delivered |
| payload bytes, before → after | 61 566 → 58 409 | 142 861 → 146 145 | 40 626 → 39 193 |
| payload `sha256`, after | `5bf6ab78…` | `af5829cd…` | `9a9e7860…` |
| `okf_contract_check` | exit 0, 14 rules, 0 findings | idem | idem |
| lookup hits found | 1 | 1 | 1 |
**3 of 3, at rank 1.** The order's target was rank ≤ 8 on 3 of 3.
**The three spellings are one lookup.** For each bundle, the question was asked
with an ASCII hyphen, an en dash and an em dash. All three deliver the **same
eight concepts in the same order**, with the gold at rank 1 — the payload
digests differ only because the question string is echoed in the payload, so
the excerpt list is compared instead.
**Cost.** The lookup is one pass over the concepts already in memory:
**0.011 s** over 446, **0.024 s** over 1 133, **0.006 s** over 270. The `df`
pass the rarity weight uses, re-measured under the new matcher: 0.055 s,
**0.227 s**, 0.064 s. Both are inside the order's one-second bound on the
largest bundle by a factor of four or better, and a question carrying no
identifier does not tokenise the corpus at all.
---
## 4. The consumer corpus: nothing moves, and one unreachable thing becomes reachable
Every named control was measured twice — once from a `git archive` copy of
`116d3e1` extracted to a scratch tree, and once from this working tree — on the
same bundle, with `PYTHONPATH` and `__file__` both checked so the frozen run
could not import the working copy.
| control | before | after |
|---|---|---|
| good question, candidate rank | 1 of 233 | **1 of 233** |
| good question with the cost vocabulary | 1 of 248 | **1 of 248** |
| the same two with `--rarity-weight` | 1 / 1 | **1 / 1** |
| priced sheet, mandate question, vocabulary off | 251 of 271 | **251 of 271** |
| priced sheet, `--cost-vocabulary --k 12` | 10 of 280 | **10 of 280** |
| the same two with `--rarity-weight` | 78 / 2 | **78 / 2** |
| mandate question, default payload | `78fec2ef…`, 169 583 B | **byte-identical** |
| mandate question, `--cost-vocabulary --k 12 --limit 160000` | `1f11e56b…`, 240 021 B | **byte-identical** |
| mandate question, `--cost-vocabulary --k 12` | `777986a5…`, 172 246 B | **byte-identical** |
| good question, default payload | `f7ff803e…`, 165 109 B | **byte-identical** |
| hit@8 over the six published questions | 5 of 6 | **5 of 6**, every rank, byte count and spend identical |
| both known-negatives | 8 delivered each | **byte-identical** |
**Every figure in that table is identical** — the four payload digests and
their byte counts, the eight candidate ranks, all six hit@8 rows with their
spends, and both known-negatives. No consumer's bytes move because these two
changes exist — which is what makes them safe to ship on by default rather than
behind a flag.
**Generality, on a corpus with no requirement numbers at all.** The consumer
bundle has no `req_number` key and no requirement grammar; its documents are
numbered in their titles instead (`… 9.2 …`). One control question was written
that names such a number. Before, that document was withheld `below_k` at
position **621 of 621** — dead last, because a three-character identifier
matched nothing under `MIN_SHARED_PREFIX`, not even itself. After, it is
**delivered at rank 1**. The mechanism is not a vegnormal mechanism.
---
## 5. The decision, made by the number
The order's rule: (a) is a defect fix and ships as default if every control on
the consumer corpus stands byte-identical; (b) ships as default if the
known-negative holds.
| mechanism | condition | measured | decision |
|---|---|---|---|
| (a) identifier matches by equality | all consumer controls byte-identical | **every control identical** | **default, no flag** |
| (b) the lookup partition | byte-identical for a question with no identifier | **structural: 0 identifiers in 10 of 10 control questions; every payload identical** | **default, no flag** |
Nothing fell, so no flag is added and nothing has to be named as a loss.
`--rarity-weight` is untouched and stays **off**; with it on, the three lookups
also deliver at rank 1, so the two mechanisms do not fight.
---
## 6. What did NOT close
- **`.` and `-` are not interchangeable in a lookup.** `_DASH_TO_HYPHEN` folds
every dash spelling to one, so em dash, en dash and hyphen are one lookup —
but a question spelling `1.10` does not find a document whose id spells it
`1-10`, and this library's own id grammar produces exactly that hyphenated
form from a dotted source. Measured and left open: the fix is a
canonicalisation inside the lookup, and it needs its own before/after.
- **A bundle whose identifiers live only in frontmatter is not served.** Today
that set is empty (0 of 1 846), which is why no key list was declared. If a
consumer arrives with one, this is the rule that has to grow.
- **An identifier that appears only in a concept's BODY is not a lookup.** It
is a search, and it ranks like one.
- **The matcher alone made two of three lookups worse** (§ 1). Only the
partition delivers. Anyone lifting one half without the other should read
that table first.
- **The two S7 byte figures from the O1 bundle are not reproduced**, because
neither that bundle nor the rebuilt one is on this machine (§ 0).
- **"Delivered" is not "read".** All three golds are now in the payload at rank
1. Whether a model then answers the engineer's question is the consumer's
measurement, and this document does not claim it.
---
## 7. Tests, mutations, gates
Thirteen new tests. **Twelve were written before either rule existed and
eleven of them failed**; the twelfth is the known-positive that had to pass
from the start (the Norwegian compounds), so the red was the rules' absence and
not a broken fixture. The thirteenth was written afterwards, against a mutation
that survived, and is red against it. One fixture defect was found by the red
run itself: the neighbours were numbered so that one of them *was* the gold's
number, which made the `df` 2 rather than 1.
**Seven mutations, seven red.** Three survived the first pass and none was
banded blind:
- **M2** (`is_identifier` matches a substring instead of the whole token)
survived because no token `normalise` emits can tell the two apart — the
generic split keeps neither `.` nor `-`, so a split token can never contain
an identifier. The commitment is real even where no output distinguishes it,
so it is now asserted on the function.
- **M4** (the early return for a question with no identifier) survived because
an empty identifier set intersects to nothing anyway. The guard is a **cost**
commitment, not a semantic one, so it is now measured as one: a question
without an identifier must not tokenise the corpus at all.
- **M6** (the lifted concepts re-sorted by id instead of keeping the fused
order) survived because the fixture's two holders tied. The fixture was
corrected so byte order and fused order disagree on purpose.
| gate | result |
|---|---|
| `pytest -q` | **1 320 passed**, 1 skipped (1 307 on this branch before, +13) |
| `ruff check src tests tools` | clean |
| `ruff format --check .` | clean, 88 files |
| `mypy --strict src/` | clean, 17 files |
| `okf_contract_check` on the three lookup payloads | exit 0, 14 rules, 0 findings |
| `git diff --stat` | `tools/okf_consume.py`, `tests/test_okf_consume.py` |
**No rebuild.** The change is consumption-side only, confirmed by
`git diff --stat`: no production module under `src/` is touched, so every
bundle ref in flight is unchanged.

2
docs/2026-09-08-k3-runde5-hitat8-og-skriftakse.md
View file

@ -307,7 +307,7 @@ reason and it does not hold (§ 7), so the default stays where it is.
- **`pdf` position 1** (17 against a reference of 15) is now refused on four
axes: number, outline, typography, and the reserve — the reserve is silent
there, because that document has two outline runs.
- **vegnormal-okf FUNN 1 and FUNN 2** are still not investigated in this repo.
- **A downstream consumer's FUNN 1 and FUNN 2** are still not investigated in this repo.
- **`tests/test_okf_consume.py:940` pins hit@8 against the Arm B bundle alone.**
The default bundle is pinned nowhere. That is a gap, and it is named in
STATE's next block rather than closed here: O4/O5 fence that file, and this

343
docs/2026-09-08-kravnummer-tokenisering.md
View file

@ -1,343 +0,0 @@
# The requirement number the pre-pass could not see, and the title the orphan gate destroyed, 2026-09-08
Two consumer-reported defects, one session, one rebuild. The first is a
consumption-side defect reported by `vegnormal-okf`: `okf_consume`'s tokeniser
shatters a requirement number, so the one concept an engineer names is withheld.
The second is a production-side defect this repository measured on itself in the
previous session: a table that opens directly under a heading destroys that
heading's name.
---
## 0. What IS measured, and what is NOT
**Measured.** The five tokeniser strings the consumer published, reproduced
here before anything was changed; where the number is lost, decomposed to both
mechanisms; the gold concept's rank on three real bundles, before and after, with
denominators; one alternative rule falsified against the same three bundles; the
K2 control question as a payload digest with the tokeniser as the only variable;
the title axis reproduced from the previous session's number before it was
touched; a full K2 rebuild with the conservation identity, the concept count, the
number of ids that moved against a denominator of 629, and the new content
identity; and six mutations against the new tests.
**Known-positive, run first.** Nine figures published elsewhere were reproduced
on this machine before any of the after-columns were read:
| # | known-positive | source | measured here |
|---|---|---|---|
| 1 | `normalise('Krav 10.2—2')` → `['krav']` | consumer § 3.1 | `('krav',)` |
| 2 | `normalise('10.2-2')` → `[]` | consumer § 3.1 | `()` |
| 3 | `normalise('3.3.1—13')` → `[]` | consumer § 3.1 | `()` |
| 4 | `'N500'` → `['n500']`, `'abc123'` → `['abc123']` | consumer § 3.1 | identical |
| 5 | considered / withheld / delivered per bundle | consumer § 3 | 446/438/8, 1133/1125/8, 270/262/8 |
| 6 | gold position in `withheld` | consumer § 3.1 | 119 of 438, 204 of 1125, 66 of 262 |
| 7 | payload tokens (o200k) per bundle | consumer § 3 | 22 138, 53 444, 14 694 |
| 8 | priced concept candidate rank 10 → 19 | this repo, 2026-09-08 (O1) § 7 | 10 → 19, and 251 / 1 / 1 for the other three rows |
| 9 | K2 payload bytes, two commands | this repo, O1 § 7 | 169 573 B and 172 588 B |
> **Superseded literal (added 2026-09-08, round 3).** The S7 control figures in
> this document were correct when it was written and are not the ones to reuse:
> `a37d5ce`, `17c49fc` and `c95d189` each widened the excerpt form afterwards.
> The current known-positive for the S7 control question under the default
> command is **`c759a657dbb41d71`, 171 614 B**, measured on `c3b645b` against
> the same bundle. `docs/2026-09-08-k3-runde3-per-filtype.md` § 4.
**NOT measured.** Whether a live model answers *better* with the named
requirement in the payload. This document measures a ranking; the consumer
measures the answer. Also not measured: that either rule helps a corpus other
than the four read here.
**Not closed.** The gold requirement is still `below_k` on all three bundles
after the fix. § 8 states why, with the number.
---
## 1. The five strings, and one correction to how they are read
```
PYTHONPATH=… python3 -c "from okf_consume import normalise; …"
'Krav 10.2—2' -> ('krav',) 'N500' -> ('n500',)
'10.2-2' -> () 'abc123' -> ('abc123',)
'3.3.1—13' -> () '10' -> ()
'2.9.2—12' -> () '2' -> ()
```
All five reproduce, and so do the two known-positives. So does the gold's place
in the `withheld` list: 119 of 438, 204 of 1125, 66 of 262.
**That place is a list position and not a rank, and the difference matters.**
The `withheld` array is emitted in `concept_id` order — measured, not assumed
(`sorted(ids) == ids` is `True` on all three payloads) — so it cannot move when
the ranking moves. It is a correct number that answers a different question. The
instrument that does move is the fused rank, and § 4 uses it.
One more figure the consumer's mechanism claim predicts, confirmed here: of the
262 withheld entries on the smallest bundle, **262 are `below_k` and none is
`no_lexical_match`**. Every concept matches, because every concept carries the
word the question carries.
---
## 2. Where the number is lost: both mechanisms, not one
The order asked which of the two is responsible. **Both**, and neither alone
would lose it:
```
'10.2-2' split-> ['10', '2', '2'] kept-> []
'3.3.1—13' split-> ['3', '3', '1', '13'] kept-> []
'R610.4' split-> ['r610', '4'] kept-> ['r610']
```
`_TOKEN_SPLIT_RE` (`tools/okf_consume.py:523` before this change) splits on
every non-alphanumeric character, which destroys the identifier; the
`MIN_TOKEN_LENGTH` filter in `normalise` (line 537) then removes the digit runs
that are left. Lowering the floor alone would not recover the number — it would
return `10`, `2` and `2`, which match every page number in the corpus. The
identifier has to survive the split.
---
## 3. The rule, and the alternative that was falsified
**The rule.** Numeric groups joined by `.` or `-` are held together as one
token, with a leading letter run allowed only where it touches its digits
without a separator. Dash variants fold to the ASCII hyphen first, so the same
number written with an em dash, an en dash or a hyphen is one token.
`MIN_TOKEN_LENGTH` is unchanged; the identifier is exempt from it by
construction rather than by lowering it for everyone.
**It was narrowed by a measurement, not by taste.** The first version joined
*alphanumeric* groups across a separator. That version swallowed a whole
document slug — a slug of the form `…-3-6-…` became ONE token — and that
document's stage-one score for a question naming its subject fell from **0.735
to 0.0**, costing one row of the repository's own hit@8 measurement. The
narrowed rule is purely additive on such a slug: the identifier is added, the
words stay.
**The falsified alternative.** Because `tokens_match` compares a four-character
prefix, `10.2-2` matches `10.2-5`: on the smallest bundle exactly **5 of 270**
concept titles match the question's identifier, and all five are the same
section. Making identifier tokens match only on *equality* is the obvious
narrowing, and it was measured before being adopted or rejected:
| | prefix (shipped) | equality (probe) |
|---|---|---|
| N100:2023 | 96 | **53** |
| N200:2024 | **9** | 26 |
| N500:2024 | **35** | 37 |
It improves one bundle and makes two worse. **Falsified; not built.**
---
## 4. The three bundles, before and after
Command, one per bundle, the questions quoted verbatim from the consumer's § 3:
```
python3 tools/okf_consume.py <bundle> \
--question "Hva krever Krav 10.2—2 i N500? Gjengi det sentrale vilkåret."
```
| | N100:2023 | N200:2024 | N500:2024 |
|---|---|---|---|
| concepts | 446 | 1 133 | 270 |
| gold fused rank, **before** | 160 | 143 | 100 |
| gold fused rank, **after** | **96** | **9** | **35** |
| gold lexical overlap, before → after | 3 → 4 | 4 → 6 | 3 → 4 |
| considered / withheld / delivered | 446/438/8 | 1133/1125/8 | 270/262/8 |
| budget spent, before → after | 8 977 → 12 095 | 17 818 → 17 818 | 10 517 → 10 517 |
| payload o200k, before → after | 22 138 → 23 583 | 53 444 → 53 444 | 14 694 → 14 694 |
| `okf_contract_check` | exit 0, 14 rules, 0 findings | idem | idem |
| **gold delivered?** | **no, `below_k`** | **no, `below_k`** | **no, `below_k`** |
Two of the three payloads are byte-identical before and after: the gold moved a
long way without reaching the top eight, so the delivered set did not change.
**The bonus measurement the order asked for.** Three spellings of the same
requirement number, on the same bundle:
| question spelling | identifier token | gold rank |
|---|---|---|
| `Krav 10.2—2` (em dash) | `10.2-2` | 35 |
| `krav 10.2-2` (hyphen, lower case) | `10.2-2` | 35 |
| `Krav 10.2–2` (en dash) | `10.2-2` | 35 |
All three normalise alike and rank alike.
---
## 5. The K2 control: the tokeniser alone changes nothing
The order required the S7 control question to be compared byte for byte with
the tokeniser as the only variable, so it was run against ONE unchanged bundle
with the committed tool and the new tool:
| command | before | after |
|---|---|---|
| default | `sha256 2ae46f683a3b17fb…`, 169 573 B | **identical** |
| `--cost-vocabulary --k 12 --limit 160000` | `sha256 8aaf8558369b3d6b…`, 172 588 B | **identical** |
Byte for byte, both commands. The two byte counts are the ones published in the
previous session's report, which is what makes this a control and not a
coincidence. The corpus does carry identifiers of other shapes (`NS3935:2019`,
`TEK 17`); measured, the rule leaves both untouched, because a colon is not a
separator it claims and a space was never one.
---
## 6. The title axis in the orphan gate
Reproduced first, from the previous session's numbers, before the rule was
touched — all four rows to the integer:
| question | vocabulary | O1 measured | reproduced here |
|---|---|---|---|
| mandate-shaped | off | 251 of 271 | 251 |
| mandate-shaped | on | **19** of 280 | **19** |
| specific | off | 1 of 233 | 1 |
| specific | on | 1 of 248 | 1 |
**The rule.** When a heading is dropped by the orphan check and the candidate
immediately after it is a table block, the table block takes that heading's
title *and* its section number instead of the mechanical `Tabell linje <n>`.
Three properties of that sentence are each a decision:
1. **Conditioned on the drop.** A heading that keeps its own body is still
carried by a live candidate; copying its title onto the table as well would
put one name on two concepts and rescue none.
2. **Immediately after.** The name goes to the adjacent segment or nowhere.
Carried further, a table several candidates away would take the name of a
heading it does not contain — an invention rather than a rescue. This is the
one property no test caught until a surviving mutation exposed it (§ 9).
3. **Number as well as title.** `_segment_path` reads both — the number becomes
the directory and is stripped from the stem — so inheriting the title alone
would emit `3-1-prissammenstilling.md` at the top level, a name the heading
never had.
It is format-agnostic by construction: the orphan check and the table rule are,
so a sheet name in a spreadsheet and a heading in a PDF go the same way.
---
## 7. The rebuild: 629 concepts, two ids, a new identity
One rebuild for both fixes, `okf build` over the same 43-document folder, run
from a frozen copy of `src/` so no later edit could reach a measurement already
in flight.
| | value |
|---|---|
| `merged + coded rejections == N` | `39 + 4 = 43 = N`, exit 0 |
| concepts | **629**, unchanged |
| concept ids that moved | **2 of 629** (0.32 %) |
| new content identity | `sha256-tree:2f82fcfea91c3bd3f8ef7147f80cd613227d3ca7975c41d88810233f3f79ab4b` |
**The two ids move BACK.** `…/tabell-linje-3` becomes `…/prissammenstilling-sheet-1`
and `…/ark1-sheet-1` again — the names they carried in the bundle delivered on
2026-09-03. Against that delivered bundle, **1 106 of 1 108 files are now
identical**, and the two that differ are the two spreadsheet bodies the previous
session deliberately changed. A consumer holding the 2026-09-03 tree gets its
file names back; the ref, however, is new, and `c26eed6a…` from the previous
session is superseded and stale.
**The ranking, on the rebuilt bundle:**
| question | vocabulary | O1 bundle | rebuilt |
|---|---|---|---|
| mandate-shaped | off | 251 of 271 | 251 |
| mandate-shaped | on | 19 of 280 | **10** |
| specific | off | 1 of 233 | 1 |
| specific | on | 1 of 248 | **1** |
Rank 10 is exactly where the concept sat before the previous session's form fix.
The regression is closed, and the question this repository's own hit@8
measurement uses is untouched.
**What it now costs to deliver that concept**, which is the figure a consumer
budgets against:
| command | bytes | o200k | priced sheet |
|---|---|---|---|
| default | 169 583 | 53 420 | `below_k` |
| `--cost-vocabulary --k 12` (default budget) | 183 798 | **58 907** | **delivered, rank 10** |
| `--cost-vocabulary --k 20` (default budget) | 190 438 | 61 955 | delivered, rank 10 |
| specific question, default | 108 877 | 37 189 | delivered, rank 1 |
The previous session's delivering command was `--cost-vocabulary --k 20` at
**65 912** o200k. The delivering command is now `--cost-vocabulary --k 12` at
**58 907** — 10.6 % cheaper, and it fits the default budget. `okf_contract_check`
returns exit 0 with 14 rules and 0 findings on all three payloads, and the
identity closes: `8 + 621 = 629`, `12 + 617 = 629`.
---
## 8. What did NOT close, and why — stated rather than glossed
**The requirement question is still `below_k` on all three bundles.** The fix
moved the gold from 160 to 96, 143 to 9 and 100 to 35, which is a large move and
not a delivery. The remaining blocker was decomposed rather than guessed:
`_overlap` returns a COUNT. On the smallest bundle the gold's overlap is 4 out
of a possible 7, and **70 of 270 concepts also score 4** while 12 score 5.
Matching the exact requirement number is worth exactly as much as matching the
verb in `Hva **krever** …` or the noun in `… det sentrale **vilkåret**`. The
identifier is now visible to the ranker; it is not yet *worth* more than a
common word.
Making it worth more is a weighting decision — a rarity weight, or a declared
class of high-specificity tokens — and it is a different change from this one:
it moves every ranking in every corpus, and it has no measurement behind it yet.
It is not in this order and is not decided here.
**And the honesty limit that governs all of the above.** This document measures
a *ranking*: which concepts a deterministic pre-pass puts in a payload. Whether
a live model then answers the engineer's question correctly is the consumer's
measurement, not this one. "Delivered" is not "read".
---
## 9. Tests, and the mutation that survived
Eight new tests, each written red first (1 287 -> 1 295). Six
mutations, six red, with the unmutated control green after every restore:
| mutation | what it breaks | result |
|---|---|---|
| widen the identifier back to alphanumeric groups | the slug regression | 3 red |
| drop the dash fold | one spelling finds, two do not | 2 red |
| never emit the identifier token | the whole rule | 3 red |
| inherit the title but not the number | the emitted path | 1 red |
| inherit onto any rule, not only a table | one name on two concepts | 1 red |
| never clear the carried name | a distant table takes the name | **survived** |
The survivor was read as code before it was called equivalent, per this
repository's own trap note: it is not equivalent — it lets a heading's name
travel arbitrarily far — and the band simply had no fixture where an orphan is
followed by something other than a table. One was added, and the mutation now
dies.
Suite: **1 295 passing** (was 1 287), `ruff` clean, `mypy` clean on 17 source
files. Both byte-pinned proposer goldens are unchanged, which is what says the
title rule does not fire where no heading is being destroyed.
---
## 10. Verification log
| # | claim | how it was checked |
|---|---|---|
| 1 | the five tokeniser strings | run against the committed tool before any edit; all five identical to the consumer's |
| 2 | the gold's `withheld` position is not a rank | `sorted(ids) == ids` on all three payloads → `True` |
| 3 | both mechanisms lose the number | split output printed separately from the length filter |
| 4 | the wide rule costs a hit@8 row | 5 of 6 → 4 of 6, and the losing row's document score 0.735 → 0.0 |
| 5 | equality matching is worse | measured on all three bundles; 1 better, 2 worse |
| 6 | the tokeniser alone is byte-stable on K2 | same bundle, two tools, two commands, identical sha256 |
| 7 | rank 10 → 19 (O1's regression) | reproduced on O1's own bundles before the rule was touched |
| 8 | 2 of 629 ids moved | set difference of the two concept-id sets |
| 9 | 1 106 of 1 108 files identical to the delivered bundle | `diff -rq`, and `find -type f | wc -l` for the denominator |
| 10 | the rebuild reads frozen source | `import llm_ingestion_okf; __file__` under the build's own `PYTHONPATH` |
| 11 | contract conformance | `okf_contract_check.py` exit 0, 14 rules, 0 findings, on six payloads |

44
docs/2026-09-08-proveniens-k2.md
View file

@ -25,8 +25,7 @@ per format ("can a consumer find the file, verify it, open the right place?");
one hand-walked known-positive from `source_offset` back to a page of the
original, counted in steps; the guard's verdict on seven candidate `sources`
forms, known-positive first; a full K2 rebuild with the file-set,
byte-diff and content identity; the payload a consumer receives; and the
viewer's own route table.
byte-diff and content identity; and the payload a consumer receives.
**Known-positive, run first.** The rebuild reproduces the bundle this
repository has been measuring on all week before anything else is read from it:
@ -45,9 +44,6 @@ byte-identical to the previous bundle.
actually opens the original is `portfolio-optimiser`'s measurement, not this
one — and it cannot be, because the payload does not carry these keys at all
today (§ 6).
- **The viewer's per-requirement anchor.** One route table and one JS bundle
were read (§ 7). Whether `?reference=` positions a requirement needs a
browser, and the app lazy-loads chunks this measurement did not fetch.
- **A `.doc`, `.zip` or `.smc` row.** Those three types are rejected before any
concept exists, so they have no provenance to carry and no denominator here.
@ -117,10 +113,9 @@ SPEC v0.2 § 5.1:303-306, verbatim:
> it cannot (for example `all queries in BigQuery project X`).
A dropped file's inbox-relative path is precisely "a bundle-relative path", so
the ADDRESS needs no key of ours. `docs/okf-nokkelinventar.md` had already
named that re-alignment for `source_file` (Del B, `sources[].resource` as a
bundle-relative or `references/` path, § 5.1:303-305 + § 6.3); this step
executes it.
the ADDRESS needs no key of ours. An earlier key inventory had already named
that re-alignment for `source_file` (`sources[].resource` as a bundle-relative
or `references/` path, § 5.1:303-305 + § 6.3); this step executes it.
§ 5.1 lists exactly six entry keys — `resource`, `id`, `title`, `author`,
`usage_count`, `last_modified` — and **none of them is a location within the
@ -349,34 +344,6 @@ paragraph is the whole of the fix's cost.
---
## 7. The viewer anchor, for `vegnormal-okf`
`https://viewers.vegnorm.vegvesen.no/product/859990/nb` answers **200** with an
**863-byte SPA shell** — no content, so the page itself decides nothing. The
route table and the query keys come from the app's own bundle
(`/assets/index-D1Z6iWLz.js`, 3 407 504 B):
- The route is **`/product/:id/:languageCode?/:parameter?`**. Read in the app,
`:parameter` is compared against exactly two literals — `"selection"` and
`"preview"`. **It is not a per-requirement anchor.**
- The app parses a QUERY STRING into `filePath`, `organization`,
`publicationUrn`, `tenantId`, **`reference`**, `isPrint`, `theme`, and passes
them to its content API call. **`?reference=` is the only candidate** for a
per-requirement address.
- **0 occurrences of `scrollIntoView`** in that bundle, so there is no
fragment-anchor scroll in it. `location.hash` appears 3 times, all inside the
router.
**Denominator, stated:** ONE bundle file, fetched once. The app lazy-loads
further chunks (`_mapDeps([20, 21])`) that this measurement did not fetch, and
what `reference` does to the rendered document needs a browser. So: a
per-requirement URL is **plausible via `?reference=<value>`** and **not
demonstrated**. Either way the value is the producer's to choose, and
`sources[].resource` is where it belongs — `vegnormal-okf` owns `sources` for
those bundles.
---
## 8. Honesty limits
1. **`source_lines` is not a paragraph number**, and for `.docx`, `.pptx`,
@ -399,7 +366,7 @@ those bundles.
public tender.
6. **`sources` carries no `id`.** § 5.1 says it SHOULD be present when the body
cites the source, and no body here cites with a `[^id]` footnote — measured
0 across the corpus in `docs/okf-nokkelinventar.md`. When one does, the key
0 across the corpus by an earlier key inventory. When one does, the key
is available and unwritten.
---
@ -415,6 +382,5 @@ those bundles.
| docx paragraphs ≠ markdown lines | `<w:p>` count from `word/document.xml` against `extract_text(...).split("\n")` |
| spreadsheet rows against the workbook | `<row>` elements from `xl/worksheets/sheet1.xml` against `source_units(...).covering(0, len(text))` |
| the known-positive page walk | an independent script re-extracting and bisecting page offsets: pages 12–13, matching the emitted `source_pages` |
| the viewer's route table | `curl` the SPA shell and its one JS asset; `path:"…"` and query-key extraction |
| the payload members | `json.load` over a shipped K2 payload |
| suite, lint, types | `pytest -q`, `ruff check src tests tools`, `ruff format --check`, `mypy --strict src` |

88
docs/2026-09-08-rangeringsbom-sammensatte-ord.md
View file

@ -1,7 +1,8 @@
# The compound-word ranking miss: a degenerate signal, not a matcher, 2026-09-08
A consumer asked the N500 bundle about `vann- og frostsikring` in a subsea
tunnel and got eight excerpts, none of which covered the subject; the bundle
A consumer asked a 270-concept third-party bundle about a subject named by a
hyphenated pair of nouns (*A*- og *B*) and a compound adjective, and got eight
excerpts, none of which covered the subject; the bundle
covers it on 16 concepts. The consumer behaved as the contract asks — it
answered `[sourced-not-sufficient]` and named its denominator — so the miss is
the ranker's (`docs/2026-09-08-claude-code-skill-vilkaarlig-bundle.md` § 5).
@ -28,9 +29,9 @@ reproduced on this machine before any after-column was read:
| # | known-positive | source | measured here |
|---|---|---|---|
| 1 | files matching `frostsikring` | C1 § 5 | **17**, case-insensitively, `index.md` among them |
| 2 | files matching `vann- og frostsikring` | C1 § 5 | **10** |
| 3 | files matching `undersjøisk` | C1 § 5 | **13** |
| 1 | files matching the compound noun *B* | C1 § 5 | **17**, case-insensitively, `index.md` among them |
| 2 | files matching the whole pair *A*- og *B* | C1 § 5 | **10** |
| 3 | files matching the compound adjective | C1 § 5 | **13** |
| 4 | the miss itself: covering concepts in the cut | C1 § 5 | **0** |
| 5 | hit@8 over the six published questions | 2026-09-07 § hit@k | **5 of 6**, ranks `1,1,1,1,1,–` |
| 6 | the three requirement lookups, rank of 8 | O2c § 3 | **1, 1, 1** |
@ -54,8 +55,8 @@ matched.
- **`o200k_base` token counts.** No encoder is installed and the
single-dependency rule does not admit one. Every figure is in `BUDGET_UNIT`
plus `sha256`.
- **Whether the rule helps a third corpus class.** Two were measured (the three
N-bundles and K2). A corpus whose document prior is neither degenerate nor
- **Whether the rule helps a third corpus class.** Two were measured (three
third-party bundles and K2). A corpus whose document prior is neither degenerate nor
coarse is unmeasured, and the flag's default reflects that.
---
@ -63,18 +64,17 @@ matched.
## 1. The reproduction, with the command and the numbers
The order asks for the question C1 § 5 used, quoted. **It is not in that
document** — § 5 describes the question in prose ("about water and frost
protection in a subsea tunnel") and quotes no line. The question below is the
document** — § 5 describes the question in prose and quotes no line. The question below is the
order's own wording, and it is marked as such rather than presented as C1's:
```
python3 tools/okf_consume.py ~/repos/vegnormal-okf/build/ferdig/n500-2024 \
--question "Hvilke krav gjelder vann- og frostsikring i undersjøisk tunnel?" \
python3 tools/okf_consume.py <consumer build>/corpus-c \
--question "Hvilke krav gjelder <A>- og <B> i <adjective> <place>?" \
--out /tmp/o4-baseline.json
```
270 concepts, 8 delivered, 262 withheld. **0 of the 16 covering concepts** — the
concepts whose title or body says `frostsikring`, case-insensitively — is in the
concepts whose title or body says *B*, case-insensitively — is in the
cut. The best of them fuses to **rank 14**.
The eight delivered, and the three covering concepts nearest the cut, with the
@ -82,12 +82,12 @@ per-signal ranks that decide the fusion:
| | fused | body | title+id | doc prior | lexical | title |
|---|---|---|---|---|---|---|
| delivered 1 | 1 | 17 | 6 | 4 | 6 | Krav 4.3—1 Tunnelportaler |
| delivered 2 | 2 | 24 | 1 | 50 | 7 | Krav 3.2—2 Utslipp av vann fra tunnel |
| delivered 3 | 3 | 138 | 5 | 1 | 4 | Krav 5.3.6—1 Fjernstyrte bommer |
| **covering 1** | **14** | **6** | 33 | **130** | **7** | Krav 8.4.2—1 Frostisolering med PE-skum eller XPS |
| covering 2 | 39 | 34 | 30 | 121 | 6 | Krav 9.3.2—2 Fundament, omfylling og frostsikring |
| covering 3 | 44 | 25 | 101 | 63 | 5 | Krav 1.1—2 Generelle bestemmelser |
| delivered 1 | 1 | 17 | 6 | 4 | 6 | an unrelated requirement |
| delivered 2 | 2 | 24 | 1 | 50 | 7 | a requirement naming *A* |
| delivered 3 | 3 | 138 | 5 | 1 | 4 | another unrelated requirement |
| **covering 1** | **14** | **6** | 33 | **130** | **7** | a requirement on *B* |
| covering 2 | 39 | 34 | 30 | 121 | 6 | a second requirement on *B* |
| covering 3 | 44 | 25 | 101 | 63 | 5 | general provisions |
The best covering concept answers **7 of 7** question tokens — more than any
delivered one — and leads the body signal at rank 6. It is not invisible to the
@ -99,13 +99,12 @@ matcher. It is behind on one signal: the document prior, at rank 130.
Three measurements, each of which alone rules the tokeniser out as the site:
1. **The compound is already resolved.** `normalise("vann- og frostsikring")`
returns `('vann', 'frostsikring')` on HEAD. The hyphen splits, `og` falls
1. **The compound is already resolved.** `normalise` on the hyphenated pair
returns its two nouns on HEAD. The hyphen splits, `og` falls
below `MIN_TOKEN_LENGTH`. There is no coordination left for a rule to undo.
2. **The prefix rule already bridges the inflections.**
`tokens_match('frostsikring', 'frostsikringen')`,
`('vann', 'vannsikring')` and `('undersjøisk', 'undersjøiske')` are all
`True` at `MIN_SHARED_PREFIX = 4`.
2. **The prefix rule already bridges the inflections.** `tokens_match` on
each question token against its definite, compounded and plural forms in the
bundle is `True` at `MIN_SHARED_PREFIX = 4`.
3. **The covering concept already scores the maximum.** Its lexical overlap is
7 of 7 question tokens. A rule widening the tokeniser cannot raise 7.
@ -114,8 +113,8 @@ Recorded as a test rather than as a memory:
green on HEAD and labelled a characterisation.
One thing the same measurement DID surface, and it is not fixed here: the
prefix rule over-matches in the other direction. `undersjøisk` matches `under`,
so its document frequency is **159 of 270** — a term that should be one of the
prefix rule over-matches in the other direction. The compound adjective matches its first element, a
common four-letter preposition, so its document frequency is **159 of 270** — a term that should be one of the
rarest in the question reads as one of the commonest. That belongs to the
rarity weight (`--rarity-weight`, already shipped and already off), and this
session was allowed one rule.
@ -130,17 +129,17 @@ declared tie-break, `(-score, concept_id)`, then orders that group **by
`concept_id`**, and the fusion reads the result as though it were a
measurement.
On N500 the concept ids are UUIDs and the document prior is degenerate:
On that bundle the concept ids are UUIDs and the document prior is degenerate:
| corpus | documents | distinct prior values | concepts sharing one value |
|---|---|---|---|
| N500 | 2 | **2** | **269 of 270** |
| that bundle | 2 | **2** | **269 of 270** |
| K2 | 39 | 6 | — |
So on N500 the third signal contributed alphabetical UUID order, spread from
So there the third signal contributed alphabetical UUID order, spread from
`1/61` down to `1/329`. That spread is worth more than several positions on
the two signals that did the measuring, and it is what put a concept answering
7 of 7 tokens behind concepts sharing only `tunnel` and `vann`.
7 of 7 tokens behind concepts sharing only two common words with the question.
**The rule.** `tie_shared_rank`: every concept a signal scores equally takes
that score group's **first** position, so a signal that separates nothing
@ -164,9 +163,9 @@ Flag `--tie-shared-rank`, off and on. Every row measured in one run,
| row | off | on | verdict |
|---|---|---|---|
| N100 lookup `Krav 3.3.1-13`, rank of 8 | 1 | **1** | holds |
| N200 lookup `Krav 2.9.2-12`, rank of 8 | 1 | **1** | holds |
| N500 lookup `Krav 10.2-2`, rank of 8 | 1 | **1** | holds |
| corpus-a requirement-number lookup, rank of 8 | 1 | **1** | holds |
| corpus-b requirement-number lookup, rank of 8 | 1 | **1** | holds |
| corpus-c requirement-number lookup, rank of 8 | 1 | **1** | holds |
| hit@8 over the six published questions | **5 of 6** | **4 of 6** | **FALLS** |
| — per-question rank | `1,1,1,1,1,–` | `1,–,1,1,1,–` | row 2 lost |
| S7 control question, default command | `c759a657…` 171 614 B | `5f4cd665…` 145 228 B | moves (on purpose) |
@ -175,9 +174,8 @@ Flag `--tie-shared-rank`, off and on. Every row measured in one run,
| **the O4 miss: best covering rank** | **14** | **3** | **closes** |
| **the O4 miss: covering concepts in the cut** | **0 of 16** | **2 of 16** | **closes** |
The two concepts the rule delivers are `Krav 8.4.2—1 Frostisolering med PE-skum
eller XPS` and `Krav 9.3.2—2 Fundament, omfylling og frostsikring for
ledninger`.
The two concepts the rule delivers are the two requirements on *B* in the
table in § 1.
**Default stays OFF**, by the order's own rule: it requires every row to hold
*and* the miss to close. The miss closes; the hit@8 row does not.
@ -213,10 +211,10 @@ read by accident) and from this working tree with both new flags omitted:
| payload | digest |
|---|---|
| N100 lookup | `a2a91d8b99157a0c`, 58 082 B |
| N200 lookup | `0c828a5f8c432864`, 144 932 B |
| N500 lookup | `ad2b35a556278553`, 41 364 B |
| N500 O4 miss | `d4a75828c147d9e9`, 47 056 B |
| corpus-a lookup | `a2a91d8b99157a0c`, 58 082 B |
| corpus-b lookup | `0c828a5f8c432864`, 144 932 B |
| corpus-c lookup | `ad2b35a556278553`, 41 364 B |
| corpus-c O4 miss | `d4a75828c147d9e9`, 47 056 B |
| S7 control | `c759a657dbb41d71`, 171 614 B |
| K2 price question | `f17c8725ee6bbe1a`, 167 360 B |
@ -238,9 +236,9 @@ are not**, and that is what moved the decision:
| | without title | with title |
|---|---|---|
| N500 payload | 41 364 B | 57 023 B (**+37.9 %**) |
| N100 payload | 58 082 B | 88 251 B |
| N200 payload | 144 932 B | 215 148 B |
| corpus-c payload | 41 364 B | 57 023 B (**+37.9 %**) |
| corpus-a payload | 58 082 B | 88 251 B |
| corpus-b payload | 144 932 B | 215 148 B |
| K2 bookkeeping (629 concepts, everything not an excerpt) | ~75 KB | **122 704 B** |
The last row is the one that decided it. `skills/okf-consume/SKILL.md` — a
@ -250,7 +248,7 @@ The last row is the one that decided it. `skills/okf-consume/SKILL.md` — a
immediately: at 629 concepts the bookkeeping is already **past** the limit. The
generated skill's own figure moved from a 1206-concept ceiling to 762.
So `--withheld-titles`, off by default. On, every one of N500's 262 withheld
So `--withheld-titles`, off by default. On, every one of corpus-c's 262 withheld
entries carries its title; off, the entries are `{concept_id, rule}` exactly as
before, and the eight contract-checked payloads all exit 0 either way.
@ -260,7 +258,7 @@ before, and the eight contract-checked payloads all exit 0 either way.
- **14 of the 16 covering concepts are still outside the cut** with the flag on.
Rank 3 is not rank 1, and `k = 8` over 270 concepts is a narrow door.
- **The over-matching prefix.** `undersjøisk` reads as 159-of-270 common. One
- **The over-matching prefix.** The compound adjective reads as 159-of-270 common. One
rule per session; this was not it.
- **hit@8 row 6** — unchanged, not delivered either way, both before and after.
- **The default.** Two flags shipped this session and both are off. Whether

48
docs/2026-09-08-sjeldenhetsvekt.md
View file

@ -4,7 +4,7 @@ The previous session made a requirement number VISIBLE to the consumption
pre-pass and could not make it WORTH more than a common word: `_overlap`
returns a count, so `10.2-2` scored exactly what `krever` scored, and the gold
concept stayed `below_k` on all three bundles
(`docs/2026-09-08-kravnummer-tokenisering.md` § 8). This session weights each
(measured in the previous session, whose report is retired). This session weights each
hit by the token's rarity in the bundle and measures what that buys.
It buys one of the three requirement lookups and a large move on K2. It does
@ -29,7 +29,7 @@ was read:
| # | known-positive | source | measured here |
|---|---|---|---|
| 1 | gold fused rank per bundle | previous session § 4 | **96, 9, 35** |
| 2 | concepts sharing the gold's lexical score on N500 | previous session § 8 | **70 of 270** |
| 2 | concepts sharing the gold's lexical score on corpus-c | previous session § 8 | **70 of 270** |
| 3 | considered / withheld / delivered | consumer § 3 | 446/438/8, 1133/1125/8, 270/262/8 |
| 4 | gold position in `withheld` | consumer § 3.1 | 119 of 438, 204 of 1125, 66 of 262 |
| 5 | K2 S7 control, two commands | previous session § 5 | `2ae46f68…` 169 573 B, `8aaf8558…` 172 588 B |
@ -113,11 +113,11 @@ list, and the payload's own verdict from `build_payload`. Command:
```
python3 tools/okf_consume.py <bundle> \
--question "Hva krever Krav <n> i <normal>? Gjengi det sentrale vilkåret." \
--question "Hva krever Krav <n> i <dokument>? Gjengi det sentrale vilkåret." \
[--rarity-weight]
```
| | N100:2023 | N200:2024 | N500:2024 |
| | corpus-a | corpus-b | corpus-c |
|---|---|---|---|
| concepts | 446 | 1 133 | 270 |
| gold fused rank, weight **off** | 96 | 9 | 35 |
@ -129,23 +129,23 @@ python3 tools/okf_consume.py <bundle> \
| payload `sha256`, on | `ba09a56a…` | `403ee989…` | `1758f6ca…` |
| `okf_contract_check` | exit 0, 14 rules, 0 findings | idem | idem |
**One of three delivers.** N200's gold moves from withheld position 204 of
1 125 to delivered at rank 8. N500 does not move at all. N100 moves the wrong
**One of three delivers.** corpus-b's gold moves from withheld position 204 of
1 125 to delivered at rank 8. corpus-c does not move at all. corpus-a moves the wrong
way, by seven positions.
---
## 3. Why N100 got worse: the matcher makes the identifier common
## 3. Why corpus-a got worse: the matcher makes the identifier common
`df` under the matcher, against `df` under exact token equality:
| bundle | identifier | `df` under the matcher | `df` exact | `w(identifier)` | `w('sentrale')` |
|---|---|---|---|---|---|
| N100 | `3.3.1-13` | **135 of 446** | 1 | 1.195 | **3.798** |
| N200 | `2.9.2-12` | 17 of 1 133 | 1 | 4.199 | 4.835 |
| N500 | `10.2-2` | 5 of 270 | 1 | 3.989 | 3.519 |
| corpus-a | `3.3.1-13` | **135 of 446** | 1 | 1.195 | **3.798** |
| corpus-b | `2.9.2-12` | 17 of 1 133 | 1 | 4.199 | 4.835 |
| corpus-c | `10.2-2` | 5 of 270 | 1 | 3.989 | 3.519 |
The identifier is unique in all three bundles. It is *common* in N100 only
The identifier is unique in all three bundles. It is *common* in corpus-a only
because `tokens_match` compares four leading characters, so `3.3.1-13` matches
every requirement number beginning `3.3.` — 135 of them. The weight then does
what it was asked to do and ranks a common adjective (`sentrale`, borne by 10
@ -160,18 +160,18 @@ here.
---
## 4. Why N500 did not move: RRF consumes ranks, and the gold already led
## 4. Why corpus-c did not move: RRF consumes ranks, and the gold already led
The gold's rank on each of the three fused signals, and how many concepts each
signal reorders:
| bundle | signal ranks, off | signal ranks, on | positions moved per signal |
|---|---|---|---|
| N100 | 39 / 186 / 127 | 42 / 184 / 127 | 122 / 367 / **0** of 446 |
| N200 | 8 / 4 / 212 | **4** / 10 / 212 | 37 / 277 / **0** of 1 133 |
| N500 | **1** / 137 / 74 | **1** / 137 / 74 | **0** / 127 / **0** of 270 |
| corpus-a | 39 / 186 / 127 | 42 / 184 / 127 | 122 / 367 / **0** of 446 |
| corpus-b | 8 / 4 / 212 | **4** / 10 / 212 | 37 / 277 / **0** of 1 133 |
| corpus-c | **1** / 137 / 74 | **1** / 137 / 74 | **0** / 127 / **0** of 270 |
On N500 the gold is **already rank 1** on the title-and-id signal, by counting,
On corpus-c the gold is **already rank 1** on the title-and-id signal, by counting,
before any weighting. Weighting raises its score from 3.000 to 3.989 and its
rank from 1 to 1. RRF was chosen precisely because it **consumes ranks only**
(`RRF_K`'s comment says so), which makes it invariant to any monotone transform
@ -184,7 +184,7 @@ bundles (0 positions moved, three times).
**The general statement, and it is a limit rather than a complaint:** a rarity
weight can only move a fused rank where it reorders some signal. Where the
identifier appears in exactly one field and already leads that field, no
weighting scheme inside the signals can do anything. Closing N500 means
weighting scheme inside the signals can do anything. Closing corpus-c means
changing the fusion — giving a signal a weight, or letting a strong single
signal carry more than 1/(60+1) — and that is a different mechanism from this
one. It is named here and not built.
@ -221,7 +221,7 @@ of excerpts and costs 6.3 % more bytes.
The order's rule: default ON if the weight wins on all four corpora with no K2
control falling; behind a flag otherwise, naming what fell and by how much.
**It does not win on all four. N100's gold fused rank moves 96 → 103, a loss of
**It does not win on all four. corpus-a's gold fused rank moves 96 → 103, a loss of
seven positions**, on a question where the gold is not delivered either way. So
the weight ships as `--rarity-weight`, **off by default**.
@ -234,9 +234,9 @@ frozen copy of `56c1205` (`git archive` into a scratch tree, `PYTHONPATH` and
**8 of 8 payload digests identical.** No consumer's bytes move because this
change exists.
What the flag buys the caller who sets it, in one line each: N200's requirement
What the flag buys the caller who sets it, in one line each: corpus-b's requirement
lookup goes from withheld to delivered; K2's priced sheet goes from candidate
rank 10 to 2; N100 and N500 are unchanged in verdict and worse and equal in
rank 10 to 2; corpus-a and corpus-c are unchanged in verdict and worse and equal in
rank respectively.
---
@ -244,16 +244,16 @@ rank respectively.
## 7. What did NOT close
- **Two of the three requirement lookups are still `below_k`**, and the target
the order set — rank ≤ 8 on all three — is **not met**. It is met on N200.
- **N500 cannot be closed by any weighting inside the signals** (§ 4). The next
the order set — rank ≤ 8 on all three — is **not met**. It is met on corpus-b.
- **corpus-c cannot be closed by any weighting inside the signals** (§ 4). The next
mechanism is a change to the fusion, not to the weight, and it has no
measurement behind it yet.
- **N100 needs the matcher, not the weight** (§ 3). `MIN_SHARED_PREFIX = 4`
- **corpus-a needs the matcher, not the weight** (§ 3). `MIN_SHARED_PREFIX = 4`
makes a unique identifier read as 135-of-446 common. That is the same defect
`54a0bc2` § 1 named a year of sessions ago in another form, and it is now
measured on the case it costs most.
- **The 34 % skewed deletion in the orphan gate** is untouched; different axis.
- **"Delivered" is not "read".** N200's gold is now in the payload. Whether a
- **"Delivered" is not "read".** corpus-b's gold is now in the payload. Whether a
model then answers the engineer's question is the consumer's measurement.
---

14
docs/2026-09-09-k3-runde10-rtf-prefiks-shim-ruff.md
View file

@ -49,7 +49,7 @@ is bold text.
### 1.2 The grammar is markdown, not `rtf`
The converter already emits that title as `**Kravspesifikasjon for
tunnelbelysning**` — a whole bold line in the SAME markdown every office row
kjellerbelysning**` — a whole bold line in the SAME markdown every office row
produces. Bold inside a paragraph is `text **bold** text`, which is not a whole
line. That is the discriminator, and it means no `rtf`-only heading grammar
exists — the same shape of decision as the PDF font reader emitting ATX.
@ -148,11 +148,11 @@ and it is one position, not twelve.
| word | equality | prefix | share | bundle |
|---|---|---|---|---|
| `under` (control) | **79** | 172 | 38.0 % | default-20260912 (453) |
| `undersjøisk` | 0 | 172 | 38.0 % | " |
| `under…` | 0 | 172 | 38.0 % | " |
| `bilateral` | 0 | **400** | **88.3 %** | " |
| `standhaftig` | 0 | 219 | 48.3 % | " |
| `under` (control) | 76 | 174 | 27.7 % | Arm B (629) |
| `undersjøisk` | 0 | 174 | 27.7 % | " |
| `under…` | 0 | 174 | 27.7 % | " |
Round 9's figures reproduce exactly on the bundle they were taken on
(`-20260911`, 436 concepts: 171 and control 79).
@ -165,7 +165,7 @@ and not an independent source.
### 2.2 Three candidates, all three failed on the same row
| candidate | `bilateral` | `standhaftig` | `undersjøisk` | control | ranks, default | ranks, Arm B |
| candidate | `bilateral` | `standhaftig` | `under…` | control | ranks, default | ranks, Arm B |
|---|---|---|---|---|---|---|
| shipped, floor 4 | 400 | 219 | 172 | 172 | `1,1,1,1,1,–` | `1,1,1,1,1,–` |
| floor 5 | 0 | 56 | 171 | 171 | **`2`**,1,1,1,1,– | **`–`**,1,1,1,1,– |
@ -196,7 +196,7 @@ threshold on length or coverage separates them.
`pris` occurs as a token on its own; `bila` and `stan` never do.
| bundle | `bilateral` | `standhaftig` | `undersjøisk` | control | ranks |
| bundle | `bilateral` | `standhaftig` | `under…` | control | ranks |
|---|---|---|---|---|---|
| default-20260912 | 400 → **0** | 219 → **56** | 172 → 162 | 172 | `1,1,1,1,1,–` |
| Arm B | 512 → **0** | 235 → **33** | 174 → 160 | 174 | `1,1,1,1,1,–` |
@ -205,7 +205,7 @@ Both acceptance criteria met at once. **ON by default**, opt-out
`--no-stem-prefix`, pinned in `tests/test_default_bundle_pin.py` with its own
known-negative on the shipped bytes.
**What remains is a different answer, not a ceiling.** `undersjøisk` still
**What remains is a different answer, not a ceiling.** `under…` still
reaches 162 because it shares `under`, which IS a word here — a productive
Norwegian prefix. A rule cutting that would cut the behaviour
`MIN_SHARED_PREFIX` exists for. The residual is a statement about Norwegian.
@ -231,7 +231,7 @@ patches a binding the implementation never reads, which is why the alias stays.
**What the consumer actually asked for is also done**, and it was a line rather
than a code change: the shim docstring and a `CHANGELOG.md` note under 0.7.0
(no new version, tag unmoved) both say to read the module back out of
`sys.modules` after `exec_module`. Sent to `vegnormal-okf` by `coord-send`.
`sys.modules` after `exec_module`. Sent to the reporting consumer by `coord-send`.
## 4. Ruff

4
docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md
View file

@ -242,7 +242,7 @@ changed every bundle's first concept with no hit@8 number behind it would be
the move this round refused to make everywhere else. The measurement it needs
is named in STATE.
## 6. vegnormal-okf FUNN 1 and FUNN 2
## 6. A downstream consumer's FUNN 1 and FUNN 2
Both reproduced against a known positive first, then fixed.
@ -271,7 +271,7 @@ editing commons' contract.
## 7. What still stands, unchanged
- **The prefix over-matches the other way** (round 3): `undersjøisk` matches
- **The prefix over-matches the other way** (round 3): `under…` matches
`under`, df 159 of 270. It belongs to `--rarity-weight` and was not touched;
this round changed `document_scores` and not `rarity_weights`.
- `--pdf-headings font` and `font-reserve` stay off, on round 4's and round

6
docs/2026-09-09-k3-runde9-restene.md
View file

@ -229,12 +229,12 @@ class, and asserts the `unmeasured` set is now empty rather than dropping it.
## 4 The prefix overmatch, re-measured
Round 5's `df 159 of 270` was measured on the N500 bundle and has not been
Round 5's `df 159 of 270` was measured on a retired 270-concept bundle and has not been
re-measured since. It is not reproduced here — that bundle is a different one —
but the mechanism is, on both K2 bundles, **with a known-positive control run
first because a query that cannot find would report zero either way**:
| bundle | N | `df(undersjøisk)` prefix rule | `df(undersjøisk)` equality | control `df(under)` |
| bundle | N | `df(under…)` prefix rule | `df(under…)` equality | control `df(under)` |
|---|---|---|---|---|
| `K2-bundle-default-20260911` | 436 | **171** | 0 | 79 |
| `K2-bundle-20260903` | 629 | **174** | 0 | 76 |
@ -274,7 +274,7 @@ named "pdfplumber"`. The green command is
## 6 A finding reported, not fixed: the `tools/okf_consume.py` shim
`vegnormal-okf` reported that the v0.7.0 shim broke a caller using
A downstream consumer reported that the v0.7.0 shim broke a caller using
`importlib.util.spec_from_file_location`. Reproduced here, with numbers: such a
caller holds a module object carrying **4** public names against the
`sys.modules` entry's **90**, and `build_payload` is absent. `sys.modules[__name__]

317
docs/2026-09-10-k3-runde12-pdf-outlines.md
View file

@ -1,317 +0,0 @@
# K3 round 12: the PDF shipped a finished structure index and the build discarded it unopened
`okf build` reads a PDF's page text and recovers boundaries from the shape of
that text. It never opens the file's `/Outlines` bookmark tree. On a document
whose publisher wrote one, that is the difference between guessing the
structure and being told it.
This round adds `--pdf-outline`, **off by default**, which cuts a PDF at the
boundaries its own bookmark tree declares. It is a segmentation arm: the
extracted text is byte for byte the same either way.
Every number below carries its denominator. The two that matter most are that
the fasit is the **publisher's own** structure, and that it describes **one
document, one format, one publisher** (§ 7).
## 0. Three divergences from the numbers this round was handed
The order and the incoming FYI both rest on `vegnormal-okf`'s SK2 measurement.
Its headline reproduces here **exactly** -- 1 967 of 2 761, depth 1 `0 of 28`,
false positives 163 of 2 182, 794 of 794 misses present in the text -- both
against SK2's own bundle and against one rebuilt on this repository's HEAD
(§ 2). Three characterisations around it do not survive re-measurement.
**(a) "No okf concept directory is a bare number" is a property of that
BUNDLE, not of okf.** The claim is what makes 81 of 2 761 fasit rows
unreachable for SK2's pair key, and all 28 depth-1 rows sit in that group. It
is true of the arm-off bundle: **0 of its directories** is a bare integer.
It is false of okf: with the arm on, **57 of 2 738** directories are, because
`_split_outline_title` reads the same bare one-or-two-digit grammar Arm D
already carries. The rows were unreachable because the boundaries were never
opened, not because the id form forbids it.
**(b) The corrected instrument does NOT lift the arm-off number.** The
correction -- accept a hit when the concept's own normalised title equals the
fasit's, alongside the pair key -- was expected to show that "depth 1: 0 of 28"
was partly an instrument artifact. Measured on the arm-off bundle it changes
**1 967 to 1 967** and depth 1 stays **0 of 28 in both forms**. So on the
shipped default that row was a real segmentation hole measured with an
instrument that *could* have reached it, and not only an artifact. The
correction earns its keep on the arm-**on** bundle, where it moves 2 737 to
2 759 and depth 1 from 12 of 28 to 28 of 28 (§ 3).
**(c) The 145 truncated titles and the 65 contents duplicates reproduce as
described**, and both are closed as a side effect rather than as separate work
(§ 3). They are re-characterisations of SK2's own numbers, not new
measurements on new material.
## 1. Where the gap is
- 794 of 794 missed STS titles have their heading text **present** in the
extracted text; 0 of 794 are absent. The gap is segmentation, not
extraction.
- The file carries an `/Outlines` tree of **2 763 nodes**. All **2 763 of
2 763** resolve to a page index, and all **2 763 of 2 763** carry an `/XYZ`
destination with a non-null `top`. After `re.sub(r"\s+","",s).lower()`,
**2 761 of 2 761** STS titles match a node exactly; without the
normalisation, 1 of 2 761.
- **Three holes, not one**, and the tree closes all three: the boundary that is
never opened (794 of 2 761); the title cut short at the PDF's line break
(145 of the 163 "false positives" and 81 of the 794 "misses" are this -- a
bookmark title does not come from the page, so it is complete); and the table
of contents becoming a second set of concepts (65 directories carrying both a
contents copy and a body section under one id -- a bookmark's destination
page *is* the body).
## 2. Reproduced before anything was built on it
| row | SK2 (`de9564d`) | this HEAD, arm off |
|---|---|---|
| boundaries | 1 967 of 2 761 (71.2 %) | **1 967 of 2 761 (71.2 %)** |
| depth 1 … 7 | 0/28 · 59/118 · 410/500 · 913/1141 · 551/868 · 31/97 · 3/9 | **identical** |
| false positives | 163 of 2 182 | **163 of 2 182** |
| misses present in the text | 794 of 794 | **794 of 794** |
| directories with two concept files | 132 of 2 050 | **132 of 2 050** |
| bare-integer directories | — | **0** |
`git log de9564d..HEAD -- extract.py propose.py` names exactly one commit
(`732f84d`, the HTML extractor), so the PDF path was unchanged between the
bundle SK2 measured and this tree -- and the rebuild confirms it rather than
resting on it.
## 3. The arm, measured
Built with `okf build … --pdf-outline`, same input, same epoch stamp.
| row | arm off | arm on (SK2's instrument) | arm on (corrected) |
|---|---|---|---|
| **boundaries** | 1 967 of 2 761 | 2 737 of 2 761 (99.1 %) | **2 759 of 2 761 (99.9 %)** |
| depth 1 | 0 of 28 | 12 of 28 | **28 of 28** |
| depth 2 | 59 of 118 | 116 of 118 | **118 of 118** |
| depth 3 | 410 of 500 | 499 of 500 | **500 of 500** |
| depth 4 | 913 of 1 141 | 1 140 of 1 141 | **1 141 of 1 141** |
| depth 5 | 551 of 868 | 866 of 868 | **868 of 868** |
| depth 6 | 31 of 97 | 95 of 97 | **95 of 97** |
| depth 7 | 3 of 9 | 9 of 9 | **9 of 9** |
| **titles identical to the STS title after `NORM`** | — | — | **2 761 of 2 761** |
| **false positives** | 163 of 2 182 | 25 of 2 762 | **3 of 2 762** |
| **directories with two concept files** | 132 of 2 050 | **2 of 2 738** | (same) |
| concepts in the front matter (page < 68) | 72 of 2 182 | **2 of 2 762** | (same) |
| concepts | 2 182 | **2 762** | (same) |
| K3-9: ends in a digit | 16 of 99 | 99 of 99 | **99 of 99** |
| K3-9: opens with a numbering token | 1 967 of 2 739 | 2 737 of 2 739 | **2 737 of 2 739** |
| K3-9: unnumbered | 0 of 22 | 0 of 22 | **22 of 22** |
The gate was **2 700 of 2 761** with the corrected instrument; the result is
2 759, and the ceiling is 2 761.
**The distance from the ceiling, named row by row rather than left as a
remainder.** Two fasit rows are missed, and they are the *same two nodes* the
false-positive column counts from the other side: `36.7331 48V strømforsyning
PoE` and `36.7332 24V strømforsyning`. The publisher's bookmark for each omits
the numbering token the STS `<title>` glues on, so the concept lands with no
number directory and the pair key cannot reach it. Both sections exist as
concepts with the right body; only the address differs. The third false
positive is the tree's own **root node**, `R761 Prosesskoden` -- one candidate,
the document's own title.
**The two remaining directories with two concept files are not contents
duplicates.** They are `11` and `12`, and each holds two genuinely distinct
declared sections that share a number: `11 ARBEIDSSTIKNING, TEKNISK KONTROLL`
(page 183) beside `11 Oversikt over prosesser` (pages 99–181), and the same
shape at 12. The 65 contents-copy pairs are **0**. The collision that remains
is the publisher reusing a number in two chapters, which is a different
mechanism and is not closed by anything here.
**Depth 6 stays 95 of 97 in both instrument forms.** The publisher's own tree
and the publisher's own STS disagree about four nodes: the tree's levels 2..8
hold 28/118/500/1141/**872**/**93**/9 against the STS depths 1..7 at
28/118/500/1141/**868**/**97**/9. The mapping is level minus one on five rows
and off by four on two of them, so the level is read from the tree and reported
as declared rather than normalised (`OutlineMark.level`). Reporting a depth-1
row as `28 of 28` because a `+1` shift was *assumed* would be the same failure
`0 of 28` was.
### 3.1 The bridge, both routes, with denominators
A bookmark states a page and a y position; a candidate needs a line index.
**2 706 of 2 761 bookmarks share a destination page with another bookmark**, so
the page alone is never a cut point.
- **Y route (primary).** `page.extract_text_lines()` carries a `top` per line.
It requires the line splitting to be the one `page.extract_text()` produced,
which is an assumption and is therefore CHECKED per page: measured **701 of
701 pages identical**, and the check ships, so a page where it fails uses the
title route instead of guessing.
- **Title route (fallback).** The normalised bookmark title, searched in the
destination page's own lines: **2 762 of 2 763** resolved.
- **They disagree on 0 of 2 762.** Swept over tolerances 0, 1, 2, 3, 5, 8 and
12 pt, the agreement is **flat at 2 762 with 0 disagreements from 0 to 8 pt
and collapses to 924 at 12 pt** (the document's line spacing). Because 0 pt
is already exact, the rule ships with **no tolerance constant at all** -- the
first line at or below the destination.
- The first naive rule -- *nearest* line -- was wrong on **1 840 of 2 762**
nodes, one line early every time, because `/XYZ top` sits above the heading.
It is recorded because it produced a plausible-looking bundle.
**Provenance.** For **2 762 of 2 762** concepts, the first page of
`source_pages` equals the bookmark's own destination page. `source_units`
still indexes exactly the string `extract_text` returned: the arm moves
boundaries, never units. A PDF concept's address still names the page range
its span covers -- now the range the publisher's own bookmark opened.
### 3.2 Consumption
Fasit presence was validated against the bundle **before** any rank was read.
| | arm off | arm on |
|---|---|---|
| fasit present in the bundle | 4 of 7 | **7 of 7** |
| hit@1 · hit@8 · hit@50 (default `k`) | 1/6 · 2/6 · 4/6 (SK2) | **3/6 · 5/6 · 5/6** |
| hit@1 · hit@8 · hit@50 (`k` = 50) | — | **3/6 · 5/6 · 6/6** |
| payload slots taken by the front matter | 0 of 8 (SK2) | **0 of 8, all eight questions** |
S3, S4 and the known-positive existed in no bundle before, because the section
that answers them never became a boundary.
**The known-positive is not delivered at default `k`, and that is stated rather
than rounded.** `1 Bruksområder for Prosesskoden` is a real 1 827-byte concept
now, reading as the answer to the question asked of it, and it ranks **13** --
delivered at `k` = 50, not at `k` = 8. SK2 reported it as `null` because the
boundary did not exist. The segmentation half of that row is closed; the
ranking half is not, and no part of this round touches ranking.
`S5` (`84.3 Armering`) is the one row still missing at `k` = 8: rank 11 at
`k` = 50, with `Armering av betonghvelv` at rank 1. `Armering` occurs nine
times in this structure, and that collision is in the ranking, not the bridge.
### 3.3 Cost
| | arm off | arm on |
|---|---|---|
| wall (`/usr/bin/time -l real`) | 119.22 s | **183.31 s** (+53.8 %) |
| peak RSS | 3 410 038 784 B = **3 252 MiB** | 3 409 002 496 B = **3 251 MiB** |
| times the PDF's pages are parsed | 1 | 1 |
| times the file is opened | 1 | 2 |
`/usr/bin/time -l` on macOS reports maximum resident set size in **bytes**, not
KB; both figures above are the byte value converted. Neither is a 2×
increase. SK2 measured the arm-off build at 180.50 s on their machine against
119.22 s here, so the honest comparison is the pair measured on one machine.
The extra open reads the bookmark tree and the line geometry; the page text
comes from the memoised extraction pass and is not re-rendered.
**No new dependency.** `pdfminer.six` 20260107 already ships under
`pdfplumber` in the existing `[extract]` extra, and exposes
`PDFDocument.get_outlines()`. `uv.lock` and `[extract]` are untouched;
`pypdf` is not introduced.
## 4. Nothing else moved, and it is measured rather than argued
- **`~/okf-test/dokumenter`, three ways: identical.** A build from a frozen
`git archive` export of `8e19179`, a build on this tree with the arm off, and
a build on this tree with the arm **on** -- `diff -r` is empty between all
three. 52 md / 26 concepts, 0 of 5 rejected. Both PDFs there carry no
`/Outlines`, which is what makes the third comparison a real control.
- `okf project` is still byte-equal to `okf build` (`diff -r` empty), and the
new flag's argparse default and Python signature default agree -- the test
that compares every same-typed parameter covers it.
- `okf skill` + `okf consume` + `okf check --skill … --payload …`:
**conformant, 15 rules over 8 excerpts and 18 withheld entries, 0 findings.**
- The pinned K2 bundle's concept count, per-row hit@8 ranks and all three
known-negatives are unchanged: the arm is off by default and the arm-off code
path is byte-identical.
- **The 12-position reference is untouched by the shipped default** and stays
`pdf` 7 of 8, `docx` 3 of 3, in both readings of the `xlsx` fasit
(`|F|` = 11 → 10 of 12, `|F|` = 12 → 11 of 12). This is byte-identity, not a
re-score.
**Reach, and the number that decides whether the default should move.** Of the
eight `pdf` positions on the reference, **1 of 8** carries a usable bookmark
tree (position 9, 11 nodes). One more (position 0) carries a tree whose two
nodes both fail to resolve -- the drop-and-count branch firing on a real
document rather than only on a fixture. The other six carry none.
With the arm on, position 9 goes from **4 concepts to 11**, and the eleven are
the document's own chapter names (`Innledning`, `Organisering av prosjektet`,
`Risikoforhold og spesifikke tiltak`, `Fremdriftsplan`, `Vedlegg 1: …`) --
seven chapters the extracted text never showed as headings at all. The
operator's cross for that position says a unit is a top-level chapter and the
free text says "ville jeg hatt hvert kapittel". Whether 11 is nearer that than
4 depends on an `|F|` the sheet does not define, because the sheet's own
"document's disposition" section could only list the four the text showed.
**That is an operator question and this round does not answer it.**
## 5. What did NOT move
No HTML change, no OCR, no consumption change, no new fasit, no new corpus, no
`.xml`/NISO-STS extractor, no change to Arm D (`--outline-run`/`--outline-gate`
are a text heuristic over numbered lines and are a different mechanism
entirely), no re-rating of the reference positions, no `--bold-title` move, no
version bump, no tag, no push. `uv.lock` and `[extract]` untouched. The
default did not move.
## 6. The tests, and what each one is for
`tests/test_pdf_outline.py`, over two hand-laid fixtures built by
`tests/fixtures/make_fixtures.py` -- no library's output, objects laid out by
hand, xref offsets computed from the emitted bytes (`outlined-krav.pdf`
2 073 B, `outline-broken-dest.pdf` 1 014 B).
- **Three levels, not two.** A two-level tree cannot tell "the level the node
declares" from "one below the root".
- **The line, not the page.** The fixture's last page carries four lines and
its second bookmark points at the third, so a bridge that resolved to the
page and stopped would fail visibly instead of looking like it worked.
- **No `/Outlines` is not an error.** `PDFNoOutlines` yields no marks, no
count, and identical behaviour.
- **An unresolvable `/Dest` is dropped and counted**, never fabricated and
never a refusal.
- **An empty mark list leaves every other rule untouched** -- "this file has no
index" and "propose nothing" must not collapse into one answer.
- **A declared section with no prose of its own survives.** The orphan check
asks whether anything stands under a candidate's first line: the right
question for a heuristic's guess, the wrong one for a publisher's
declaration. Measured: **683 of 2 762** marks are container sections, and
applying the check scores **2 079** instead of 2 759.
## 7. Honesty limits
- **The instrument was run in two forms and the two answer different
questions.** SK2's pair key `(numbering token, normalised residual title)`
cannot reach **81 of 2 761** fasit rows -- 59 with a dotless token, 22
unnumbered -- and all 28 depth-1 rows are among them. Every percentage above
says which form it rests on, and the gate was set at **2 700**, not at the
ceiling 2 761, precisely because of those 81. The corrected form lives in a
scratch copy of SK2's script; `vegnormal-okf` was read and never written, and
the correction is offered to them, not applied on their behalf.
- **The fasit is the publisher's own STS structure.** "The right boundary" is
therefore defined by the publisher, not by what is useful to a reader. A
reader might well want 2 761 sections merged into far fewer.
- **The denominator 2 761 is CHOSEN.** The document has 7 715 `<sec>`; 4 954 of
them carry a letter-point label (`a)`, `c)`) inside a process description and
0 of those 4 954 is a process number. Measuring against 7 715 would answer a
different question.
- **Matching is exact after normalisation.** A boundary set one word away from
the source title counts as a miss. That is what makes 2 759 credible and also
what makes it a floor rather than a satisfaction score.
- **One document, one format, one publisher.** R761 is a guideline with 2 739
of 2 761 titles numbered in a strict hierarchy. A running prose document
would measure something else, and this round has no such document with a
fasit.
- **A bookmark tree is the publisher's CLAIM about its own structure**, not a
measurement of the document. A PDF with a stale, incomplete or wrongly
pointing tree carries that error straight into the segmentation, and R761
says nothing about how common that is. One of the eight reference PDFs here
already carries a tree whose destinations do not resolve.
- **SK2's own figures** (1 967, 163, 794, 180.50 s, 3 253 MiB, 1/6 · 2/6 · 4/6)
are `vegnormal-okf`'s documentation and stand as premises wherever they were
not re-measured here. The rows in § 2 were re-measured.
- **The 12-position reference is n = 12**, stratified 8 `pdf`, 3 `docx`,
1 `xlsx`, so every per-type fraction is a fraction of 8, 3 or 1. Both
readings of the `xlsx` fasit are reported, never one.
- **"Delivered" is not "answered."** hit@k asks whether the gold concept
reached the payload, and nothing here evaluates an answer.
- Position 0's `|F|`, the `xlsx` fasit's `|F|`, the `pptx` fasit, re-rating the
reference positions and routing between bundles on `--id` remain blocked on
an **operator decision**, not on code. This round moved none of them.

361
docs/2026-09-10-k3-runde14-deklarert-struktur-tar-ruten.md
View file

@ -1,361 +0,0 @@
# K3 round 14: the extractor reached the ceiling, the build did not
Round 13 gave `.xml` a reader that emits **2 761 of 2 761** ATX lines for
R761's titled `<sec>` elements, with the text preserved exactly. At **shipped
defaults** the build then delivered **23 concepts and 15 of 2 761 boundaries**.
The distance between the two was not extraction and not a missing rule: it was
two steps that run *after* the reader and judge a publisher's own declaration
as if it were a guess.
This round closes that distance at the defaults, not behind a flag.
| row | before (HEAD `9d5ebc3`) | after (`ee12f64`) | denominator |
|---|---|---|---|
| concepts, shipped defaults | 23 | **2 761** | — |
| declared sections that became a concept | 15 | **2 761** | 2 761 |
| concept title == source `<title>` after NORM | — | **2 761** | 2 761 |
| concepts matching no declaration | 8 | **0** | 2 761 |
| `a)`-points that became their own concept | 0 | **0** | 4 954 |
| markdown table blocks | 10 | **10** | 10 `<table-wrap>` |
| directories holding two concept files | 14 (at `--no-unit-fold`) | **2** | 2 738 |
| hit@1 · hit@8 · hit@50 (k=50) | 0/6 · 0/6 · 0/6 | **3/6 · 5/6 · 6/6** | 6 |
| known-positive rank | 1 of 17 delivered of 23 | **1** | 1 |
Every figure in this file was measured in this session. Where a figure differs
from what the order stated, the measured one is used and the difference is
named.
## 1. What was reproduced first, and what did not reproduce
`HEAD` was `9d5ebc3`, `git status --short` empty, `git ls-remote origin
refs/heads/main` = `9d5ebc3c29b417f3bae90458ee041ea1b10a19ea`. **STATE.md said
15 unpushed commits and a remote at `6ff84d7`; the remote is at HEAD, so the
push list was already empty when this session opened.** That is the one premise
of the order's own package that did not hold, and it holds in the safe
direction.
Suite in the working tree before any change: **1 567 passed, 1 skipped**.
`ruff --version` 0.16.6, `ruff check .` and `ruff format --check .` clean over
116 files, `mypy src` clean over 21 files. Install channel
`uv sync --frozen --extra extract`. `pyproject.toml` version `0.7.0`, tag
`v0.7.0` on `1260fac` — no bump, no tag in this round.
The baseline was rebuilt from a frozen `git archive` export of `9d5ebc3`
(never from the editable tree) and reproduced the order's numbers exactly:
23 concepts, **15 of 2 761** under the corrected instrument key and **0 of
2 761** under the shipped one, 8 false positives of 23, and hit@1/8/50 =
**0/6 · 0/6 · 0/6** with the known-positive at rank 1 of 17 delivered.
### The instrument, and both of its faults are real
`experiments/sk2_grenser.py` sets `cnorms = ckeys` and uses the `literal` set
on the `[ARTEFAKT]` line **alone**, so HOVEDTALL, depth, FALSKE POSITIVE,
DELMENGDE, FORSIDE and K3-9 all score the **pair** key `(concept directory,
normalised title)`. `experiments/hitk_sk2.py` pairs `fasit_key` (which splits
the numbering token off) with `excerpt_key` (which does not). Confirmed by
reading both files. Both were corrected in a scratch copy under `/tmp`, run
with `PYTHONDONTWRITEBYTECODE=1` from a scratch cwd with the inputs symlinked
in; nothing was written into `vegnormal-okf` and no instrument code was copied
into this repository. The correction in both is the same: **accept a hit when
the pair key matches OR when the concept's own normalised title equals the
fasit's `norm`.** Every row below is reported in both forms.
**The two forms cannot both reach 2 761, and that is a property of the
instrument rather than of any bundle.** The pair key requires the concept
title to equal the fasit title *without* its numbering token; the literal form
requires it to equal the fasit title *with* it. **2 739 of 2 761** fasit rows
carry such a token, so for those rows the two forms demand mutually exclusive
strings. No bundle can satisfy both. The gate below is therefore stated on the
corrected (union) form, with the shipped form reported beside it and its
shortfall accounted for row by row.
`sk2_grenser.py`'s "present in the extracted text" row reads
`build/sk2-pagetext.json`, which is the **PDF's** page text and not the
extraction under measurement. It was given a haystack of this repository's own
extracted XML text instead (`SK2_HAYSTACK`), and the row is labelled with which
haystack it used.
## 2. The mechanism, and why a new rule rather than a flag
`find_candidates` returns early when `outline_marks` is given: the text
heuristics, both gates and Arm F's fold are all skipped, and the orphan check
is not applied. That is why the PDF bookmark arm reaches 2 762. An STS
`<sec><title>` is the same class of declaration — stronger, in fact, since the
publisher states it as an element rather than as a bookmark over a typeset
page — but until now it arrived as `rule:heading`, indistinguishable from a
heading a converter recovered from a font size, and was judged by:
- **the orphan check**, which removed **710 of 2 761**. Verified as the only
mechanism rather than assumed: 710 of 710 removed headings are followed
immediately by another heading, and **0 of 2 051** delivered ones are. They
are container sections. `propose.py`'s own comment already named the outcome:
the check "judges whether a GUESS was a heading — a question a publisher's
own tree has already answered, and one that deletes every container section
if asked".
- **Arm F** (`--unit-fold`, a build default since 2026-09-08), which took
**2 066 more**, 2 089 → 23. On a process code whose sections are almost all
`14.121 Langsgående sikring T1 oppgjort etter lengde`, the shallowest
repeated heading level is 1, so every deeper level folds into its chapter.
The change is **one new rule constant reached from one row**:
- `extract.xml_outline(name, data)` returns the marks the XML reader wrote
itself. There is **no bridge**: the reader appended the line it is naming, so
the index is exact, there is no tolerance constant and there is no
`unresolved` bucket. This is the difference from `pdf_outline`, whose naive
nearest-line rule was wrong on 1 840 of 2 762 nodes.
- `propose.RULE_XML_SECTION` (`rule:xml-section`) is its own name in
`RULE_NAMES` and in `_ORPHAN_EXEMPT`, so an artifact still says whether a
boundary came from a bridged bookmark, an element the reader transcribed, or
a heading somebody guessed.
- `find_candidates` gained one keyword, `outline_rule`, defaulting to
`RULE_PDF_OUTLINE` so the PDF arm is reached by exactly the call it was
reached by before.
- `build_plan` chooses the route by the **row**
(`DECLARED_STRUCTURE_IDS = {"xml"}`), never by the text. The same markdown
arriving from a `.md` file is still a guess and still carries `rule:heading`
— pinned by a test.
The broader variant — a flag exposing the orphan check to every type — was not
taken. It was not needed and it would put twelve unit-worksheet positions at
risk for one row's benefit.
`--pdf-outline` stays a flag and stays off; nothing in this round moved it.
That remains an operator question on a reach of **1 of 8** reference PDFs.
## 3. The gate, in both instrument forms
Built with the installed console script and no arm flag at all:
```
okf build <the STS folder> --bundle <scratch> --bundle-id r761xml --okf-version 0.2
```
**2 761 segments proposed, 1 plan, 0 documents with no boundary, 109
unreadable (the images), `merged + coded rejections = 110; N = 110`.**
Against the source's own elements — every titled `<sec>`, its `<label>` reduced
to a directory and its `<title>` to a title, which is what `_segment_path`
does:
| row | after | denominator |
|---|---|---|
| declared sections that became a concept (directory **and** title) | **2 761** | 2 761 |
| declarations with no concept | **0** | 2 761 |
| concepts matching no declaration | **0** | 2 761 |
| concept title == source `<title>` after NORM | **2 761** | 2 761 |
| `a)`-points (label, no title) that became a concept | **0** | 4 954 |
| contiguous pipe-table blocks | **10** | 10 `<table-wrap>` |
Against the 2 761-row fasit, through the corrected instrument:
| row | CORRECTED | AS-SHIPPED | denominator |
|---|---|---|---|
| titles that became a boundary | 2 759 (99.9 %) | 2 737 (99.1 %) | 2 761 |
| depth 1 | 28 | 12 | 28 |
| depth 2 | 118 | 116 | 118 |
| depth 3 | 500 | 499 | 500 |
| depth 4 | 1 141 | 1 140 | 1 141 |
| depth 5 | 868 | 866 | 868 |
| depth 6 | 95 | 95 | 97 |
| depth 7 | 9 | 9 | 9 |
| false positives | 2 (0.1 %) | 24 (0.9 %) | 2 761 |
| pages 68–266 | 566 (100.0 %) | 561 | 566 |
**The two shortfalls are both instrument artefacts and both are accounted
for, not waved past.**
- The **2** rows the corrected form misses are `36.7331 48V strømforsyning
PoE` and `36.7332 24V strømforsyning` — the only two of 2 761 whose
`<title>` itself begins with a digit. The fasit glues `<label>` and `<title>`
with no separator, so its own key reads `36.733148` as the number and `V
strømforsyning PoE` as the title. Both sections **are** in the bundle, with
the right number and the right title, which is why the same two appear as the
corrected form's only two "false positives". Measured against the source
rather than against the glued string, they are hits: **2 761 of 2 761**.
- The **24** the shipped form misses are those two plus the **22** rows with no
numbering token at all (`Forord`, `Hovedprosess 81 Løsmasser`, …). Those
sections get no directory, so the pair key compares the per-document
directory name against `""` and can never match. That is the structural
proof in §1, in numbers: 22 of 22 unnumbered rows score 0 as-shipped and 22
of 22 score under the corrected form.
## 4. Consumption, cross-arm, and cost
hit@k over the eight published questions at `k = 50`, corrected instrument
(the as-shipped form agrees on every row here, because a concept title no
longer carries its numbering token):
| | before | after |
|---|---|---|
| hit@1 | 0/6 | **3/6** |
| hit@8 | 0/6 | **5/6** |
| hit@50 | 0/6 | **6/6** |
| known-positive rank | 1, of 17 delivered of 23 concepts | **1**, of 43 delivered of 2 761 |
Row by row after: S1 rank 3, S2 rank 1, S3 rank 1, S4 rank 1, S5 rank 11,
S6 rank 3, KP rank 1, KN no rank (it has no fasit). **S5 is rank 11 on the PDF
arm as well** — unchanged, and a ranking question rather than a segmentation
one.
**Cross-arm, and the number moved a long way.** Round 13 measured 2 022 common
concept ids — 96.8 % of the XML bundle and 73.2 % of the PDF arm's. Re-measured
after this change: **2 761 common ids, 100 % of the XML bundle's 2 761 and
2 761 of 2 762 (99.96 %) of the PDF arm's.** The single PDF-only concept is
`r761-prosesskoden.md`, the front-matter node the bookmark tree carries and the
STS body does not. The 740 that were PDF-only in round 13 were the
orphan-check bucket; they are now on both sides.
**Cost, and cost figures are machine-dependent — direction and order of
magnitude only.** `/usr/bin/time -l` reports BSD **bytes**, converted here.
XML build 7.6 s → **17.4 s**, peak RSS **145 MiB**. PDF arm with
`--pdf-outline`, built once: **209.2 s**, peak RSS **3 251 MiB**. The XML
channel remains roughly an order of magnitude cheaper in time and more than an
order of magnitude cheaper in memory, on the same machine in the same session.
## 5. Two-file directories: round 13's 14 were a defect, and they are gone
Round 13 reported 14 directories holding two concept files on the XML arm and
classified them as the publisher reusing a title. **Measured this round, that
classification is wrong.** Rebuilt from a frozen `9d5ebc3` export with
`--no-unit-fold` (2 089 concepts), the 14 are pairs like `35.54 Nøkkelsafe for
brannvesen` / `Nøkkelsafe for brannvesen`: the first is a line from the
document's **own contents listing**, a plain `<p>` that happens to read as
`<number> <title>` and was proposed as a boundary by the text grammar; the
second is the real `<sec><label>35.54</label><title>…</title>`. They were false
positives of the text route, not the publisher's doing.
On the shipped default after this change there are **2**, of 2 738 numbered
directories:
- `11` — `ARBEIDSSTIKNING, TEKNISK KONTROLL` and `Oversikt over prosesser`
- `12` — `Prosesser med tekster` and `RIGG, BYGNINGER OG GENERELLE
DRIFTS-OMKOSTNINGER`
These are **identical to the PDF arm's 2 of 2 738**, same two directories and
same two title pairs: the publisher reuses a section number for two distinct
sections. **0 is not reachable here without inventing an id**, so 2 is the
floor and it is named rather than counted.
The per-document root directory holds 22 concept files. That is not a
collision: those 22 sections carry no numeric `<label>`, so no directory is
formed, and each filename is distinct.
## 6. Generic XML: 0 plans and a failed build, and it is not an XML defect
The order to round 13 required "1 plan, exit 0, never 0" for XML that is not
STS. Measured:
```
okf build <folder with one generic-feed.xml> --bundle <scratch> ...
okf-propose-segments: nothing to propose for generic-feed.xml — the mechanical rules found no boundary.
okf build: proposed 0 plan(s); 1 document(s) with no boundary; 0 unreadable
okf build: FAILED - no segmentation plans in ... -- exit 2
```
**The known-positive that decides what this is:** a folder holding one `.txt`
with prose and no headings gives **exactly the same three lines and exit 2**.
This is general `okf build` behaviour for any structureless document, not an
`.xml` defect. Extraction works — 0 unreadable, the text is there — and the
proposer has nothing to propose. Round 13's own test asserts
`propose.find_candidates(text) == []`, which is the opposite of what its order
demanded, and the test is right; the divergence was simply never stated, and it
is stated here and in the CHANGELOG.
**Should such a folder fail the build at all?** The gate exists because a run
that replays zero plans would emit a flat bundle and call it a success. That is
a good reason and it is not being removed. The question is whether "0 plans,
N documents with no boundary, **0 unreadable**" is the same state as "0 plans
because nothing could be read". The deciding number:
| reference corpus | documents | plans | no boundary | unreadable | outcome changes if split? |
|---|---|---|---|---|---|
| K2 `trinn1` | 43 | > 0 | some | 4 | no |
| the five-document folder | 5 | 4 | 1 | 0 | no |
| the STS delivery | 110 | 1 | 0 | 109 | no |
| the 828-file HTML export | 828 | 828 (recorded, not re-measured here) | — | — | no |
**0 of 4.** The distinction only fires on a corpus where *every* document
yields no boundary, and none of the four is such a corpus. Separating the two
states would therefore change no outcome anyone has measured, and it is left
unchanged — as a measured result with its denominator, not as an omission. The
message a caller sees already distinguishes them by the counts it prints on the
same line.
## 7. Nothing else moved, and it was measured
- **The five-document folder is byte-identical.** `diff -r` between a tree
built from a frozen `9d5ebc3` export and one built from the same export with
only `src/` and `tests/` replaced by this round's code: **0 differences**.
52 md files, 26 concepts, 0 of 5 rejected. That folder holds **0 `.xml`
files**, so it is a real known-negative.
- **`okf project` is still byte-equal to `okf build`** on that folder;
`diff -r` empty.
- **The pinned K2 bundle is unchanged**: `tests/test_default_bundle_pin.py`
7 passed, 453 concepts, per-row ranks `(1, 1, 1, 1, 1, 5)`, and the
`--no-tie-shared-rank` known-negative still reproduces the fall it names.
- **The 43-document reference corpus is byte-identical.** Rebuilt in full with
this round's code and compared against the pinned
`K2-bundle-default-20260912`, which was built before the change:
**`diff -r` = 0 differences**, 865 md files on both sides, same rejection
codes (`extractor_empty_pdf` 1 of 43, `extractor_unknown` 3 of 43). The
corpus holds **0 `.xml` files of 86** (66 pdf, 10 docx, 4 xlsx, 2 zip, 2 smc,
2 doc), so the unit worksheet's 12 positions could not be reached by this
change — and now that is measured on the bytes rather than argued from the
code path. Build wall time 874 s, 20.3 s per file.
- **The PDF arm is unchanged**: built once with `--pdf-outline`, **2 762
segments**, exactly as round 12 and round 13 measured it.
- Suite after: **1 575 passed, 1 skipped** (1 567 + 8 new). `ruff check src
tests tools` and `ruff format --check .` clean over 117 files, `mypy src`
clean over 21 files.
## 8. The instrument correction is reported, not assumed
Round 13 corrected `hitk_sk2.py` in its scratch copy with the same fix as
`sk2_grenser.py`, but its report only said the fasit set was validated
"independent of key form". Verified this round: with the **shipped** matcher
the XML bundle scores `rang: null` on every question — including the
known-positive whose `topp_1` **is** the fasit — and 0/6 · 0/6 · 0/6; with the
correction it scores 3/6 · 5/6 · 6/6. **A reader who runs the shipped runner
gets zeros and concludes round 13 was wrong.** The two instruments corrected in
scratch are `experiments/sk2_grenser.py` and `experiments/hitk_sk2.py`, and the
correction in both is: accept a hit when the pair key `(directory, normalised
title)` matches **or** when the concept's own normalised title equals the
fasit's `norm`. This is sent to `vegnormal-okf` as a proposal for their
instrument, not applied there.
## Honesty limits
- **The denominator for `.xml` is ONE file, from ONE publisher, in ONE
schema.** Closing the build's ceiling does not change that. `.xml` as a file
type is far wider than NISO-STS; a document in any other schema keeps its
text in document order and gets no structure at all. The `_EVIDENCE` class
for the row stays `measured` — the file is a publisher's own delivery,
written for their purposes long before any lookup of ours, and the fasit is
theirs — and the row now carries that limit in its own comment.
- **The ceiling here is structural, not recovered.** 2 761 of 2 761 on this arm
is not comparable to a similar percentage on the PDF arm, which had to bridge
from (page, `/XYZ` top) to a line index and reports what did not bridge. This
arm has nothing to bridge and nothing to lose in bridging; the number is
cheaper than it looks.
- **The denominator 2 761 is a CHOICE.** The document carries 7 715 `<sec>`, of
which 4 954 are lettered points with a `<label>` and no `<title>`. Counting
those as sections would give a different fraction of a different question.
- **The generic fallback is measured on fixtures, not on a corpus.** The `.txt`
known-positive is one hand-written file; the four-corpus table in §6 counts
outcomes, not documents that would land differently.
- **The unit worksheet is n = 12 positions and contains no `xml` row.** Its
`pdf` 7 of 8 and `docx` 3 of 3 stand here by byte identity on the default
path, never by a re-score.
- **Cost figures are machine-dependent.** Nothing above should be compared
against a number from another machine except by direction and order of
magnitude.
- **"Delivered" is not "answered".** hit@50 = 6/6 says the gold document
reached the payload at k = 50; it says nothing about whether a reader would
answer the question from it. hit@8 is 5/6 and hit@1 is 3/6.
- **hit@8 is not neutral with respect to a rule that changes how many distinct
documents a payload holds.** This round changed segmentation inside one
document, so that particular caveat does not bite here, but the metric is the
same one and the caveat travels with it.
- The report for round 13 is dated `2026-09-11` in its filename while its
commit is dated 2026-09-10. This file uses the real date.

6
docs/2026-09-10-k3-runde15-bundle-mismatch.md
View file

@ -26,12 +26,12 @@ byte for byte, including the exact report lines.
Two real artefacts, both read and never written: a SKILL generated by
`okf skill` from the pinned K2 bundle (`k2-trinn1-20260903` at
`sha256-tree:18ae18ab...`, 453 concepts) and a payload from an unrelated corpus
(`vegnormal-n100-2023` at `sha256-tree:da6b8204...`, 446 concepts).
(a 446-concept bundle at `sha256-tree:da6b8204...`).
| # | pair | rc before | rc after | line after |
|---|---|---|---|---|
| A | skill(k2) x payload(vegnormal-n100) | 0 | **1** | `NOT conformant: 16 rules over 8 excerpts and 438 withheld entries, 1 findings` |
| B | unfilled template x payload(vegnormal-n100) | 0 | **1** | same denominators, 1 finding |
| A | skill(k2) x payload(unrelated corpus) | 0 | **1** | `NOT conformant: 16 rules over 8 excerpts and 438 withheld entries, 1 findings` |
| B | unfilled template x payload(unrelated corpus) | 0 | **1** | same denominators, 1 finding |
| C | skill(k2) x payload(k2 id, foreign ref) | 0 | **1** | same denominators, 1 finding, message names **`ref` only** |
| KP | skill(k2) x its own payload | 0 | **0** | `conformant: 16 rules over 8 excerpts and 445 withheld entries, 0 findings` |
| KN | skill(k2) x `{}` | 1 | **1** | `NOT conformant: 16 rules over 0 excerpts and 0 withheld entries, 9 findings` |

88
docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md
View file

@ -23,9 +23,9 @@ cases the number below is the one measured here.
| worktree | clean | clean, `uv.lock` untouched | `git status --short` |
| suite at HEAD | 1575 passed, 1 skipped | **1575 passed, 1 skipped**, 95.84 s | `pytest -q` |
| collected at HEAD | 1576 | **1576** | `pytest --collect-only -q` |
| R761 build | rc 0, 23.2 s, 2 737 dirs, 2 761 concepts | rc 0, **25 s**, 2 737 dirs, **2 761** concepts | `okf build inn-xml …` |
| R761 at default `k` | 3/6 · 5/6 · 5/6, KP 1 | **identical**, and S1 rank 4 · S5 not delivered · S6 rank 3 | `hitk_sk2.py` |
| R761 at `--k 50` | 3/6 · 5/6 · 6/6, KP 1 | **identical** | `hitk_sk2.py --k 50` |
| reference standard build | rc 0, 23.2 s, 2 737 dirs, 2 761 concepts | rc 0, **25 s**, 2 737 dirs, **2 761** concepts | `okf build inn-xml …` |
| reference standard at default `k` | 3/6 · 5/6 · 5/6, KP 1 | **identical**, and S1 rank 4 · S5 not delivered · S6 rank 3 | the consumer's hit@k script |
| reference standard at `--k 50` | 3/6 · 5/6 · 6/6, KP 1 | **identical** | the same script, `--k 50` |
**Deviation 1.** The order's table gives S1 rank **4**; that is its rank at
default `k`. At `--k 50` it is **3**. Both are measured here; the order states
@ -41,7 +41,7 @@ Every harness figure in § 3 is labelled `candidate rank`; every figure in § 4
Everything was measured in frozen `git archive` exports under
`/tmp/k3r16/{export,export2}`, each `uv sync --frozen --extra extract`, using
that export's own `.venv/bin/okf`. `~/repos/vegnormal-okf` and
that export's own `.venv/bin/okf`. A downstream consumer repository and
`~/okf-test/dokumenter` were read and never written.
## 1 The diagnosis, per miss
@ -52,7 +52,7 @@ The third is the document prior.
**The third signal carries no information on this bundle, and that is measured
rather than inferred.** `document_score` is keyed on
`concept_id.split("/", 1)[0]`, which is the source document; R761 is one XML
`concept_id.split("/", 1)[0]`, which is the source document; the reference standard is one XML
file, so the prior takes **one distinct value over all 2 761 concepts**. With
`tie_shared_rank` on, every concept receives the same constant from it. The
order's hypothesis is CONFIRMED. Its evidence — that `--no-tie-shared-rank`
@ -63,43 +63,45 @@ So on this bundle the ranking is decided by two coverage counts, and the
question is which of them turns each miss over. Decomposed per candidate, with
the question tokens each one answers:
**S1** — «Hvordan er Prosesskoden delt inn i hovedprosesser?», tokens
`(hvordan, prosesskoden, delt, inn, hovedprosesser)`.
**S1** — "how is the document divided into its main parts?", five tokens: an
interrogative, the document's own name, *divided*, a function word and the
plural of the main-part term.
| rank | concept | title cov | body cov | body tokens answered | bytes |
|---|---|---|---|---|---|
| 1 | `11/oversikt-over-prosesser` | 1 | 3 | prosesskoden, delt, hovedprosesser | 114 728 |
| 2 | `hovedprosess-81-l-smasser` | 1 | 3 | **hvordan**, prosesskoden, hovedprosesser | 2 103 |
| 3 | `hovedprosess-83-…` | 1 | 3 | — | 14 529 |
| 4 | `hovedprosess-84-betong` | 1 | 3 | **hvordan**, prosesskoden, hovedprosesser | 4 410 |
| **5** | **`2-1/hovedprosesser`** | 1 | **2** | prosesskoden, hovedprosesser | 1 047 |
| 1 | an overview section | 1 | 3 | name, *divided*, main-part term | 114 728 |
| 2 | one main part | 1 | 3 | **interrogative**, name, main-part term | 2 103 |
| 3 | another main part | 1 | 3 | — | 14 529 |
| 4 | a third main part | 1 | 3 | **interrogative**, name, main-part term | 4 410 |
| **5** | **the gold, titled with the bare main-part term** | 1 | **2** | name, main-part term | 1 047 |
The body signal turns it over, and the token it turns on is `hvordan` — an
interrogative pronoun. The title signal separates nothing: every one of these
The body signal turns it over, and the token it turns on is the interrogative
pronoun. The title signal separates nothing: every one of these
scores 1 and sits in a tie group thousands of concepts wide.
**S5** — «Hvilke krav stilles til armering i betongkonstruksjoner?».
**S5** — "which requirements apply to *X* in *Y*?", with *X* and *Y* both
nouns and *Y* a compound.
| rank | concept | title cov | body cov | body tokens answered | bytes |
|---|---|---|---|---|---|
| 1 | `84-4/betongst-p` | 1 | **5** | **hvilke**, krav, **stilles**, armering, betongkonstruksjoner | 19 658 |
| 2 | `34-43/armering-av-betonghvelv` | **2** | 3 | krav, armering, betongkonstruksjoner | 442 |
| 1 | a large neighbouring section | 1 | **5** | **interrogative**, requirements, **verb**, *X*, *Y* | 19 658 |
| 2 | a short section titled *X* plus a qualifier | **2** | 3 | requirements, *X*, *Y* | 442 |
| … | | | | | |
| **12** | **`84-3/armering`** | 1 | 3 | krav, armering, betongkonstruksjoner | 2 759 |
| **12** | **the gold, titled *X* alone** | 1 | 3 | requirements, *X*, *Y* | 2 759 |
Two mechanisms, not one. The rank-1 concept wins on `hvilke` and `stilles`,
both grammatical; its body is 7x the gold's, and a longer body can only reach
MORE distinct question tokens. The rank-2 concept wins on the TITLE, at
`betonghvelv ~ betongkonstruksjoner` through the four-character stem `betong` —
a longer title can only reach more too.
Two mechanisms, not one. The rank-1 concept wins on the interrogative and the
verb, both grammatical; its body is 7x the gold's, and a longer body can only
reach MORE distinct question tokens. The rank-2 concept wins on the TITLE, where
a compound in it shares a four-character stem with *Y* — a longer title can
only reach more too.
**S6** — «Hva omfatter inspeksjon av bruer og konstruksjoner?».
**S6** — "what does *Z* of *A* and *B* cover?", *Z* a noun.
| rank | concept | title cov | body cov | body tokens answered | bytes |
|---|---|---|---|---|---|
| 1 | `88-14/enkel-inspeksjon` | 1 | 4 | omfatter, inspeksjon, bruer, **konstruksjoner** | 446 |
| 2 | `81-6/masser-…-konstruksjoner-under-vann` | 1 | 3 | omfatter, bruer, konstruksjoner | 715 |
| **3** | **`88-1/inspeksjon`** | 1 | 3 | omfatter, inspeksjon, bruer | 585 |
| 1 | a section titled *Z* with a qualifier | 1 | 4 | *cover*, *Z*, *A*, ***B*** | 446 |
| 2 | an unrelated section | 1 | 3 | *cover*, *A*, *B* | 715 |
| **3** | **the gold, titled *Z* alone** | 1 | 3 | *cover*, *Z*, *A* | 585 |
Rank 2 is a TIE with the gold on both signals, broken by `concept_id` — the
declared tie-break, doing what it is declared to do because nothing else
@ -107,7 +109,7 @@ separates them.
**The order's reading of the pattern is confirmed and can be stated more
precisely.** The three golds carry the bare term as their title
(`Hovedprosesser`, `Armering`, `Inspeksjon`) and lose to a title that is that
and lose to a title that is that
term plus a qualifier, or to a much larger neighbour. The mechanism under all
three is one thing: **both signals measure how much of the QUESTION the
candidate answers, and nothing measures how much of the CANDIDATE the question
@ -121,7 +123,7 @@ defect seen from different sides.
**It is not a segmentation miss on this bundle, and the control says so.**
`bundle-xml/<doc>/2-1`, `/84-3`, `/88-1` and `/1` all exist as concept
directories. On the PDF arm (`build/sk2-bundle-default`, read only)
directories. On the PDF arm (`build/std-pdf-default`, read only)
`find . -maxdepth 2 -type d -name '22' / '53' / '1'` are all EMPTY — only
`22-1`, `53-1` … exist — so S3/S4/KP are segmentation misses there. That is the
known-positive proving the presence query can find a difference.
@ -150,7 +152,7 @@ that treats it. Length is not what is wrong with the body signal.
**Title precision as a signal — falsified on the control.** `|Q ∩ T| / |T|` over
the title alone reaches candidate hit@1 5/6 in one arrangement, and takes the
known-positive from rank 1 to **4** every time it does. KP's gold is titled
`Bruksområder for Prosesskoden` — a phrase, one of whose three tokens the
with a three-token phrase naming the document's scope, one of whose tokens the
question answers — so a ratio that rewards short titles demotes it. The fasit's
own comment says a KP below rank 1 means the instrument is broken, so this is
not a trade to make.
@ -199,18 +201,18 @@ that an identifier matches by equality alone.
reduction of the title, so counting it would let the same words qualify twice,
and on a single-document bundle the id carries a uuid that is in no question.
**Reach, measured, and it is narrow.** Group size per question on R761:
**Reach, measured, and it is narrow.** Group size per question on the reference standard:
| S1 | S2 | S3 | S4 | S5 | S6 | KP | KN |
|---|---|---|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 9 | 2 | **0** | **0** |
It fires on 6 of 8 questions there and on **0 of 6** K2 questions and **0 of 15**
N-bundle cells (§ 5). It never fires on the known-negative.
cells on three other bundles (§ 5). It never fires on the known-negative.
## 5 The row, and every other row measured
**R761, delivered ranks from `okf consume`, 2 761 concepts, one source document:**
**The reference standard, delivered ranks from `okf consume`, 2 761 concepts, one source document:**
| | before, default `k` | after, default `k` | before, `--k 50` | after, `--k 50` |
|---|---|---|---|---|
@ -233,16 +235,16 @@ own cause.
| K2 pinned default bundle, hit@8 ranks | 6 questions, 453 concepts | `(1,1,1,1,1,5)` | **identical**, payloads **byte-identical on 6 of 6** |
| Arm B bundle, hit@8 ranks | 6 questions, 629 concepts | `(1,1,1,1,1,5)` | **identical**, payloads **byte-identical on 6 of 6** |
| `tests/test_default_bundle_pin.py` | 7 tests | 7 passed | **7 passed**, file unedited |
| n100-2023 | 5 questions, 446 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| n200-2024 | 5 questions, 1 133 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| n500-2024 | 5 questions, 270 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| corpus-a | 5 questions, 446 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| corpus-b | 5 questions, 1 133 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| corpus-c | 5 questions, 270 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| 828-file HTML corpus | 828 documents | 828 plans, 0 unreadable, 6 015 md | **828 plans, 0 unreadable, 6 015 md**; `diff -rq` empty apart from `log.md`'s converter PATH (same pandoc 3.9) |
| `~/okf-test/dokumenter` | 5 documents | 26 concepts / 52 md | **26 / 52**, `diff -r` **EMPTY** |
| `okf project` vs `okf build` | same folder | byte-equal | **byte-equal**, `diff -r` empty |
| suite | — | 1575 passed, 1 skipped | **1582 passed, 1 skipped**, 1583 collected |
| `ruff check src tests tools` · `ruff format --check .` · `mypy src` | ruff 0.16.6 | clean | **clean** (118 files, 21 source files) |
**hit@k on N100/N200/N500 is NOT MEASURED, and the denominator is why:** this
**hit@k on the three other bundles is NOT MEASURED, and the denominator is why:** this
repository holds no gold set for those three bundles — 0 gold sets, not 0 hits.
What is measured there is that the payload does not change.
@ -253,7 +255,7 @@ What is measured there is that the payload does not change.
question; this round moves three. Three questions is not a retrieval
evaluation.
- **The fasit is another repository's and was not touched.** It was set in
`vegnormal-okf` on 2026-09-10 before any lookup ran, and this round read it
the consumer on 2026-09-10 before any lookup ran, and this round read it
and the instrument and wrote to neither.
- **KP and KN are CONTROLS, not results.** Neither enters hit@k. KP holding rank
1 is what says the instrument still counts; KN staying not-a-hit is what says
@ -265,11 +267,11 @@ What is measured there is that the payload does not change.
does not mean a fourth failure shape does not exist. Coverage is still
unnormalised; this rule adds a precision statement about the title and none
about the body.
- **The rule's reach outside R761 is ZERO in everything measured.** "No
regression on K2 and the three N bundles" is really "the rule never fires
- **The rule's reach outside the reference standard is ZERO in everything measured.** "No
regression on K2 and the three other bundles" is really "the rule never fires
there" — 0 of 21 cells — not "it fires and is harmless". A corpus whose
section titles are phrases rather than terms gets nothing from it.
- **S5's group is 9 concepts, all titled `Armering`**, and the fusion's own
- **S5's group is 9 concepts, all carrying the same one-word title**, and the fusion's own
order put the right one first. That is one case, not a property: where a
publisher reuses a title, this rule floats all holders and decides nothing
between them.
@ -278,5 +280,5 @@ What is measured there is that the payload does not change.
arithmetic in § 3 are harness figures, everything in § 4 and § 5 is not.
- **This is the fourth reading-side change that moves a payload with NO bundle
changing.** A consumer pinned to the previous excerpt order needs
`--no-title-covered`. Measured, that consumer is nobody on K2 or the N
bundles: their payloads are byte-identical either way.
`--no-title-covered`. Measured, that consumer is nobody on K2 or the three
other bundles: their payloads are byte-identical either way.

4
docs/2026-09-10-r1-release-v080.md
View file

@ -121,8 +121,8 @@ Three rows, each with its denominator, all run from the installed binary:
| `okf skill` + `okf consume` + `okf check` | `conformant: 15 rules over 8 excerpts and 18 withheld entries, 0 findings` | 15 rules, 0 findings |
The HTML build was run twice into separate scratch directories and the two are
`diff -rq`-identical. `~/okf-test/dokumenter` and `~/repos/vegnormal-okf` were
read only; no file under `build/860019-side-inn-100` has an mtime inside this
`diff -rq`-identical. `~/okf-test/dokumenter` and a downstream consumer repository were
read only; no file under `build/html-arm-100` has an mtime inside this
session, and the two modified files in that repo's worktree
(`STATE.md` 21:05, `experiments/resonnering/kjor_m2.sh` 21:11) belong to a
session running there in parallel, not to this one.

8
docs/2026-09-10-r3-release-v081.md
View file

@ -1,8 +1,8 @@
# R3: release v0.8.1
A patch release and nothing else. It carries round 16's ranking rule
(`--title-covered`) to a reproducible ref so `vegnormal-okf` can re-measure the
R761 product (R2b) against a tag rather than against a branch tip. No new
(`--title-covered`) to a reproducible ref so a downstream consumer can re-measure
its product (R2b) against a tag rather than against a branch tip. No new
functionality, no behaviour change in `src/`, no push.
Tag: `v0.8.1`, local and annotated, on `3daf983` (`chore(release): 0.8.1`).
@ -98,11 +98,11 @@ cd <export> && uv sync --frozen --extra extract
| suite in the export | 1573 passed, 1 failed, 9 skipped -- the export artefact |
| `okf project` on the five-document folder | **26 concepts of 5 documents, 52 md, 0 of 5 rejected** |
| HTML corpus | **proposed 828 plan(s); 0 with no boundary; 0 unreadable**, 828 of 828, 6 015 md |
| R761 XML | **2 761 concepts** of 2 761 declared sections at shipped defaults, no arm flag; 109 of 110 files are images and are coded `extractor_unknown` |
| reference standard, XML | **2 761 concepts** of 2 761 declared sections at shipped defaults, no arm flag; 109 of 110 files are images and are coded `extractor_unknown` |
| `okf check --skill ... --payload ...` | **conformant, 15 rules, 0 findings** |
Bundles were built to scratch. `~/okf-test/dokumenter` and
`~/repos/vegnormal-okf` were read and never written.
a downstream consumer repository were read and never written.
## 6 Honesty limits

285
docs/2026-09-11-k3-runde13-xml-sts.md
View file

@ -1,285 +0,0 @@
# K3 round 13: the file that IS the product had no reader
**Date:** 2026-09-11 · **Commits:** `3f8f03c` (red), `525d68e`, `324cd0b` (red),
`5bb3b04` (red), `567a63d`, `fc23844` (red), `f280b14` · **Base:** `e1f4faa`
## 1. What was measured, and against what
A zip a publisher's own viewer delivers as "Html" holds 0 html, 1 xml and 109
images. `okf build` on it gave **110 of 110 unreadable, 0 plans, exit 2**, with
`no extractor is registered for file extension '.xml'` -- reproduced here
before any code was touched. The conservation identity `merged + coded
rejections == N` was never written at all: the run aborts earlier on `FAILED -
no segmentation plans`, so the hole was not even visible as a count. After the
reader, that same run writes `merged + coded rejections = 110; N = 110`.
The one xml file is the whole product: R761 Prosesskoden:2025, the document
round 12 met as a 701-page PDF, in NISO-STS form. Measured here with
`xml.etree.ElementTree`, reproducing SK1 exactly:
| row | measured |
|---|---|
| `<sec>` total | 7 715 |
| `<sec>` with `<title>` | **2 761** |
| `<sec>` with `<label>` and no `<title>` | 4 954 |
| `<table-wrap>` | 10 |
| `<p>` | 12 528 |
| root element | `<standard>` |
| `<!DOCTYPE` / `<!ENTITY` | 0 / 0 |
| `<sec>`-nesting depth over the 2 761 titled | 28 · 118 · 500 · 1 141 · 868 · 97 · 9 |
That last row is **row for row identical** to the fasit built from the same
file, so the ceiling here is structural rather than computed: **2 761 of
2 761**, against the PDF arm's 2 759 of 2 761.
### Two premises in the material do not hold, and one instrument is uncorrected
- **`<title>` does not carry the number.** SK2's report and `hitk_sk2.py`'s
docstring say "STS glues the token into `<title>`". Measured: **2 of 2 761**
`<title>` strings begin with a digit. `<label>` carries `2.1`, `<title>`
carries `Hovedprosesser`; the fasit's `"2.1Hovedprosesser"` is SK2's own
fasit-BUILDER joining them and setting `label: null`. The fasit is right, the
account of where the number comes from is not.
- **One titled section has no label:** `Forord` (`sec-type="foreword"`, depth 1).
- **The shipped instrument carries the round-12 correction in NO block.** The
order stated that its headline, depth and false-positive rows use the
corrected key. Measured against `experiments/sk2_grenser.py` as it stands:
line 86 is `cnorms = ckeys` and the `literal` set is used only on the
`[ARTEFAKT]` line. The correction is absent everywhere, not just in the
K3-9/DELMENGDE/FORSIDE blocks. A scratch COPY was corrected in every block
(headline, depth, false positives, DELMENGDE, FORSIDE, K3-9), and the
haystack was made a parameter; both forms are reported side by side below.
The correction is proposed to `vegnormal-okf`, never applied there.
## 2. Step 0: a bookmark that shared a line left no trace
Round 12's arm lost one `/Outlines` node without counting it: **2 763 nodes in,
2 762 marks out, `unresolved` = 0**. Reproduced exactly. The lost node is
`SVV - Forside`, sharing line 0 with the tree's own root node `R761
Prosesskoden`. The cause is structural: marks are collected in a dict keyed on
the destination line index, so `setdefault` discards the second in silence.
**The rule was chosen by measurement.** Keeping both as sibling marks was run
through `find_candidates` on R761's own text: the candidate list goes 2 762 ->
2 763 and the FIRST candidate closes with an **empty span (0, 0)**, which the
orphan check deletes without a word. Keeping both loses the same node one step
later and adds a front-matter title that is in no fasit. So: first in tree
order wins, the loser is COUNTED, and the identity holds -- **2 763 = 2 762 + 0
+ 1** on R761 with the shipped function.
Known-negatives: `outlined-krav.pdf` and `outline-broken-dest.pdf` unchanged in
outcome and reporting `collided == 0`; `~/okf-test/dokumenter` byte-identical
against a frozen `e1f4faa` export; the pinned K2 bundle's concept count and
per-row ranks unchanged.
## 3. The reader
`.xml` is a **core** type beside `.html`, not an `[extract]` one: the parser is
stdlib, and putting it behind the extra would make a pure-stdlib file type
binary-dependent. `defusedxml` and `lxml` occur **0 times** in `uv.lock` before
and after.
The output grammar is MARKDOWN -- the same markdown the office rows and round
11's HTML row reach the proposer through -- and **`propose.py` was not
touched**. `<sec>` with a `<title>` becomes one ATX line carrying `<label>` +
space + `<title>` at its own nesting depth; `<sec>` with only a `<label>`
becomes a body line with the label in front; `<table-wrap>` becomes its label
plus one markdown table through this package's own `render_table`.
Two choices are measurements rather than preferences:
- **Inline by allowlist, block by default** -- the inverse of the HTML reader,
because block-versus-inline is a property of HTML and XML has no such
universal. The allowlist is load-bearing: that document carries 1 701
`<italic>` and 1 396 `<bold>` inside its prose, and breaking on them would
shred every paragraph.
- **The ATX ceiling is 6 and STS nesting reaches 7**, so the depth is CLIPPED,
not dropped: 9 of 2 761 sit at depth 7 and `#######` matches nothing. A
clipped heading still sets its boundary and states its nesting one level too
shallow; a dropped one loses the section. The fasit's depth row is the
source's own, so the clipping does not show there -- it shows in frontmatter
nesting. `tests/test_frontmatter_nesting.py` is unchanged and green.
**A DTD is refused unparsed**, with its own code. Measured on this interpreter
(3.14.0, `pyexpat.version_info` 2.7.3): an external SYSTEM entity is refused by
the stdlib and never fetched, but the amplification limit that stops a
billion-laughs comes from **libexpat >= 2.4.0 and not from Python** -- five
levels still expand -- while `pyproject.toml` requires only `>=3.10` and no
lockfile pins an interpreter. `XMLParser` exposes no `.parser` attribute on the
C accelerator, so the handler route is not portable. Refusing every DTD is a
guarantee about the code instead of about the machine, and it costs nothing
here: 0 of 1 file carries one.
`.xml` never routes through the converter, which is a second parser that would
never see that refusal. The existing fence test says so in one more line and
`test_every_office_row_names_its_reader` is untouched.
## 4. The gate, in both instrument forms
The extractor reaches the ceiling. **The build does not**, and the whole
distance is two proposer rules.
| row | measured | denominator |
|---|---|---|
| ATX heading lines emitted | **2 761** | 2 761 |
| text preserved, non-whitespace | **1 283 395 = 1 283 395, ratio 1.000000** | exact, not a percentage |
| provenance: unit table vs `extract_text` | starts == line offsets, numbers 1..N | 15 271 lines |
| concepts, shipped defaults | **23** | -- |
| boundaries, shipped defaults | **15 of 2 761 (0.5 %)** corrected · 0 of 2 761 pair-only | 2 761 |
| concepts, `--no-unit-fold` | **2 089** | -- |
| boundaries, `--no-unit-fold` | **2 065 of 2 761 (74.8 %)** corrected · 28 of 2 761 pair-only | 2 761 |
| by depth, `--no-unit-fold`, corrected | 15/28 · 98/118 · 430/500 · 935/1141 · 554/868 · 30/97 · 3/9 | 2 761 |
| by depth, pair-only | 0/28 · 2/118 · 5/500 · 10/1141 · 8/868 · 3/97 · 0/9 | 2 761 |
| false positives | **10 of 2 089 (0.5 %)** corrected · 2 061 of 2 089 pair-only | 2 089 |
| lettered points that became concepts | **0 of 4 954** | 4 954 |
| tables as table blocks | **10 of 10** (separator lines emitted) | 10 |
| unnumbered titles | 10 of 22 corrected · 0 of 22 pair-only | 22 |
| directories with two concept files | 14 numbered + the document folder | 2 089 |
**The pair-only column is an instrument artefact, not a segmentation result**,
and the two are separated rather than mixed. The pair key looks up `(concept
directory, normalised residual title)`; this bundle's concept TITLES carry the
numbering token, because `propose._ATX` keeps the whole title and fills
`number` in addition. The corrected form -- accept a match when the concept's
own normalised title equals the fasit's `norm` -- is the one that measures
segmentation here. A row the instrument's key cannot reach is not a boundary
okf failed to set.
**The 10 false positives are the ten table blocks**, named `Tabell linje N`.
The instrument's table regex reports 12 rather than 10 because two data rows
are followed by a row whose cells contain only dashes, which its pattern reads
as a separator; counting the separator lines this reader actually emits gives
**10 of 10**. SK2 measured **0 of 10** on the PDF path, and the PDF bundle
built here reproduces that: 0 table blocks over 0 files.
### The measurement that fells the first attempt, and the second attempt
**Attempt 1 -- the pure `_HTMLTextExtractor` pattern, shipped defaults: 23
concepts, 15 of 2 761.** Two mechanisms, both in `propose`, both decomposed
with a denominator and a control:
1. **The orphan check removes 710 of 2 761 (25.7 %).** Verified as the SOLE
mechanism rather than assumed: **710 of 710** removed headings are
immediately followed by another heading line, and **0 of 2 051** delivered
ones are. This is not the reader losing anything -- 2 735 of the 2 761
titled sections carry no prose of their own in the SOURCE, and the 710 are
exactly those whose children are themselves titled sections. `propose.py`'s
own comment names this outcome: the bookmark arm is exempt because the check
"judges whether a GUESS was a heading -- a question a publisher's own tree
has already answered, and one that deletes every container section if
asked". An STS `<title>` is that same declaration; it just arrives with
`rule:heading`, indistinguishable from a heading recovered from a converted
`docx`.
2. **Arm F (`--unit-fold`, a build default since 2026-09-08) removes 2 066
more, 2 089 -> 23.** Swept one flag at a time: every other default leaves the
count at 23 or 24, and `unit_fold=False` alone restores 2 089. On a process
code whose headings are almost all `14.121 Langsgående sikring T1 oppgjort
etter lengde`, Arm F reads the whole document as a unit sheet. That is the
arm doing what it was built for, on a document where it is wrong.
**Attempt 2 -- `--no-unit-fold`: 2 089 concepts, 2 065 of 2 761 (74.8 %).** The
gate of 2 761 is NOT reached, and the remaining 696 are the orphan bucket, each
one named in the instrument's output rather than left as a remainder.
The PDF arm reaches 2 762 because `find_candidates` **early-returns** when
`outline_marks` are given: the bookmark route bypasses the orphan check AND Arm
F together. The closing move is therefore to let a declared XML structure take
the route declared PDF structure already takes. That is a `propose.py` change
this round was fenced out of, and it is in DU SKAL BESVARE with the number that
decides it.
## 5. Consumption, and the cross-arm number
hit@k over SK2's six questions, fasit validated as present in the bundle FIRST
(**7 of 7**, on both bundles, independent of key form):
| | hit@1 | hit@8 | hit@50 | KP | KN |
|---|---|---|---|---|---|
| XML bundle (`--no-unit-fold`) | **3/6** | **5/6** | **6/6** | **rank 1** | no rank |
| PDF arm (`--pdf-outline`), same machine | 3/6 | 5/6 | 6/6 | rank 13 | no rank |
| SK2's arm-off reference | 1/6 | 2/6 | 4/6 | -- | -- |
No row is worse. Per row at k=50: S1 4 against 3, S2/S3/S4 1 against 1, S5 11
against 11, S6 3 against 3. **KP goes from rank 13 to rank 1** -- the one row
round 12 left open on the ranking side is closed on this channel, and it is
delivered at the default `k` as well. KN (`prosess 97.4`, fabricated) still
gets no rank on either bundle.
**Cross-arm:** **2 022 concept ids are shared -- 96.8 % of the XML bundle and
73.2 % of the PDF arm's.** Only in XML: 67. Only in PDF: 740, which is the
orphan bucket plus front matter. A named differing pair:
`1-bruksomr-der-for-prosesskoden` (XML) against `1/bruksomr-der-for-prosesskoden`
(PDF) -- the same section, because a bare `1` is not split by `_NUMBERED`
(which requires a dot) while the bookmark arm's `_split_outline_title` does
split it. Normalised TITLES intersect on only 38, for that same reason: the ids
agree and the titles do not.
## 6. Cost, and it is machine-relative
Both arms measured in the SAME session on the same machine, `/usr/bin/time -l`,
whose maximum-resident figure is in **BSD BYTES** -- converted here and said so.
| build | wall | peak RSS | concepts |
|---|---|---|---|
| XML, shipped defaults | 2.10 s | 107 180 032 B = **102.2 MiB** | 23 |
| XML, `--no-unit-fold` | 14.93 s | 114 040 832 B = **108.8 MiB** | 2 089 |
| PDF arm, `--pdf-outline` | 181.52 s | 3 408 691 200 B = **3 251.1 MiB** | 2 762 |
**Direction and order of magnitude, not a number against a number:** the XML
route is about **12x faster and about 30x smaller in peak memory** at a
comparable concept count. Absolute values are not comparable across reports --
round 12's child measured 183.31 s / 3 251 MiB and PM measured 145 s / 3 347
MiB on the same commit; this machine reproduces the child's figures.
## 7. Nothing else moved, and it was measured
- `~/okf-test/dokumenter` **byte-identical** against a frozen `e1f4faa` export,
`diff -r` empty, 52 md / 26 concepts. That folder holds **0 `.xml` of 5
files**, so it is a real known-negative and not merely an unchanged number.
- The K2 corpus holds **0 `.xml` of 86 files**. The pinned bundle's
`EXPECTED_CONCEPTS` and `EXPECTED_RANKS` are unchanged (7 passed), including
its `--no-tie-shared-rank` known-negative.
- `okf project` still byte-equal to `okf build`, `diff -r` empty.
- `okf skill` + `okf consume` + `okf check --skill … --payload …`:
**conformant, 15 rules over 8 excerpts and 2 081 withheld entries, 0
findings**.
- Suite **1567 passed, 1 skipped** (1554 before); `ruff check` and
`ruff format --check` clean over 116 files; `mypy src` clean over 21 files --
all four after `git add` and after commit.
- `_EVIDENCE` gains a `.xml` row at **`measured`**, asserted by the suite. The
class was chosen against the definitions above the table rather than
assumed: `measured` is "real CORPUS files and a hand-counted fasit -- someone
wrote the document for their own purposes and we counted what we got", and
this is a publisher's own published document with the publisher's own
structure as the fasit, written long before any lookup of ours. It lands in
the same place as `.html`, whose limit already reduces its 828 files to one
product from one publisher; the file count is smaller and the fasit is
stronger, being explicit rather than a generator's cut.
## Honesty limits
- **The denominator is ONE file, from ONE publisher, in ONE schema.** `.xml` as
a file type is far wider than NISO-STS. 828 HTML files and eight corpus PDFs
are different denominators and not comparable ones.
- **The fasit is the source's own structure**, so "correct boundary" here means
what the publisher declared, not what is useful to a reader.
- **The denominator 2 761 is CHOSEN.** The file holds 7 715 `<sec>`; 4 954 of
them are lettered points inside a process description. Whether those should
ever be concepts is a different question, answered here as "no" and measured
at 0 of 4 954.
- **The ceiling is structural, not recovered.** This route reads the
publisher's claim about its own structure directly, where the PDF arm had to
bridge from (page, y) to a line index. A high percentage here is therefore
**not** comparable to an equally high percentage there.
- **A STS document with wrong or stale `<sec>` nesting carries that error
straight into the segmentation**, and one document says nothing about how
common that is.
- **The generic fallback is measured on fixtures only**, never on a corpus.
That half of the row is `constructed` whatever the STS half earned.
- **SK1's and SK2's figures are `vegnormal-okf`'s own documentation** except
where re-measured here and stated as such.
- **The unit worksheet is n = 12 positions** (8 `pdf`, 3 `docx`, 1 `xlsx`) and
holds no `xml`, so this row is measured outside it, as `md`/`odt`/`pptx`/
`html` are.
- **"Delivered" is not "answered."** hit@k says a gold document reached the
payload, not that the payload answers the question.

64
docs/2026-09-11-k3-runde17-dekningen-stopper-ved-en-bredere-tittel.md
View file

@ -30,9 +30,9 @@ number below is the one measured here.
| known-negative | before rank 1 (rival rank 2), after rank 2, ranks 3-8 identical | **identical**, row for row | `okf consume`, with and without `--no-title-covered` |
| question tokens | 9, `er` and `og` dropped | **9**, the same nine | `normalise` |
| `title_covered_hits` | exactly one concept, one title token | **one**, one token | — |
| R761 build | ~23 s, 2 737 dirs, 2 761 concepts | **16 s**, 2 737, 2 761 | `okf build` |
| R761 before / after | 3/6·5/6·5/6 → 6/6·6/6·6/6 at default `k`; 3/6·5/6·6/6 → 6/6·6/6·6/6 at `--k 50` | **identical**, KP rank 1 at both | `hitk_sk2.py` |
| R761 group sizes | 1, 1, 1, 1, 9, 2, 0, 0 | **identical** | § 1 |
| reference standard build | ~23 s, 2 737 dirs, 2 761 concepts | **16 s**, 2 737, 2 761 | `okf build` |
| reference standard before / after | 3/6·5/6·5/6 → 6/6·6/6·6/6 at default `k`; 3/6·5/6·6/6 → 6/6·6/6·6/6 at `--k 50` | **identical**, KP rank 1 at both | the consumer's hit@k script |
| reference standard group sizes | 1, 1, 1, 1, 9, 2, 0, 0 | **identical** | § 1 |
**Deviation 1, remote state.** STATE described `main` and both tags as
unpushed; the remote holds `7cca9e0` and both tags. The push list at the end of
@ -47,12 +47,12 @@ this round: the tracked `uv.lock` records the package at `0.7.0` while
`pyproject.toml` says `0.8.1`, so any non-frozen `uv` command rewrites it.
**Deviation 3, wall time.** The known-negative build took 15 s here against
2.91 s; the R761 build 16 s against ~23 s. Neither enters any result.
2.91 s; the reference standard's build 16 s against ~23 s. Neither enters any result.
Everything below was measured in frozen `git archive` exports — `7cca9e0` under
`/tmp/k3r17/head`, `e15de71` under `/tmp/k3r17/new`, each
`uv sync --frozen --extra extract` — or through a scratch harness that imports
one of those `src` trees. `~/repos/vegnormal-okf`, its fasit, its instrument,
one of those `src` trees. `~/repos/the consumer`, its fasit, its instrument,
its XML source and `~/okf-test/dokumenter` were read and never written.
## 1 The diagnosis: what makes the partition harmful there and useful here
@ -70,14 +70,14 @@ are fusion (candidate) ranks, not delivered ranks.**
| P1 | 6 | 1 | 1 | 0.167 | 5 | 0 | 5 (is the covered one) | help: 5 → 1 |
| P2 | 6 | 0 | — | — | — | — | 1 | nothing |
| P3 | 6 | 1 | 1 | 0.167 | 3 | 0 | 3 (is the covered one) | help: 3 → 1 |
| R761 S1 | 5 | 1 | 1 | 0.2 | 5 | 0 | 5 (covered) | help |
| R761 S2 | 8 | 1 | 2 | 0.25 | 1 | 0 | 1 (covered) | nothing |
| R761 S3 | 5 | 1 | 2 | 0.4 | 2 | 0 | 2 (covered) | help |
| R761 S4 | 6 | 1 | 1 | 0.167 | 1 | 0 | 1 (covered) | nothing |
| R761 S5 | 6 | **9** | 1 each | 0.167 | 12, 27, 31, 71, 73, 89, 120, 126, 132 | 1 | 12 (covered, first of the nine) | help |
| R761 S6 | 5 | 2 | 1 each | 0.2 | 3, 29 | 1 | 3 (covered, first of the two) | help |
| R761 KP | 22 | 0 | — | — | — | — | 1 | nothing |
| R761 KN | 7 | 0 | — | — | — | — | — | nothing |
| std S1 | 5 | 1 | 1 | 0.2 | 5 | 0 | 5 (covered) | help |
| std S2 | 8 | 1 | 2 | 0.25 | 1 | 0 | 1 (covered) | nothing |
| std S3 | 5 | 1 | 2 | 0.4 | 2 | 0 | 2 (covered) | help |
| std S4 | 6 | 1 | 1 | 0.167 | 1 | 0 | 1 (covered) | nothing |
| std S5 | 6 | **9** | 1 each | 0.167 | 12, 27, 31, 71, 73, 89, 120, 126, 132 | 1 | 12 (covered, first of the nine) | help |
| std S6 | 5 | 2 | 1 each | 0.2 | 3, 29 | 1 | 3 (covered, first of the two) | help |
| std KP | 22 | 0 | — | — | — | — | 1 | nothing |
| std KN | 7 | 0 | — | — | — | — | — | nothing |
**Three properties do NOT separate the harmful case from the useful ones.**
The group size is 1 in the known-negative and 1 on four of the standard's six
@ -97,7 +97,7 @@ right; the partition overwrote it.
**It is not a segmentation miss, and the presence check says so.** Both
concepts of the known-negative are concept directories of their own and both
are delivered in every run of this round, at ranks 1 and 2 — only their order
moves. On R761 the answering directories `2-1`, `84-3`, `88-1`, `1`, `8-4`,
moves. On the reference standard the answering directories `2-1`, `84-3`, `88-1`, `1`, `8-4`,
`22` and `53` each exist once, and the control `99-9` does not, so the query
can find an absence.
@ -107,9 +107,9 @@ Every rank here is a DELIVERED rank from `build_payload`, with the partition
replaced by the variant and everything else — the lookup partition, the cut,
the quota — the shipped code. The harness was validated first: its
re-implementation of round 16 is byte-identical to the shipped code on all 8
R761 payloads and all 5 known-negative-bundle payloads; its "off" is
reference-standard payloads and all 5 known-negative-bundle payloads; its "off" is
byte-identical to `okf consume --no-title-covered` on the known-negative's
payload and reproduces the pre-0.8.1 R761 row rank for rank.
payload and reproduces the pre-0.8.1 reference-standard row rank for rank.
The known-negative bundle's four questions, verbatim, the three probes written
and their answers fixed at 22:55:36Z on 2026-09-10 before any of them was run:
@ -129,7 +129,7 @@ and their answers fixed at 22:55:36Z on 2026-09-10 before any of them was run:
The answers are named here by form only; the documents are the operator's, and
their titles stay out of a tracked file.
| form | KN | P1 | P2 | P3 | R761 hit@1/8/50, default `k` | R761, `--k 50` | KP | what it adds | KN-short |
| form | KN | P1 | P2 | P3 | std hit@1/8/50, default `k` | std, `--k 50` | KP | what it adds | KN-short |
|---|---|---|---|---|---|---|---|---|---|
| off (pre-0.8.1) | 1 | 5 | 1 | 3 | 3/5/5 | 3/5/6 | 1 | — | 1 |
| round 16 (0.8.1) | **2** | 1 | 1 | 1 | 6/6/6 | 6/6/6 | 1 | — | 2 |
@ -151,7 +151,7 @@ their titles stay out of a tracked file.
Read per candidate, each component alone first:
- **(a) felled.** Every answering title on the standard that the rule lifts is
one token (`Hovedprosesser`, `Armering`, `Inspeksjon`), so excluding
one token (each the bare term), so excluding
one-token titles sells hit@1 straight back to 3 of 6, at 2 and at 3 tokens
alike. It also loses P1 and P3.
- **(b) holds every measured row at 0.125 and 0.15, and is not shipped.** It
@ -174,10 +174,10 @@ Read per candidate, each component alone first:
for no measured effect.
- **(e) holds every row the order named.** Its two known-negatives say why each
of its choices is the one taken: blocking on EQUAL overlap takes S5 and S6
(an `Armering av …` or `Enkel inspeksjon` above the answer shares exactly
the one token the answer holds), and counting the blocker by shared prefix
takes S5 (`betonghvelv` reaches `betongkonstruksjoner` through the word
`betong`). Strict, and by equality — the matcher the covered group itself is
(a title that is the answer's term plus a qualifier, ranked above the answer,
shares exactly the one token the answer holds), and counting the blocker by shared prefix
takes S5 (a compound in a title reaches a compound in the question through
a shared four-character word). Strict, and by equality — the matcher the covered group itself is
read with.
- **(f) and (g)** are the two neighbours of (e), measured to locate it. (f)
compares token SETS instead of counts and fixes KN-short at the price of S6;
@ -224,8 +224,8 @@ no new parameter, no new constant:
## 4 The row, and every other row measured
**The standard (R761 XML, 2 761 concepts, one source document), delivered
ranks from `okf consume` through `hitk_sk2.py`, run from the `e15de71` export:**
**The standard (a 701-page reference standard, XML, 2 761 concepts, one source document), delivered
ranks from `okf consume` through the consumer's hit@k script, run from the `e15de71` export:**
| | off (pre-0.8.1) | round 16 (0.8.1) | round 17 |
|---|---|---|---|
@ -253,9 +253,9 @@ ranks from `okf consume` through `hitk_sk2.py`, run from the `e15de71` export:**
| K2 pinned default bundle | 6 questions, 453 concepts | payloads **identical 6 of 6**; group size 0 on 6 of 6 |
| `tests/test_default_bundle_pin.py` | 7 tests | **7 passed**, `EXPECTED_RANKS = (1, 1, 1, 1, 1, 5)`, file untouched |
| Arm B bundle | 6 questions, 629 concepts | payloads **identical 6 of 6**; group 0 on 6 |
| n100-2023 | 5 questions (round 16's), 446 concepts | **identical 5 of 5**; group 0 on 5 |
| n200-2024 | 5 questions, 1 133 concepts | **identical 5 of 5**; group 0 on 5 |
| n500-2024 | 5 questions, 270 concepts | **identical 5 of 5**; group 0 on 5 |
| corpus-a | 5 questions (round 16's), 446 concepts | **identical 5 of 5**; group 0 on 5 |
| corpus-b | 5 questions, 1 133 concepts | **identical 5 of 5**; group 0 on 5 |
| corpus-c | 5 questions, 270 concepts | **identical 5 of 5**; group 0 on 5 |
| 828-file HTML corpus | 828 documents | **828 plans, 0 of 828 rejected, 6 015 md**; `diff -r` **empty**, old and new source on one interpreter |
| `~/okf-test/dokumenter` | 5 documents | **26 concepts / 52 md**; `diff -r` **empty**, old and new source on one interpreter (built in two venvs, one line differs: `log.md`'s converter path) |
| `okf project` vs `okf build` | same folder, `--id d` | **byte-equal**, `diff -r` empty, 52 md |
@ -264,7 +264,7 @@ ranks from `okf consume` through `hitk_sk2.py`, run from the `e15de71` export:**
| gates | ruff 0.16.6 | `ruff check src tests tools` clean · `ruff format --check .` 120 files clean · `mypy src` clean, 21 files |
| `okf check` | `len(RULES)` | **16**, no rule added, `contract_check.py` untouched |
**hit@k on N100/N200/N500 is NOT MEASURED**, as in round 16: this repository
**hit@k on the three other bundles is NOT MEASURED**, as in round 16: this repository
holds no gold set for them — 0 gold sets, not 0 hits. What is measured there is
that the payload does not change.
@ -272,9 +272,9 @@ that the payload does not change.
KN-short stays at rank 2, as under round 16. It shares ONE question token with
each of the two titles, and different tokens (`ritb` with the named section,
`prøvedrift` with the covered one), so under (e) neither title answers more
and precision decides. R761 S6 is the same shape with the opposite answer: the
title above the right one shares `konstruksjoner`, the right one's title
`inspeksjon` — one each, different tokens — and there the covered title IS the
and precision decides. The standard's S6 is the same shape with the opposite answer: the
title above the right one shares one question noun, the right one's title
another — one each, different tokens — and there the covered title IS the
answer. Form (f), which blocks on any question token the covered title lacks,
fixes KN-short and takes S6 from rank 1 to 3; (e) keeps S6 and leaves KN-short.
Reading titles alone the two are indistinguishable. The trade is explicit:
@ -306,7 +306,7 @@ own body names the same role four times.
- **The sweep harness is scratch, not the product.** It lives outside the
repository; its fusion ranks in § 1 are candidate ranks. Every rank in § 2
came out of `build_payload`'s real cut, and every figure in § 4's first table
came out of `okf consume` itself, run by `vegnormal-okf`'s instrument, which
came out of `okf consume` itself, run by the consumer's instrument, which
was read and not modified.
- **This is still a reading-side change that moves a payload with no bundle
changing** — on the one measured question where a covered title is blocked.

275
docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md
View file

@ -1,275 +0,0 @@
# K3 round 19: a document's own identity, per-run frontmatter, and a description from the source
**Date:** 2026-09-11 · **Base:** `56b00e7` · **Commits:** `be169ee` (red),
`ee8d5b5`, `72617ac` (red), `912b850`, `0dbc331` (red), `de7849e`, `31ffc2c`
(red), `77e7cab` · **Measured from:** a frozen `git archive` export of
`77e7cab`, never the working tree and never the `okf` on PATH.
A NISO-STS document built with `okf build` landed every concept under a
directory named for its delivery file, and every `sources` entry named that
file twice. A consumer's own build of the same kind of document carried an
edition key, a publisher's address and a `description`, and `okf build` could
carry none of them. This round closes those four points in general code: no
name of a document, a publisher or a corpus appears in `src/`.
## 0. Where the measurement differs from the order, first
- **HEAD was `56b00e7`, not the `e15de71` the order recorded.** The R4 release
landed after the order was written. No commit since `e15de71` touched
`inbox.py`, `propose.py`, `cli.py`, `extract.py`, `profiles.py` or
`tests/test_xml_sts.py` (empty `git log`, rc 0), and `uv.lock` was clean, so
nothing had to be checked out.
- **The known-positive falls, and the directory name is the whole cause.**
S1–S6 hold at 6/6 · 6/6 · 6/6 at both `k`, as the order required. The
known-positive (KP) row went from **rank 1 at both `k`** on the base to **not
delivered at the default `k` and rank 13 at `k` = 50**. Decomposed in § 7:
the same bundle with only the directory renamed back to the file name puts KP
at rank 1 again, and the same bundle with every `description` removed does
not move it.
- **The order's own `--frontmatter 'sources=...'` value is not YAML that
PyYAML reads.** With it, **2 761 of 2 761** concepts' frontmatter fail
`yaml.safe_load` (a `:` inside a flow-mapping plain scalar, in the URL and in
`R761:2025`). The consumer bundle the order modelled it on has the same
property: **446 of 446** of its `sources` lines fail the same way. `okf
check` accepts both, with 0 findings. The flag writes the operator's value
verbatim, as ordered; this is reported, not repaired.
- **1 807 descriptions are written, not 2 026.** 2 026 titled sections carry a
direct-child spec point. 2 of those points have no `<p>`, and **217 carry
`": "`**, which a YAML reader refuses in `description: a: b`. The first build
wrote them and made 217 frontmatters unreadable to PyYAML. That was repaired
before delivery (`31ffc2c` red, `77e7cab`); § 5.
- **The 43-document reference corpus was not rebuilt.** It holds **0 `.xml` of
86 files** (known-positive in the same query: 66 `.pdf`), and every change
here is reached only through `.xml` or through a flag the build did not
receive. The pinned bundle's test runs as before (§ 8).
## 1. The spec, and what it does not say
Checked against the canonical text first (`docs/okf-nokkelinventar.md § 0.1`):
`curl` of upstream `okf/SPEC.md` exit 0, 37 748 bytes, `diff` against the local
copy **0 lines**, 1 006 lines, "Version 0.2".
- § 4.1, l. 205: "**Extensions:** Producers MAY include any additional keys.
Consumers SHOULD preserve unknown keys when round-tripping and MUST NOT
reject documents with unrecognized fields."
- § 11, l. 760: "Unknown additional frontmatter keys." among what a consumer
MUST NOT reject a bundle for.
- § 4.1 on `description`: "A single sentence summarizing the concept." **No
length limit exists** in § 4.1, § 8 or § 11 (`grep -n description` over the
spec is the denominator: 17 lines, none a limit). The one-paragraph limit in
§ 5 is this package's own.
Conformance is the floor and never the proof: every number below that says
"`okf check` 0 findings" is a statement about the contract checker, and § 0
already has one bundle it accepts that a YAML parser does not.
## 2. (a) The document's identity names its directory and titles its address
`extract.declared_identity(name, data)` reads what a NISO-STS document states
about itself: exactly one `<std-ident>` (`<doc-number>`, `<year>`) and exactly
one `<title-wrap>` (`<full>`, else `<main>`). It is `None` for every other file
type, for XML that is not STS, for an unparseable file and for a document
stating neither. A value stated more than once is not read: an adopted
standard carries one `<std-ident>` per issuing body, and taking the first is a
guess.
Measured on R761: **1** `<std-ident>`, **1** `<doc-number>` (`R761
Prosesskoden`), **1** `<title-wrap>`, `<year>` 2025. The delivery file's stem
occurs **0 times** in the document. `<doc-type>` reads `Innledning`, the name
of a chapter, and nothing uses it.
**The directory.** `okf build` replaces the file's stem with the
`<doc-number>` reduced to the id grammar and keeps the folders above it:
`r761-prosesskoden/`, where the base wrote the UUID.
**A declared name two documents claim is used by neither.** If a second
document in the run declares the same name, or its file name reduces to it,
both keep their file name, and stderr says which documents and which name. The
collision gate Door B already has would refuse both with "rename one", and a
name read from inside a document is not one a rename can change. Tested with
two documents sharing a `<doc-number>` and with a document whose file name
holds the declared name.
**The `sources` title.** The candidates are `<doc-number>` + `<year>`, then the
`<title-wrap>` title, then the file name. The first that passes the gate and
can be written into the flow mapping verbatim wins. The order left the choice
between the first two to measurement:
| `sources[0].title` | hit@k S1–S6, both `k` | KP (default / 50) | bytes spent over 8 questions (default / 50) | excerpts delivered at `k` = 50 |
|---|---|---|---|---|
| `R761 Prosesskoden 2025` (shipped) | 1,1,1,1,1,1 | – / 13 | 278 693 / 853 803 | 348 |
| `<full>` with its commas removed | 1,1,1,1,1,1 | – / 13 | 282 289 / 854 200 | 346 |
**They rank identically.** `consume` passes `sources` through to the excerpt
and ranks on neither. The full title costs 3 596 bytes more at the default `k`
and two excerpts at `k` = 50. The order's rule for a tie is the shorter one.
The deciding fact is structural anyway: R761's `<full>` carries **commas**,
which end a flow mapping, and the guard refuses the quoted scalar that could
carry them. It cannot be written verbatim, so the comma-free row exists only
in this measurement. A title is never cleaned up, because a title with its
commas removed is not the document's title.
`resource` stays the inbox-relative file. The address says where the bytes
are; the title says what they call themselves.
## 3. (b) `--frontmatter KEY=VALUE`
One form, repeatable, and a `--frontmatter-file` was not built. The consumer
bundle measured carries eight keys of its own, and eight flags are still
readable. A file would be a second parser for the same mapping.
- **Split on the FIRST `=`, and only there.** A publisher's address carries
`?languageCode=nb`.
- **Written verbatim on ONE line.** This package's readers are line-oriented,
and a block-form `sources` is invisible to `parse_frontmatter` (measured in
`vegnormal-okf` 2026-09-08 and cited by the order). The flow form is the only
one that survives our own readers. The test reads a stated `sources` back
through both `parse_frontmatter` and `consume.read_sources`.
- **Precedence: flag > what the document declares > file name.** A run may
ADD any key and REPLACE only `sources` and `description`, the two keys with a
layer below the flag. Tested on both layers. A stated `sources` gives exactly
one `sources` line, on the STS document and on a markdown one.
- **Refused before a proposal is written** (`run_frontmatter_invalid`): every
key the door writes itself (`inbox._door_keys`). That covers the hash, the
offsets, the locators, `type`, `ingested_at`, the bundle id, the ownership
stamp, the derived facets and `derived` itself. It also covers Door A's
`ingest_manifest`, which would make that door claim a Door B file. Also
refused: a key outside `[A-Za-z_][A-Za-z0-9_-]*`, and a value that is empty,
spans lines or has surrounding whitespace, since `parse_frontmatter` strips
it. A key given twice is refused rather than resolved.
- **Nothing moves without it.** A test holds the flagged tree to the plain
one minus exactly the stated line. `okf project` does not take the flag,
because it owns no flag that moves a bundle's bytes.
## 4. (c) and the YAML repair
The NISO-STS reader records, per titled `<sec>`, the **first `<p>` of the
first direct-child `<sec sec-type="spec">`**, whole. The plan entry carries it
beside `title`, because a plan is the record a rebuild replays. The door writes
it after the gate has seen it: it is document text persisted outside the body
the gate screens, so it is kept only on the non-blocking floor and only as the
sanitized text. A section with no such point gets no key. Nothing is derived
from the title.
Measured on R761: **2 026 of 2 761** titled sections carry a direct-child spec
point. 264 of those points hold more than one `<p>` and 2 hold none. The first
`<p>` runs **17 / 109 / 273 / 521 / 942** characters at min / median / p90 /
p99 / max. A character cap was not chosen, because a cut inside a paragraph
writes a sentence the source never wrote.
**The repair.** The first build from `de7849e` wrote 2 024 descriptions, and
PyYAML's `safe_load` refused **218 of 2 761** frontmatters: 217 on
`description`, all carrying `": "`, and 1 on a `title` the base already had.
`inbox._yaml_plain` is now checked where the value is written. It refuses a
leading YAML indicator, `": "`, `" #"`, a trailing `:`, a tab or a line break.
Over the 2 024 measured values the rule and PyYAML agree on **every one**: 217
refused, 0 refused that PyYAML reads, 0 kept that it does not. Omitted rather
than quoted, because a quoted value comes back from our line readers with its
quotes. After: **1 807** descriptions, and **1 of 2 761** frontmatters refused
by PyYAML, which is the base's own `title`.
That `title` class is older than this round and is not touched here: **41 of
436** concepts on the pinned 43-document bundle fail `safe_load` on `title`.
## 5. (d) `--ingested-at` alone: confirmed, nothing built
Built with `--ingested-at 2026-09-08T12:00:00Z` and no `--proposed-at`:
**2 761 of 2 761** concepts carry that stamp, on the segmented route where the
door reads the plan's `proposed_at`, and `log.md` is dated `## 2026-09-08`. The
base carried the same, so the `1970-01-01T00:00:00Z` in the consumer's bundle
came from a build that passed no stamp. The flag works as specified.
`tests/test_cli_build.py::test_ingested_at_alone_stamps_every_concept_the_same`
already held it.
## 6. V1 FERDIG — R761, base `56b00e7` against `77e7cab`
Built with `--bundle-id r761-prosesskoden-2025 --okf-version 0.2 --ingested-at
2026-09-08T12:00:00Z`. The flagged build adds `--frontmatter utgave=R761:2025`
and a `sources` value addressing the publisher's viewer.
| row | base | after | denominator |
|---|---|---|---|
| document directory | `14ce59dc-…` (the file's UUID) | **`r761-prosesskoden`** | 1 |
| concepts | 2 761 | **2 761** | — |
| conservation | 110 = 110 | **110 = 110** | N = 110 (109 images `extractor_unknown`, expected) |
| shells (body with no non-heading line) | 710 | **710** | 2 761 |
| `sources[0].title`, no flag | the file name | **`R761 Prosesskoden 2025`** | 2 761 |
| `sources`, flagged | — | the stated value, **one line** | 2 761 |
| `utgave`, flagged | — | **2 761** | 2 761 |
| `description` | 0 | **1 807** | 2 026 with a spec point |
| `ingested_at` = the stamp | 2 761 | **2 761** | 2 761 |
| frontmatter refused by PyYAML, no flag | 1 (`title`) | **1** (`title`) | 2 761 |
| hit@1 · hit@8 · hit@50, S1–S6, default `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| hit@1 · hit@8 · hit@50, S1–S6, `k` = 50 | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KP rank, default `k` / `k` = 50 | 1 / 1 | **not delivered / 13** | 1 |
| `okf check`, KP and S5 × both `k` | — | **4 of 4, 16 rules, 0 findings, exit 0** | 4 |
hit@k is vegnormal-okf's `experiments/hitk_sk2.py`, run read-only against the
scratch bundle with its five flags. Its payloads land at fixed
`/tmp/hitk-sk2-<id>.json` paths and were moved out between runs. The same
cells hold on the flagged and the unflagged build. On `de7849e`, `okf check`
was also run over all 8 default-`k` payloads: 8 of 8, 0 findings.
## 7. The known-positive, decomposed
The KP question contains the document's own number and name. `consume`'s
signal 1 is `title + concept_id.replace("/", " ")`, so once every concept id
starts with `r761-prosesskoden/`, every concept matches those two question
tokens, and the gold concept's distinguishing match on its own title
disappears into a 2 761-way tie. With the UUID directory, the id matched
nothing.
| variant (all from `de7849e`, unflagged) | KP default `k` | KP `k` = 50 | S1–S6 |
|---|---|---|---|
| as built | not delivered | 13 | all rank 1 |
| V1: every `description` line removed | not delivered | 13 | all rank 1 |
| V2: only the directory renamed back to the UUID (root index link rewritten to match) | **1** | **1** | all rank 1 |
| as built, `okf consume --rarity-weight` | **4** | **4** | all rank 1 |
| base `56b00e7`, `--rarity-weight` | 6 | 6 | all rank 1 |
`--rarity-weight` weighs a token by `log(N/df)`, and a token in every concept
weighs 0. It delivers KP at the default `k` on this bundle with S1–S6 unmoved.
It stays off: it is a ranking decision this round was fenced out of, and its
wider measurement (`docs/2026-09-08-sjeldenhetsvekt.md`) was mixed. **Which way
to go is the operator's.** The options are to keep the directory and repair
the id-segment signal for a segment every concept shares, or to accept the row.
## 8. Nothing else moved
- **Five-document folder.** 5 = 5, 52 md. `diff -r` against the base build
differs in **one line of one file**: the `Converter` line in `log.md`, which
records the absolute path of the venv that ran it, and the two exports sit
in different directories. With that line ignored (`diff -r -I`), rc 0. The
folder holds 0 `.xml`, so it is a real known-negative.
- **The four goldens.** `tests/test_segmented_golden.py` is green in the
working tree. In the export its fourth test is the one known red: it runs
`git status`, and an export has no repository.
- **The pinned 43-document bundle.** `tests/test_default_bundle_pin.py`,
**7 passed** in the export with `GOLD_SET` copied in.
- **Suite.** 1 602 → **1 658 passed**, 1 skipped (56 new). `ruff check src
tests tools`, `ruff format --check .` and `mypy --strict src` are clean, each
with its rc captured directly, after `git add` and after commit.
- Nothing was written in `vegnormal-okf`. Its `git status` is empty, and the
two `hitk_sk2` bytecode files in its ignored `__pycache__` date from
2026-09-10.
## Honesty limits
- **N = 1 STS document.** "General" is a claim about the code: no document,
publisher or corpus is named in `src/`, and every rule is keyed to a schema
element. It is not a measurement over several documents. The fixtures are
hand-written from the measured document's shapes.
- **The title choice rests on hit@k and bytes, not on what reads well**, and
hit@k could not separate the two candidates at all. The tie-break and the
flow-terminator fact decided it.
- **The one-paragraph `description` limit is ours.** The spec asks for "a
single sentence" and sets no length. 942 characters is not one sentence, and
it is what the source wrote.
- **The YAML rule is measured on 2 024 values from one document** against one
parser (PyYAML 6.0.3, YAML 1.1). A YAML 1.2 reader is not measured.
- **KP's fall is measured on one question.** Whether a question that names its
own single-document bundle is common is not measured.
- **`okf check` accepting a bundle is not a quality gate.** § 0 has a bundle
it accepts on 2 761 of 2 761 concepts that PyYAML refuses on the same count.
- **The 43-document corpus was reasoned out, not rebuilt:** 0 `.xml` of 86,
and the five-document folder is the measured known-negative in its place.

231
docs/2026-09-11-k3-runde20-delt-katalog-og-arvet-kontekst.md
View file

@ -1,231 +0,0 @@
# K3 round 20: a directory every concept shares, and a section whose body is its heading
**Date:** 2026-09-11 · **Base:** `7eeafb3` · **Commits:** `5ce8efe` (red),
`9f8a1bc`, `f7fd0d4` (red), `edd3b70` · **Measured from:** frozen `git
archive` exports of `9f8a1bc` and `edd3b70` (Python 3.11.14), never the
working tree and never the `okf` on PATH.
Two deliveries, in the order they were set. **A** repairs a ranking regression
round 19 introduced: once a one-document bundle named its directory after the
document, a question naming the document stopped finding the concept that
names it. **B** adds `okf build --shell-parent`, off by default: a concept
whose body is its heading alone points at the nearest ancestor that holds
text. A was measured and committed before B was built, so B's rows are
measured against a bundle where the known-positive already stands.
## 0. Where the measurement differs from the order, first
- **The base suite is 1 658 passed, 1 skipped, rc 0**, not "1 known red". Run
from a scratch working directory one more test fails
(`tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit`
opens `examples/...` relative to the working directory); every suite number
below is from the repository root.
- **The shell count's path moved.** The consumer's `build/ferdig/r761-2025` now
holds its own route's bundle (0 of 2 728 shells). The generic bundle the order
counted is its `-generisk` sibling: **710 of 2 761**, 22 M. N100, N200 and
N500: **0 of 446, 0 of 1 133, 0 of 270**, as stated.
- **The known-positive's token counts**, with the shipped matcher (stem prefix
on) over the 2 761-concept bundle: `r761` in **0** titles and **2 761** titles
with the id; `prosesskoden` **11 → 2 761** (the injected premise said 15);
`standard` **6 → 6** (said 17 → 17). The mechanism stands; these are the
numbers.
- **Two things moved under this session, and both were measured before a
number was used.** (1) The hit@k instrument writes its payloads to fixed
paths, and a consumer session running the same instrument writes the same
paths. 3 of the 16 payloads first moved out were that session's: `okf check`
refused them with `bundle_mismatch` and the other session's bundle id. Every
payload below comes from a scratch copy of the instrument whose only change is
its payload path, and the instrument's own printed rows agree with it. (2)
The N200 bundle was rebuilt by its owner during the session: **1 137 of
1 137** files rewritten, files carrying U+00AD **116 → 0**. Digests from
before and after are not comparable. The control is the base, A and B
exports on the same current bundle: identical on **5 of 5**.
- **Acceptance B's "all 710 carry `parent`" cannot be met by this rule:**
**35 of 710** shells have no ancestor that holds text. **675 of 710** carry
it.
- **The premise correction holds.** The canonical `SPEC.md` has no `derived`
or `references` frontmatter key: `references/` is a directory convention
(§ 6.3, l. 479), and § 5.1 l. 343 reads "Lineage is expressed through links,
not a dedicated field." `derived:` in our bundles is this package's own key.
§ 6.1 l. 458-460: a link's kind, "parent/child" among them, "is conveyed by
the surrounding prose, not by the link itself" -- so `parent:` is an
extension under § 4.1, and not a field the spec defines.
## 1. A: a directory every concept id shares
**The mechanism.** `concept_scores`' first signal reads a concept's title
together with the segments of its id, and the overlap counts each question
token once. On a one-document bundle every id starts with the same directory.
When the question names the document, every concept answers those tokens
through the id, except the one whose title already carried them, which gains
nothing. The one concept distinguished by naming the document loses exactly
that distinction, and a concept answering nothing but the directory stops
being a guess the cut withholds. Measured on R761: the known-positive (KP)
went from rank 1 to not delivered at the default `k` and 13 at `k` = 50,
reproduced here on the base export with the instrument, S1-S6 unmoved.
**Four forms, measured on the base export with a scratch-patched
`consume.py`, every rank a delivered rank:**
| form | KP (default / `k` 50) | S1-S6 | K2 default, 6 payloads | K2 Arm B, 6 | N100/N200/N500, 15 | five-document bundle, 5 |
|---|---|---|---|---|---|---|
| off (control) | – / 13 | 6/6 | 6 of 6 identical | 6 of 6 | 15 of 15 | 5 of 5 |
| drop each concept's document directory | 1 / 1 | 6/6 | **0 of 6; row 6 rank 5 → not delivered** | **1 of 6; row 6 5 → 7** | 15 of 15 | 5 of 5 |
| drop the directories every id in the same document shares | 1 / 1 | 6/6 | **0 of 6; row 6 not delivered** | **1 of 6; row 6 → 7** | 15 of 15 | 5 of 5 |
| **drop the directories every id in the bundle shares (shipped)** | **1 / 1** | **6/6** | **6 of 6** | **6 of 6** | **15 of 15** | **5 of 5** |
The control reproduces the instrument's base row rank for rank and count for
count. **Across documents the document directory carries information**: the
two per-document forms cost a K2 row, so the shipped rule touches only the
prefix every id shares. On a one-document bundle that prefix is constant by
construction. Where ids share none, which covers every multi-document bundle
measured here, the signal reads the same string as before. `shared_id_prefix`
returns directory segments only, never the leaf. The third signal, the
document score, still reads the whole id, and on a one-document bundle it is
one value for every concept. Under `--rarity-weight` a token every concept
carries weighs `log(1) = 0`, which is the same answer.
**What the rule does not reach.** On a multi-document bundle whose question
names one document, the same saturation still happens inside that document.
The per-document forms would repair it, and they cost K2 row 6. No real
bundle of that shape was measured.
## 2. B: a section whose body is its heading
**The shape is faithful, not an extractor defect.** A process code states its
lettered points once, on the section that owns them, and every section nested
below inherits them. A nested section carries a title and nothing else, and
the XML reader is right to write it as a heading with nothing under it. The
directory tree is two levels deep (`<document>/<number>/<stem>`), so the
parent is not in the path either. `SegmentEntry.parent_id` already existed. It
was validated against the plan's own ids and written by the door as `parent:`,
but the proposer never set it: **0** `parent:` lines in the base bundle.
**Which ancestor.** The order left the route to measurement. Against the
document's own `<sec>` nesting, over the 710 shells:
| route | names the same ancestor as the nesting | where it misses |
|---|---|---|
| **plan level and order (shipped)** | **708 of 710** | `36.73211` and `36.73212` sit at depth 7, which a markdown heading clips to 6, and point at `36.732` instead of `36.7321`: still an ancestor, one level too high |
| section-number prefix | 686 of 710 | `12` begins with `1`; and an unnumbered document has no number to read |
9 sections sit at depth 7, and 6 of them are shells. The level route reads the
plan, so it serves any outline route and not only NISO-STS. The test holds the
same parents through the bookmark arm's route. **An empty ancestor is passed
over**, so the pointer lands on text in one step. 531 of the 675 point at
their direct parent. **A shell with no ancestor holding text gets none**,
which applies to 35. The rule runs after the plan is laid out and adds no
boundary.
**Three forms, one built:**
- **(i) Copy the ancestor's text in.** Measured by the consumer through its own
chain, not this package's (`vegnormal-okf`,
`docs/2026-09-11-r761-d1-n-veien.md` § 4a-4c, 5). Shells went to 0 of 2 728,
and inherited text was 93.6 % of the process bodies' bytes. Median file size
went up 7.8×, to 5 543 B, and the bundle from 22 M to 28 M. hit@1 went 6/6 →
2/6 and consumption 3 of 6. Read precisely: 3 of those 5 misses had their
answer in front matter the consumer's layout put in one concept, which is
not copy-in. In the other two, copy-in grew each excerpt until the budget
held 4-8, and the right process fell to withheld place 504 and 1 069.
- **(ii) Point, copy nothing. Built.** The existing `parent:` key names the
ancestor's `segment_id`, which every concept of the run carries.
- **(iii) Copy only the nearest ancestor's first lettered point.** Not built.
The one data point for it is the consumer's S6, where an inherited scope
line answered the question with the parent never delivered. Its size and
ranking cost are not measured here.
**Known cost, not repaired.** The index is a projection of the frontmatter,
and `structure` reads `parent` as a document NUMBER. A segment id answers to
none, so every such parent renders unresolved in the index (`parent: p1?`),
even though the concept it names is in the bundle. The same key already
carries both meanings for an adjudicator's declared parent (`inbox.py`).
**Default: off, by measurement.** `okf consume` reads no `parent` key: 0 lines
quote it, and the same query finds `title` on 3. So no payload ranks
differently, and the hit@k rows below are identical with and without the
flag. The flag moves the bytes of every bundle holding a heading-only section.
A consumer reaches the pointer only by opening the concept file, and the
payload does not carry it.
## 3. V1 FERDIG
**A** (`9f8a1bc`), R761 built with round 19's flags, hit@k by the instrument's
rule:
| row | base `7eeafb3` | after | denominator |
|---|---|---|---|
| KP rank, default `k` / `k` 50 | not delivered / 13 | **1 / 1** | 1 |
| hit@1 · 8 · 50, S1-S6, default `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| hit@1 · 8 · 50, S1-S6, `k` 50 | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KN (fabricated section) | no hit | **no hit**, same top-1 | 1 |
| delivered per question, default `k` | 7 8 8 8 8 8 8 7 | 7 8 7 8 8 8 7 7 | 8 |
| `okf check` | — | **16 of 16 payloads, 16 rules, 0 findings, rc 0** | 16 |
**B** (`edd3b70`), the same build with and without `--shell-parent`:
| row | flag off | flag on | denominator |
|---|---|---|---|
| shells | 710 | 710 | 2 761 |
| shells carrying `parent` | 0 | **675** | 710 |
| concepts with text carrying `parent` | 0 | **0** | 2 051 |
| `parent` = the `<sec>` nesting's ancestor | — | **708** | 710 |
| hit@1 · 8 · 50, S1-S6, both `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KP rank, both `k` | 1 / 1 | **1 / 1** | 1 |
| consumption, one `claude -p` per question | not re-run | **6 of 6 correct; answer first cited 4 of 6** | 6 |
| `okf check` | — | **16 of 16, 16 rules, 0 findings, rc 0** | 16 |
| `du -sh` / bytes of concept files | 22 M / 3 263 257 | **22 M / 3 272 360** (+9 103) | — |
| median concept file | 840 B | **840 B** | 2 761 |
| files differing | — | **1 350** (675 concepts + 675 directory indexes) | — |
The flag-off build is byte-identical to A's bundle (`diff -r`, 0 lines). In
the consumption runs, S4 cited the terminology section first and S6 cited
main process 88 above its answer. No run self-reported an extra pre-pass. The
generic bundle before this round scored 6 of 6 correct and 5 of 6 first-cited
in the consumer's own run. One draw each is not a measured difference.
## 4. Nothing else moved
| row | denominator | result |
|---|---|---|
| K2 pinned default bundle | 6 payloads | **identical 6 of 6** after A and after B; ranks `(1,1,1,1,1,5)` |
| K2 Arm B bundle | 6 payloads | **identical 6 of 6**; ranks `(1,1,1,1,1,5)` |
| N100 / N500 | 5 + 5 payloads | **identical 10 of 10** against the base |
| N200 | 5 payloads | identical 5 of 5 after A; after the owner's rebuild, **base = A = B on 5 of 5** |
| five-document bundle (round 17's known-negative bundle) | 5 payloads | **identical 5 of 5**; the known-negative rank 1, unchanged from round 17 |
| five-document folder, rebuilt | 52 md | `diff -r` differs only in `log.md`'s converter path; with `--shell-parent` **0 shells of 26, 0 files differ** |
| `tests/test_default_bundle_pin.py` | 7 tests | **7 passed** in both exports, gold set copied in |
| suite, working tree | — | 1 658 → **1 662** (A) → **1 667** (B) passed, 1 skipped, after `git add` and after commit |
| suite, frozen exports | — | 1 661 and 1 666 passed, 1 skipped, **1 failed** = the known export artefact `test_segmented_golden.py::test_the_four_existing_goldens_are_untouched` |
| gates | — | `ruff check src tests tools`, `ruff format --check .` (125 files), `mypy --strict src` (21 files): clean, rc captured directly |
| `okf check` rules | `len(RULES)` | **16**, `contract_check.py` untouched |
## Honesty limits
- **A's KP row is ONE question**, and the form was chosen over four candidates
with that question's answer visible. The choice rests on the K2 row the other
forms lose, not on KP, which all three repairs fix.
- **A changes the ranking on any bundle whose ids all share a leading
directory**, which is what a one-document build produces. The old order is
reproducible by no flag. Measured on 1 such bundle; the 7 other bundles
measured share no prefix and are byte-identical.
- **B is measured on ONE document with shells (N = 1).** "Every outline
document" is a claim about the code, which reads the plan's level and order;
the fixture test and the five-document folder are the only other rows, and
the folder has no shells at all.
- **The 2 of 710 wrong ancestors come from ATX's six levels**, not from the
rule. Carrying the unclipped depth would fix them and was not done.
- **Form (i) was measured by the consumer through its own chain; form (iii)
was not measured by anyone.** (ii) was chosen because it moves no ranking
and D1 measured (i) moving it down, not because (ii) was shown better for a
reader. With `consume` blind to the key, (ii)'s value to a reader is
unmeasured.
- **Consumption is one non-deterministic draw per question**, six runs on a
subscription (`ANTHROPIC_API_KEY` unset), judged against the fasit section
by reading the answer. The answer texts are not reproduced here.
- **Conformance is the floor and never the proof.** `okf check` is green on
all 32 payloads, and a payload whose right answer was not delivered would
pass it as well.
- **The instrument's shared paths and the N200 rebuild were caught by
controls,** a known-positive checker rule and three exports on one bundle.
Nothing guarantees that no other movement went unseen.

346
docs/2026-09-11-k3-runde21-parent-til-leseren.md
View file

@ -1,346 +0,0 @@
# K3 round 21: `parent` all the way to the reader
**Date:** 2026-09-11 · **Base:** `e717b1c` · **Commits:** `a5cd7c5` (red),
`4f7bd61` (A), `ca83dad` (red), `839bd61` (B), `46e555d` (B, defaults),
`c188a0c` (red), `5970369` (C), `9822e55` (red), `bd13c5f` (D) · **Measured
from:** frozen `git archive` exports of the base and of `4f7bd61`, `839bd61`,
`5970369` and `bd13c5f` (Python 3.11.14, each `__file__` under a scratch
directory), never the working tree and never the `okf` on PATH.
Round 20 gave a section whose body is its heading alone a `parent:` key naming
the nearest ancestor that holds text (`okf build --shell-parent`) -- 675 of
710 such sections in one 2 761-concept standard -- and no reader could see it:
`okf consume` did not read the key, the payload did not carry it, the body held
no link, and the index rendered it unresolved. Four deliveries, in order, each
with its own red test and its own commit: **A** the pointer reaches the excerpt
and the body; **B** the ancestor's text reaches the reader, in two forms, and
both defaults are chosen on numbers; **C** the index resolves the pointer; **D**
the depth-7 trap.
## 0. Where the measurement differs from the order, first
- **HEAD was `e717b1c`, not `4595dde`,** and the suite baseline **1 753
passed, 1 skipped, rc 0**, not 1 667 / 1: K3-22 landed after dispatch. The
shell rule (`--shell-parent`, `propose._link_shells`) is in the tree.
- **The order's R761 build command exits 2 on `e717b1c`.** K3-22 refuses a
`--frontmatter sources=` flow value whose URL carries `?`. Every R761 bundle
here is built with the same command minus that flag (`utgave` kept), so
`sources` falls back to the document's own layer. **The base row still
reproduces round 20's numbers exactly:** hit@1/8/50 6/6 at both `k`, the
known-positive (KP) at rank 1 at both, delivered per question 7 8 7 8 8 8 7
7 at the default `k`.
- **The known-positive moved in every payload.** SS 7.4's known-positive IS
`docs/consumption-contract.md`, and A and B edit its SS 8, so
`budget.known_positive` moves in every payload with no bundle changing
(13 238 / 12 893 / 345 -> 14 455 / 14 083 / 372 -> 14 721 / 14 346 / 375).
Every "byte-identical" row below after A and after B is measured with that
block removed, and the whole-payload count is given beside it.
- **The hit@k instrument moved.** The consumer repository stands on `f783e52`,
not `4f70e9d`; `experiments/hitk_sk2.py` was last changed in `6fad6a2`, as
stated. The scratch copy differs from it in two lines (the payload path,
because a concurrent session writes the hard-coded one), and the form (b)
copy in one more (`--follow-parent` on the `okf consume` call). The consumer
repository's `git status --porcelain` is empty before and after.
- **The generated skill tells a reader to write `/tmp/payload.json`,** a path a
concurrent session shares. Each consumption run used a scratch copy of the
skill whose only change is that path (and, for form (b), the flag line).
- **The link's first design failed on the fixture before it shipped.** Read as
body text, `Enclosing section: [...](/...)` is a bundle-local link target,
and structure derivation turned it into a SECOND relation -- `references:
[/...md?]` with `derived: [references]`, rendered unresolved because nothing
resolves the absolute form. The link is therefore appended after derivation
and screened on its own, the rule `description` already follows.
- **One red test met a mechanism instead of the rule.** B's "no room" case
asked through `build_payload` at `limit == spent`, where the knapsack's
500-byte buckets admit nothing (`budget_admits_nothing`). The green commit
holds the rule at `attach_parent_text` and says so.
## 1. The spec, verbatim, and which paragraph each half leans on
Canonical `SPEC.md` at `ad30107`:
- **SS 4.1, Extensions:** "Producers MAY include any additional keys.
Consumers SHOULD preserve unknown keys when round-tripping and MUST NOT
reject documents with unrecognized fields." -- the ground for the `parent:`
KEY and for the payload member.
- **SS 5.1:** "Lineage is expressed through links, not a dedicated field." --
the ground for the LINK: the key alone is not the spec's form of lineage.
- **SS 6.1:** "Absolute (bundle-relative): begins with `/`, interpreted
relative to the bundle root. This is the **recommended** form because it is
stable when documents are moved within their subdirectory." And: "The
specific kind (parent/child, references, joins-with, depends-on) is conveyed
by the surrounding prose, not by the link itself." And: "Consumers MUST
tolerate broken links: a link whose target does not exist in the bundle is
not malformed; it may simply represent not-yet-written knowledge." -- the
link's form, its two words of prose, and why an unresolved pointer is named
in the payload rather than refused.
Contract SS 8, first paragraph, **changed**: "Additional members are permitted
and are not read by the checker." now reads "Additional members are permitted;
the checker reads only the members this section names." -- because A's rule
reads `parent`, which point 6 now names.
## 2. A: the pointer reaches the excerpt and the body
**The payload.** `consume.link_parents` resolves `parent` among the concepts
sharing the pointing concept's `source_file`: a segment id is unique only
inside one document's plan, so `p1` exists in every document of a
multi-document bundle. One pass over concepts already read, no file opened
again. An excerpt carries `parent: { concept_id, title }` -- never the raw
`segment_id`, which a reader can open nothing with -- and
`parent_unresolved: true` where the pointer lands nowhere. The member is
conditional, like `req_number`.
**What the excerpt carries was chosen on this:** the raw id costs least and
can be followed only by enumerating the bundle, which SS 9.2 forbids; the
concept id is one `Read`; the title is what a citation is made of. Measured
cost of the chosen member: 5 members, about 528 B, over the 60 excerpts of the
eight default-`k` payloads; 34 members, about 3 927 B, over 353 at `k` 50.
**The body.** A heading-only body whose plan entry has a parent gains ONE line:
`Enclosing section: [<title>](/<bundle-relative path>)`. Only such a body --
the segmented goldens' declared parents are bodies holding text, and stay
byte-identical. The two words are English and generic, because the code
cannot know the source's language; the relation they name is true of an
adjudicator's parent and of a shell's nearest ancestor holding text alike.
**The checker.** Seventeenth rule, `parent_unfollowable`: a `parent` that is
not a `concept_id` and a `title`, names its own excerpt, or names a concept in
neither `excerpts` nor `withheld`. Those two lists together name every concept
the pre-pass considered (SS 5.2), so the rule sees a dangling pointer without
opening the bundle. `parent_unresolved` is not a finding (SS 6.1).
| row | result | denominator |
|---|---|---|
| K2 pin in the export, gold set copied in | **7 passed** | 7 |
| K2 default + K2 Arm B + N100/N200/N500 + five-document bundle | **32 of 32 identical without `known_positive`**; 0 of 32 whole | 12 + 15 + 5 |
| K2 ranks, both bundles | `(1,1,1,1,1,5)` unchanged; `parent` members **0** | 6 · 6 |
| five-document bundle, known-negative row | rank 1 unchanged (`[1,1,1,1,2]`) | 5 |
| R761 without the flag | **16 of 16** payloads identical without `known_positive`; bundle `diff -r` only `log.md` (the converter's path) | 16 · 2 761 |
| R761 with the flag, hit@1 · 8 · 50 | **6/6 · 6/6 · 6/6** at both `k`, KP rank **1 / 1** | 6 |
| shells carrying exactly one link | **675 of 710**, 0 with more, **0 broken**, 675 of 675 targets = the `parent` | 710 |
| files differing from the flagged base bundle | 676 = 675 concepts + `log.md`; **0 index files** | 5 518 |
| `okf check` on the flagged payloads | **16 of 16 rc 0, 17 rules, 0 findings** | 16 |
| known-negatives | a dangling `concept_id` and the raw `"p1977"`: **1 finding each, rc 1** | 2 |
| link bytes | **72 265 B = 4.49 %** of all body bytes (D2's inheritance section: 51.0 %); 69.2 % of the 675 linked bodies; median line 103 B, max 247 B | 1 610 590 B |
| shells, PM script / concepts with no source text of their own | **35 / 710** | 2 761 |
**The honest shell count is 710.** The PM script counts non-empty lines not
starting with `#`, and a link line is one; 675 of the 710 now carry a line and
still no text of their own.
**What the link costs in ranking, measured.** hit@k does not move. The
delivered SET does: at the default `k` S2-S6 and KP are identical, while S1
exchanges 3 of 7 places and KN 2 of 7, every newcomer a linked shell; at `k` 50
three of eight questions move (KP 23 in / 19 out). Of the newly delivered
shells, those matching the question ONLY through the link line are 1 of 3
(S1), 4 of 7 (S1 at `k` 50), 6 of 23 (KP at `k` 50) and 0 of 2 / 0 of 8 (KN),
and in every such case through the PATH, never the title: the bundle-absolute
form repeats the document's directory in 675 bodies -- the saturation round 20
took out of the id signal, back in through the body. The rest of the movement
is not decomposed per signal.
## 3. B: the ancestor's text reaches the reader
**Form (a), the pointer**, is A. **Form (b), `okf consume --follow-parent`**,
puts the enclosing concept's `text` inside `parent`, with that concept's own
`sha256`, AFTER the cut and only from the room the cut left, in rank order. A
parent the payload already holds, or one a higher-ranked excerpt carried,
travels once; a text that does not fit is cut to the longest prefix that does
and marked `truncated`. Inherited text therefore cannot displace an excerpt --
the mechanism a consumer measured when copied-in ancestor text pushed the
right section to withheld place 504 and 1 069.
| row | (a) pointer | (b) `--follow-parent` | denominator |
|---|---|---|---|
| hit@1 · 8 · 50, default `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| hit@1 · 8 · 50, `k` 50 | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KP rank, both `k` | 1 / 1 | **1 / 1** | 1 |
| delivered list and withheld list equal to (a) | -- | **16 of 16** | 16 |
| consumption correct | **6 of 6** | **6 of 6** | 6 |
| the fasit section cited first | 4 of 6 | 5 of 6 | 6 |
| `spent`, default `k`, where it differs | S1 28 020, KN 10 514 | S1 30 508, KN 13 260 (the other six equal) | 8 |
| `spent`, `k` 50, largest difference | KP 96 965 | KP 117 782 (12 texts, 0 truncated) | 8 |
| `okf check` | 16 of 16 rc 0 | **16 of 16 rc 0, 17 rules** | 16 |
**The row that is the point: one heading-only section as a reader gets it.**
Question: a shell's own title (33.212). Top-1 is its sibling 33.211, whose
title differs only in a token too short to score; both are shells under
33.2 "Sikringsbolter". Eight excerpts delivered in each form.
| | before A | (a) | (b) |
|---|---|---|---|
| the excerpt's `text` | its heading line | its heading + `Enclosing section: [Sikringsbolter](/.../33-2/sikringsbolter.md)` | same |
| `parent` | absent | `concept_id`, `title` | + `sha256`, `text` (3 847 characters, not truncated) |
| excerpt bytes | 858 | 1 034 | 5 067 |
| payload `spent` | 6 864 | 8 272 | 12 305 |
| what the reader must do to learn what the section inherits | nothing it can do: no pointer | one `Read` of the named concept (SS 2.2: the payload names it reachable) | nothing: it is in the payload |
**The consumption difference is not the form.** S1-S6's answers are never
heading-only sections. The first-cited difference (S4) sits on a payload that
carries no parent at all and is identical in both forms apart from the
known-positive block, so it is the draw, not the form. S1's form (a) answer
used the pointer on its own: it named a delivered shell's enclosing section by
its `parent`.
**Defaults, one sentence each:**
- **`--follow-parent`: off.** It delivers the same set as the pointer by
construction and measured, and on six questions whose answers are never
shells it bought nothing a reader scored, at +2 488 and +2 746 B on the 2 of
8 default-`k` payloads holding a linked shell and up to +20 817 B at `k` 50.
- **`--shell-parent`: off, with the reason rewritten.** "`okf consume` reads no
`parent` key" is false after A; the measured reason is the path saturation
above -- hit@k intact, the delivered set moved on 2 of 8 questions at the
default `k` and 3 of 8 at `k` 50. It moves the bytes of a bundle holding a
heading-only section and nothing else: 0 of 453 K2 bodies are heading-only
and the five-document folder builds byte-identical with it.
## 4. C: the index resolves the pointer
`structure._segment_lookup` keys `(source_file, segment_id)` off each
concept's own frontmatter (`DocumentStructure.declared`, no file read again)
and `resolve_structure` asks it first for a `parent` edge, inside the pointing
concept's document. A value no segment answers to is a document number and is
looked up as before; a pointer naming nothing keeps `?`. The rendering rule is
untouched -- a resolved relation renders as its subject -- so `parent: p1977?`
becomes `parent: p1977`. `consume`'s document prior reads only an index
entry's `label`, never its facets, so C moves no rank.
| row | before | after | denominator |
|---|---|---|---|
| `parent` facets resolved, R761 with the flag | 0 | **675** | 675 |
| `parent` facets with `?` | 675 | **0** (`?` anywhere: 792 -> 117, the unflagged bundle's own count) | 675 |
| concepts whose `derived:` names `parent` | 0 | **0** (the regex finds a fabricated positive) | 2 761 |
| files differing from A's flagged build | -- | 676: index files and `log.md`, **0 concepts** | 5 518 |
| R761 without the flag, five-document folder | -- | only `log.md` differs | 2 |
| segmented goldens | `parent: s0?` | `parent: s0` -- 4 lines; the four goldens `test_the_four_existing_goldens_are_untouched` guards are not among them | 4 |
| hit@k, both `k`; KP | 6/6; 1 | **6/6; 1**, delivered and withheld lists identical to A's, 16 of 16 | 16 |
| regression payloads | -- | **32 of 32 wholly identical to B's** | 32 |
| `okf check` | -- | **16 of 16 rc 0, 17 rules** | 16 |
## 5. D: the depth-7 trap
`_StsReader._walk` wrote one clipped level into the ATX heading and the
`OutlineMark` alike. The heading keeps the clip; the mark carries the declared
depth. The extracted text does not move one character.
**The gate, every row measured:**
| row | result | denominator |
|---|---|---|
| mark levels 1-7 | 28 / 118 / 500 / 1 141 / 868 / **97 / 9** (before: 106 at 6, 0 at 7) | 2 761 |
| extracted text | identical (`text_sha256` equal) | 1 |
| plan without the flag | **0 entries changed** | 2 761 |
| plan with the flag | **2 entries changed**: `p646` and `p647` (36.73211, 36.73212), `parent_id` `p644` (36.732) -> `p645` (36.7321) | 2 761 |
| shells pointing at the ancestor the `<sec>` nesting names | **710 of 710** (708 before); direct parent 533 (531) | 710 |
| concept count | 2 761 | -- |
| bundle with the flag vs C | the 2 concepts, their 2 index files, `log.md` | 5 518 |
| bundle without the flag, five-document folder | only `log.md` | 2 |
| the four goldens | untouched (the suite's own test, repository root) | 4 |
| regression payloads | **32 of 32 wholly identical to C's** | 32 |
| hit@k, both `k`; KP | **6/6; 1**, lists identical to C's, 16 of 16 | 16 |
`_unit_fold`'s unit level, the risk the order named, is not reached: the
declared route closes its candidates with `unit_fold=False`, and the plan
confirms it with 0 changes outside the two entries.
## 6. The 35 shells with no ancestor holding text
No pointer was invented for them. **13** sit at depth 1 with no ancestor at
all; **22** have ancestors, every one heading-only too. **28** are containers
with titled descendants; **7** are leaves. None has a direct lettered point.
| segment | label | depth | titled descendants |
|---|---|---:|---:|
| p3 | 2 | 1 | 4 |
| p9 | 4 | 1 | 8 |
| p11 | 4.2 | 2 | 3 |
| p21 | 8 | 1 | 7 |
| p28 | 8.7 | 2 | 0 |
| p32 | 12 | 1 | 0 |
| p33 | Hovedprosess 1 | 1 | 97 |
| p34 | 11 | 2 | 11 |
| p42 | 11.5 | 3 | 3 |
| p46 | 12 | 2 | 28 |
| p129 | 17 | 2 | 0 |
| p130 | 18 | 2 | 0 |
| p131 | Hovedprosess 2 | 1 | 191 |
| p132 | 21 | 2 | 25 |
| p170 | 23 | 2 | 46 |
| p217 | 24 | 2 | 56 |
| p225 | 24.4 | 3 | 6 |
| p228 | 24.43 | 4 | 3 |
| p249 | 24.6 | 3 | 9 |
| p306 | 27 | 2 | 16 |
| p323 | Hovedprosess 3 | 1 | 331 |
| p653 | Prosess 37 | 2 | 0 |
| p654 | Prosess 38 | 2 | 0 |
| p655 | Hovedprosess 4 | 1 | 218 |
| p874 | Hovedprosess 5 | 1 | 97 |
| p972 | Hovedprosess 6 | 1 | 94 |
| p975 | 62 | 2 | 0 |
| p1033 | 66 | 2 | 14 |
| p1067 | Hovedprosess 7 | 1 | 264 |
| p1108 | 72 | 2 | 19 |
| p1186 | 75 | 2 | 60 |
| p1238 | 75.6 | 3 | 7 |
| p1275 | 77 | 2 | 56 |
| p2108 | Hovedprosess 86 | 1 | 44 |
| p2153 | Hovedprosess 87 | 1 | 229 |
What they are in the source: the front-matter chapters 2, 4, 8 and 12 and the
main processes 1-7, 86 and 87 are containers whose text lives below them, not
above; 17, 18, 37, 38 and 62 are placeholder processes reserved for operation
and maintenance; 8.7 is a worked-example section; the first 12 is the chapter
that introduces the processes. A rule pointing at an ancestor WITHOUT text
would reach at most 22 of the 35, and only with a pointer to another heading.
It is not built.
## 7. Suites and gates
| when | working tree | frozen export |
|---|---|---|
| base | 1 753 passed, 1 skipped | -- |
| A red, after add and after commit | 11 failed (all new), 1 754 passed, 1 skipped | -- |
| A | 1 765 passed, 1 skipped | 1 764 passed, 1 skipped, 1 failed |
| B red | 7 failed (all new), 1 766 passed | -- |
| B / defaults | 1 773 / 1 774 passed, 1 skipped | 1 772 passed, 1 failed |
| C red / C | 3 failed (new), 1 776 / 1 779 passed | 1 778 passed, 1 failed |
| D red / D | 2 failed (new), 1 780 / 1 782 passed | 1 781 passed, 1 failed |
The one export failure is the known artefact
`test_segmented_golden.py::test_the_four_existing_goldens_are_untouched` (an
export is no git repository). `ruff check src tests tools`, `ruff format
--check .` and `mypy --strict src` were clean at every green commit, rc
captured directly. The first red commit (`a5cd7c5`) carried a test file `ruff
format` rewrote in the next one; no other file moved that way. `uv.lock` was
never staged.
## Honesty limits
- **Everything about shells is measured on ONE document (N = 1).** "Every
outline document" is a claim about the code, which reads the plan's level and
order; the fixtures and the five-document folder (0 shells of 26) are the
only other rows.
- **Consumption is one non-deterministic draw per question**, twelve runs on a
subscription (`ANTHROPIC_API_KEY` unset), judged by reading the answer
against the fasit section. Answer texts are not reproduced. The six
questions never have a shell as their answer, so form (b)'s value shows only
in the probe row, and that probe is one question chosen here, not a fasit.
- **"Byte-identical" after A and B excludes `budget.known_positive`.** The
contract document is the known-positive, so whole-payload identity is 0 of
32 across A and B and 32 of 32 across B, C and D.
- **The K2 statement for `--shell-parent` is read off the pinned bundle** (0 of
453 bodies heading-only), not off a rebuild with the flag.
- **The link's ranking cost is attributed only where a newcomer matches
through the link alone;** the rest of the delivered-set movement, and any
effect of the heavier excerpts on the knapsack, is not decomposed.
- **The instrument is someone else's, with two lines changed** (three for form
(b)), and it scores a title or a section-number pair, not an answer.
- **C's "resolved" says a concept of the same document answers the id,** not
that it is the right ancestor; D's 710 of 710 is what checks that, against
the source's own nesting.
- **Conformance is the floor and never the proof.** `okf check` is green on
all 48 flagged payloads here, including the known-negative question's, which
answers nothing.

285
docs/2026-09-11-k3-runde22-yaml-lesbar-frontmatter.md
View file

@ -1,285 +0,0 @@
# K3 round 22 — frontmatter a YAML reader reads back the same
Order K3-22, 2026-09-11. Red `06e61a5`, fix `ed0418f`, base `0308169`
(v0.8.3). PyYAML 6.0.3, guard 1.3.0, Python 3.14.0 (working tree and the
frozen exports, run through the repository's own interpreter with the export
first on `PYTHONPATH`; `__file__` verified under the scratch export for both).
## Deviations, first
1. **The order's form for the flow mapping cannot ship.** It asked for
`sources: [{ resource: "https://…?x=nb", title: … }]` -- a double-quoted
leaf inside the flow mapping. PyYAML reads that; the pinned guard does not.
Guard 1.3.0 `okf.parse_frontmatter` refuses ANY quote inside a flow mapping
("a quoted scalar inside a flow mapping is not a supported form"), double
or single, measured below. Writing it would have moved 4 605 PyYAML
failures to 4 605 guard failures, and broken the promise
`tests/test_provenance.py` already holds ("what Door B writes must survive
the guard's own frontmatter grammar"). So a flow leaf is VALIDATED, never
quoted: a value no flow form both readers accept is refused with the door's
existing code. Quoting is used where the guard admits it -- a block scalar.
2. **The failure is bigger in this repository's own output than the order's
table shows, and in the other place.** The order's table has no bundle
`okf` built with its defaults. The pinned K2 default bundle fails
`safe_load` on **41 of 455** frontmatter blocks, and **1 more** parses
while PyYAML reads a different value than ours (a title carrying `" #"` --
PyYAML stops at the comment, no error). All 42 are block `title` scalars
(FEIL B); **0** `sources` values okf wrote failed anywhere measured. The
4 605 `?` failures (FEIL A) come from a consumer's own `sources` writer,
which does not go through this library.
3. **The consumer repository is at `f783e52`, not `6fad6a2`.** The hit@k
instrument was taken from `6fad6a2`, as ordered, into private scratch with
the payload path changed (2 lines differ from the original). Its tree was
clean before and after (`git status --porcelain`: 0 lines both times).
4. **The default-`k` hit@k run passes four flags, not five** (`--bundle
--questions --okf --out`, no `--k`), reproducing how the previous round ran
it; the `k = 50` run passes all five.
## 1. The standard
SPEC (`_okf-canonical` `ad30107`), verbatim:
- **§ 4, opening:** "Every concept is a UTF-8 markdown file with two parts:
1. A **YAML frontmatter block**, delimited by `---` on its own line at the
start of the file and a closing `---` on its own line." (l. 155-158)
- **§ 11, Conformance, point 1:** "Every non-reserved `.md` file in the tree
contains a parseable YAML frontmatter block." (l. 740-741)
**What the spec does not say, and it is a weakness of the spec:** it names no
YAML version and no subset -- 0 hits for a version number, "YAML version" or
"subset", with the query validated against the known-positive `YAML` (5 hits).
"Parseable" is therefore whatever reader the consumer has, and PyYAML is the
common one. That is why the reader, not this corpus, decides the rule.
## 2. The measurement, reproduced
`yaml.safe_load` over the block between the two fences; the denominator is
files WITH a frontmatter block (index files without one are not counted --
e.g. 5 514 `.md` against 2 757 with frontmatter in the first row).
| tree | with frontmatter | OK | FAIL | order's figure |
|---|---|---|---|---|
| consumer R761 (N-route) | 2 757 | 1 | 2 756 | same |
| consumer N100 | 447 | 1 | 446 | same |
| consumer N200 | 1 134 | 1 | 1 133 | same |
| consumer N500 | 271 | 1 | 270 | same |
| consumer R761, built by `okf build` | 2 763 | 2 762 | 1 | same |
| this repository's `tests/fixtures` | 12 | 12 | 0 | same |
| **K2 default bundle (pinned)** | **455** | **414** | **41** (+1 misread) | not in the order |
| R761 HTML build (v0.8.3) | 3 208 | 3 207 | 1 | not in the order |
| R761 XML build (v0.8.3) | 2 763 | 2 762 | 1 | not in the order |
| five-document project (v0.8.3) | 29 | 29 | 0 | not in the order |
| `examples/` · `skills/` | 14 · 3 | 14 · 3 | 0 · 0 | not in the order |
Every figure the order gave reproduces exactly.
## 3. The two causes
**FEIL A -- `?` inside a flow mapping.** The consumer writes
`sources: [{ resource: https://<viewer>/api/…/860019?languageCode=nb, title: … }]`
unquoted. PyYAML's scanner ends a plain scalar in FLOW context at `?`:
```
while parsing a flow mapping ... expected ',' or '}', but got '?'
```
The colon in `https:` is not the cause: `https://h.no/a/1` in the same place
parses.
**FEIL B -- a block scalar written verbatim.** The profile emitter wrote every
value as `key: value`. A title ending in `:` gives
```
mapping values are not allowed here
```
and the K2 bundle adds a leading `- ` ("sequence entries are not allowed
here"), a leading `*` or `**` ("expected alphabetic or numeric character" /
an undefined alias) and `": "` inside a title.
## 4. The constructs, measured
PyYAML 6.0.3 (`BaseLoader`), and guard 1.3.0 `okf.parse_frontmatter`:
| construct | PyYAML | guard |
|---|---|---|
| `sources: [{ resource: https://h.no/a/1?x=nb, title: T }]` | FAIL, `got '?'` | admits |
| `sources: [{ resource: https://h.no/a/1, title: T }]` | OK | admits |
| `sources: [{ resource: a=b&c.pdf, title: T }]` | OK | admits |
| `sources: [{ resource: fil.pdf, title: N100:2023 }]` | OK | admits |
| `sources: [{ resource: fil.pdf, title: N100: 2023 }]` | FAIL, `got ':'` | -- |
| `sources: [{ resource: fil.pdf, title: Kap #3 }]` | FAIL, `<stream end>` | -- |
| `sources: [{ resource: fil.pdf, title: -x }]` | OK | admits |
| `sources: [{ resource: fil.pdf, title: *x }]` | FAIL, alias | -- |
| `sources: [{ resource: "https://h.no/a/1?x=nb", title: T }]` | OK | **REFUSES** |
| `sources: [{ resource: 'https://h.no/a/1?x=nb', title: T }]` | OK | **REFUSES** |
| block `sources:` / `- resource: https://h.no/a/1?x=nb` / `title: T` | OK | admits |
| `title: Eksempel kontur:` | FAIL, mapping values | refuses |
| `title: "Eksempel kontur:"` | OK | admits (keeps the quotes) |
| `title: N100: 2023` · `title: - punkt` · `title: *stjerne` | FAIL | -- |
| `title: Kap #3` | **OK, as `Kap`** | admits |
| `utgave: R761 Prosesskoden:2025` · `title: Kap#3` | OK | admits |
## 5. The form, and why it is the narrowest
**Block scalars: double-quote exactly the values K3-19's rule refuses.** The
rule (`no leading indicator, no ": ", no " #", no trailing ":", no tab or line
break`) is now `profiles.yaml_block_plain`, and over every top-level value in
eleven measured trees it agrees with PyYAML read verbatim on every one: **0
refused that PyYAML returns verbatim, 0 kept that it does not** (K2 5 524
values, the HTML build 38 752, the XML build 35 177, the five consumer bundles
and the rest). Quoting everything would move bytes in every concept of every
bundle and buy no valid file; this moves only the values that were failing.
**Double, never single -- counted.** Values wrapped in a surrounding `"` pair:
**0** in twelve trees (the eleven plus `skills/`). Wrapped in `'`: **11 193**,
all in the consumer bundles (10 937 + 67 + 86 + 103), written that way on
purpose. A reader that unquotes `"` changes the meaning of no value that
exists; one that also unquoted `'` would change 11 193.
**Flow leaves: no quoted form exists, so validation.** Plain fails PyYAML on
`?`, `,[]{}`, `": "`, `" #"`, a trailing `:` and a leading indicator; quoted
fails the guard. `profiles.yaml_flow_plain` is the block rule plus
`,[]{}?'"`, with one exception the block rule does not make: a leading `-`
followed by a non-space, which both readers take (`del/-utkast.pdf`). Refusing
that would refuse a document both read; the block rule keeps refusing it
because it also decides which spec points are written at all, and in a block a
refusal costs only a pair of quotes. The refusal uses each door's existing
code: `inbox_source_file_unaddressable` (also when the FILE NAME, which becomes
the entry's `title`, is not plain), `inbox_source_title_unaddressable`,
`source_reference_unquotable` (Door A), `run_frontmatter_invalid`
(`--frontmatter` with a flow value).
Over a constructed list of 39+ values (in `tests/test_yaml_frontmatter.py`)
the safe direction is required and the over-refusals are pinned: block keeps 0
PyYAML would misread and over-refuses 3 (`-punkt`, `?spm`, `:kolon`, which get
quoted); flow keeps 0 that PyYAML or the guard would refuse and over-refuses 0.
**Readers read both forms.** `materialize.parse_frontmatter`, the index
reader's and the structure reader's copies, and both `read_sources` branches
unquote a `"`-wrapped value, decoding `\"` and `\\` and nothing else. Other
backslash escapes (`\n`, `\t`, `\x..`, `\u....`) are kept as written: the
emitter never writes them. `'`-wrapped values are untouched -- except in the
structure reader, which already unquoted both forms before K3-22 on its own
rule (`version: '2021'` is a string) and keeps doing so. The flow-mapping
split is quote-aware, so `{ title: "a, b" }` is one pair; a `"` opens a quoted
scalar only where a value can start, so a plain value with a `"` in its middle
splits as before.
The generated `SKILL.md` header goes through the block rule too:
`description` carries the root index's `bundle_id` raw.
## 6. What moved bytes, and what did not
Each tree built twice, from the base export and the fix export, and compared
with `diff -rq`:
| build | concepts / md | files that differ | the line |
|---|---|---|---|
| five-document project | 26 / 52 | **0** | -- |
| R761 XML (`--bundle-id r761-prosesskoden-2025 --okf-version 0.2 --ingested-at 2026-09-08T12:00:00Z`) | 2 761 / 5 501 | **1** | a title ending in `:`, now double-quoted |
| R761 HTML, 828 files | 3 206 / 6 015 | **1** | the same section's title |
| K2 default, rebuilt (43 documents) | 453 / 864 | **42** | 42 `title` lines, now double-quoted: the 41 that failed `safe_load` and the 1 that parsed to a truncated title |
The K2 base rebuild is byte-identical to the pinned K2 artefact on every file
but `log.md`, which the corpus harness had not yet written when the two were
compared -- so the 42 lines are K3-22's and nothing else's.
- **Goldens and fixtures: 0 bytes.** The golden tests are byte-exact and pass;
`tests/fixtures` is untouched and still 12 of 12.
- **Readers on existing bundles: 0 differences.** Base readers and fix readers
return identical `parse_frontmatter` and `read_sources` results on **25 273
of 25 273** files across nine trees (the five consumer bundles, the pinned K2
bundle, the HTML and XML builds, `tests/fixtures`). N100/N200/N500 are read
exactly as before.
- **One behaviour moved, and it is a refusal.** `okf build --frontmatter
'sources=[{ resource: <URL with ?>, … }]'` now exits 2 and writes nothing.
Measured on R761 with K3-19's flags: the base export writes **2 761 of 2 761**
concepts PyYAML refuses (2 760 on `?`, 1 on the title), the fix export
refuses the run. Two K3-19 tests built with such an address; they now build
with one without `?`, and a new test holds the refusal.
- **The K2 pin** holds concept count and hit ranks, not bytes, so it did not
need moving: **7 of 7 passed** in the frozen fix export, with the gold set
copied in (without it the pin is 7 skipped, which is not a measurement).
## 7. Acceptance
| check | result |
|---|---|
| PyYAML valid, fix builds | five-document 28/28 · R761 XML 2 763/2 763 · HTML 3 208/3 208 · K2 **454/454** (base 413/454) |
| dict-equal to `parse_frontmatter` + `read_sources`, every file | 28/28 · 2 763/2 763 · 3 208/3 208 · K2 **454/454** (base 412/454) |
| generated `SKILL.md` headers, `safe_load` | **4 of 4** (five-document, R761 XML, HTML, K2) |
| hit@k R761, base = fix | S1-S6 hit@1/8/50 **6/6 · 6/6 · 6/6** at default `k` and at `k = 50`; KP rank **1** at both; KN not delivered at both |
| `okf check` on every payload | **32 of 32** exit 0, "conformant: 16 rules … 0 findings" (`len(RULES)` = 16) |
| `grep -rc "import yaml" src/` | **0 on each of 22 files** |
| `[project.dependencies]` | unchanged, `["llm-ingestion-guard>=1.2,<2.0"]`; `uv.lock` +2 lines, the dev dependency |
| suite, working tree | 1 667 passed / 1 skipped before; **1 753 / 1** after (85 + 1 new) |
| suite, frozen fix export from a scratch cwd | 1 750 / 1 / 3 failed: the two known (`test_segmented_golden` four-goldens, `test_guard_adapter` cwd-relative path) and `test_cli_build::…installed_copy`, which fails identically on the base export under this runner -- the runner's `PYTHONPATH` reaches the subprocess the test installs into |
The hit@k base row reproduces the previous round's before anything else was
read.
## 8. Found, and outside this order
**The guard's parser refuses nearly every segmented concept okf writes, for a
reason that is not YAML.** Guard 1.3.0 admits a flow sequence of MAPPINGS only
("a flow sequence admits flow mappings only"), and okf writes scalar flow
sequences at top level: `source_offset: [1, 24]`, `source_lines`,
`source_pages`, `derived: [references]`, `references`. Measured with
`okf.parse_frontmatter` over whole frontmatters: five-document 26 of 28
refused, R761 XML 2 761 of 2 763, HTML 3 206 of 3 208, pinned K2 412 of 455 --
identical on base and fix, so it predates K3-22. PyYAML reads all of them.
What this means for Door C importing an okf bundle is **not measured** here
(`import_bundle` was not run).
## Honesty limits
- **PyYAML is one reader.** The rules are validated against it and the guard;
a YAML 1.2 reader accepts more (`?` inside a flow scalar), so the rules are
stricter than 1.2, never looser than PyYAML on the values measured.
- **The flow-collection exemption trusts its producers.** The emitter writes a
value shaped `[...]` or `{...}` as it stands. Over the eleven trees only
structural keys carry that shape and no `title` does; a producer that built a
malformed collection would still be written. The run-stated route
(`--frontmatter`) is validated; the internal producers are measured, not
checked at emission.
- **Two escapes.** `\"` and `\\` are written and decoded; any other escape is
neither.
- **The 217 descriptions stay omitted.** A derived spec point carrying `": "`
is still not written (K3-19), although it could now be written quoted.
Writing it would add values, which this order excluded.
- **The constructed list is constructed.** The "0 / 0" over real values is the
measurement; the constructed list pins the directions on shapes the corpora
may not carry.
- **Conformance is the floor, never the proof.** § 11 point 1 now holds on
every build measured; that says the frontmatter parses, not that a bundle is
good.
## Corrected
`README.md` said of the flow form: "both are valid YAML, and a real YAML
consumer recovers the same structure from either". Measured false: an unquoted
URL with a query string inside a flow mapping is not valid YAML for PyYAML.
The paragraph now states the limit, and `CLAUDE.md` carried the same sentence
and is corrected with it.
## Reproduce
```sh
# PyYAML validity, denominator = files with a frontmatter block
python3 - <<'EOF'
import sys, yaml, pathlib
root = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else ".")
fm = ok = 0
for p in root.rglob("*.md"):
lines = p.read_text(encoding="utf-8").splitlines()
if not lines or lines[0].strip() != "---":
continue
fm += 1
try:
yaml.safe_load("\n".join(lines[1:lines.index("---", 1)])); ok += 1
except yaml.YAMLError:
pass
print(f"with frontmatter={fm} ok={ok} fail={fm - ok}")
EOF
```

2
docs/2026-09-11-k3-runde8-tabellblokk-og-siste-spenn.md
View file

@ -181,7 +181,7 @@ The opt-out `--no-close-span-gaps` reproduces the previous spans.
remove the wrong classification, and a contents run is still discarded as a
concept. Whether a run of numeric rows should be a contents listing at all
is a separate question with its own measurement.
* **`MIN_SHARED_PREFIX`** (round 5's `undersjøisk`/`under`, df 159 of 270).
* **`MIN_SHARED_PREFIX`** (round 5's `under…`/`under`, df 159 of 270).
Not re-measured. It stands.
* **The `{#…}` leak, the `pptx` fasit, `|F|` for positions 0 and 3.** All four
are with the operator with their numbers delivered; none was re-opened here.

4
docs/2026-09-11-r4-release-v082.md
View file

@ -80,8 +80,8 @@ cd <export> && uv sync --frozen --extra extract
| `okf check` on `skills/okf-consume/` and its payload | **conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings**, rc 0 |
| `okf project` on the five-document folder | **26 concepts of 5 documents, 52 md, 0 of 5 left out** |
| HTML corpus | **proposed 828 plan(s); 0 document(s) with no boundary; 0 unreadable**, 3 206 concepts / 6 015 md |
| R761 XML | **2 761 concepts** at shipped defaults, no arm flag; the folder holds 1 file at top level and 110 in total, and 109 are images under `graphics/` (43 png, 41 jpg, 25 bmp), coded unreadable |
| hit@1 / 8 / 50 on the R761 bundle | **6/6 / 6/6 / 6/6**, N = 6 scored of 8 asked, known-positive at rank 1 |
| reference standard, XML | **2 761 concepts** at shipped defaults, no arm flag; the folder holds 1 file at top level and 110 in total, and 109 are images under `graphics/` (43 png, 41 jpg, 25 bmp), coded unreadable |
| hit@1 / 8 / 50 on the reference standard's bundle | **6/6 / 6/6 / 6/6**, N = 6 scored of 8 asked, known-positive at rank 1 |
| round 17's known-negative on the five-document bundle | the section the question names at **rank 1** (6 793 B), the neighbour at rank 2 (10 590 B); both concepts exist and both are delivered of 8 |
The suite was not run inside the export this round; R3 and K3-18 did, with the

8
docs/2026-09-11-r5-release-v083.md
View file

@ -105,9 +105,9 @@ cd <export> && uv sync --frozen --extra extract
| `okf check` on `skills/okf-consume/` and its payload | **conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings**, rc 0 |
| `okf project` on the five-document folder | **26 concepts of 5 documents, 52 md, 0 of 5 left out** |
| HTML corpus | **proposed 828 plan(s); 0 document(s) with no boundary; 0 unreadable**, 3 206 concepts / 6 015 md |
| R761 XML | **2 761 concepts** at shipped defaults, no arm flag; the folder holds 1 file at top level and 110 in total, 109 of them images under `graphics/` (43 png, 41 jpg, 25 bmp), coded unreadable |
| R761 document directory (new this round) | **`r761-prosesskoden/`**, not the delivery file's UUID stem; denominator 1 document |
| hit@1 / 8 / 50 on the R761 bundle | **6/6 / 6/6 / 6/6** at the default `k` and at `k` = 50, N = 6 scored of 8 asked |
| reference standard, XML | **2 761 concepts** at shipped defaults, no arm flag; the folder holds 1 file at top level and 110 in total, 109 of them images under `graphics/` (43 png, 41 jpg, 25 bmp), coded unreadable |
| reference standard's document directory (new this round) | **its declared `<doc-number>`**, not the delivery file's UUID stem; denominator 1 document |
| hit@1 / 8 / 50 on the reference standard's bundle | **6/6 / 6/6 / 6/6** at the default `k` and at `k` = 50, N = 6 scored of 8 asked |
| the known-positive (new this round) | **rank 1 at the default `k` and rank 1 at `k` = 50**, denominator 1 |
| `--shell-parent` (new this round) | **675 of 710** heading-only concepts carry `parent:`; **0 of 2 761** without the flag, and 0 non-heading-only concepts with it; `--shell-parent` and `--no-shell-parent` both in `okf build --help` |
| round 17's known-negative on the five-document bundle | the section the question names at **rank 1** (6 793 B), the neighbour at rank 2 (10 590 B); both concepts exist and both are delivered of 8 |
@ -133,7 +133,7 @@ The suite was not run inside the export this round.
- **828 of 828 and 2 761 of 2 761 are ONE publisher's files from ONE product.**
They say the reader lost nothing against its own denominator; they say
nothing about documents nobody in that pipeline wrote.
- **`r761-prosesskoden` is ONE publisher's `<doc-number>` on ONE document.** It
- **The directory name is ONE publisher's `<doc-number>` on ONE document.** It
says the tag carries K3-19's directory naming; it says nothing about how
other publishers fill `<std-ident>`.
- **675 of 710 are shells in ONE document, and `okf consume` does not read

6
docs/2026-09-11-r6-release-v084.md
View file

@ -135,9 +135,9 @@ cd <export> && uv sync --frozen --extra extract
| `okf check` on `skills/okf-consume/` and its payload | **conformant: 17 rules over 3 excerpts and 0 withheld entries, 0 findings**, rc 0 |
| `okf project` on the five-document folder | **26 concepts of 5 documents, 52 md, 0 of 5 left out**; the folder holds 5 files before and after |
| HTML corpus | **proposed 828 plan(s); 0 document(s) with no boundary; 0 unreadable**, 6 015 md |
| R761 XML | **2 761 concepts** at shipped defaults, no arm flag; the folder holds 1 file at top level and 110 in total, 109 coded `extractor_unknown` |
| R761 document directory | **`r761-prosesskoden/`**; denominator 1 document |
| hit@1 / 8 / 50 on the R761 bundle | **6/6 / 6/6 / 6/6** at the default `k` and at `k` = 50, N = 6 scored of 8 asked |
| reference standard, XML | **2 761 concepts** at shipped defaults, no arm flag; the folder holds 1 file at top level and 110 in total, 109 coded `extractor_unknown` |
| reference standard's document directory | **its declared `<doc-number>`**; denominator 1 document |
| hit@1 / 8 / 50 on the reference standard's bundle | **6/6 / 6/6 / 6/6** at the default `k` and at `k` = 50, N = 6 scored of 8 asked |
| the known-positive | **rank 1 at the default `k` and rank 1 at `k` = 50**, denominator 1 |
| round 17's known-negative on the five-document bundle | the section the question names at **rank 1** (6 793 B), the neighbour at rank 2 (10 590 B); both concepts exist and both are delivered of 8 |
| guard 1.4.0 against 1.3.0 on that bundle (new) | 28 frontmatter blocks in 52 md: **1.4.0 refuses 0 of 28, 1.3.0 refuses 26 of 28** (26 of 27 concept files), every refusal "a flow sequence admits flow mappings only" -- `source_offset` pairs and a `[references]` list among them |

80
docs/2026-09-12-g37-terskler.md
View file

@ -64,10 +64,10 @@ three of the five premises moved.
| premise in the order | re-measured | verdict |
|---|---|---|
| 13 extensions in the registry | `_CORE_EXTRACTORS` 7 + `_OPTIONAL_EXTRACTORS` 6 = **13** | holds |
| four evidence corpora under `~/repos/vegnormal-okf/build/ferdig/` | present: `n100-2023`, `n200-2024`, `n500-2024`, `r761-2025` — plus `r761-2025-generisk` and `r761-2025-v080`, two more R761 builds the order does not name | **holds, and is incomplete** |
| "N100 skal gi PASS på de målte typene" | `grep -rl '^source_file:' n100-2023 --include='*.md' | wc -l` → **0 of 446 concepts**. Same for `n200-2024` (0 of 1 133), `n500-2024` (0 of 270) and `r761-2025` (0 of 2 756) | **cannot hold**: three of the four name no file type at all, so no type of theirs can be PASS or FAIL |
| "41,6 % grenser" | not in this repository when the order arrived — measured on the base commit, `git grep -c "41[.,]6" d310dbb -- docs src` → **0 files**. It is `vegnormal-okf`'s: `docs/2026-09-10-r2-r761-produkt.md:452`, 1 148 of 2 761 declared STS sections, against the XML arm's 2 759 of 2 761 | **holds, and the owner is another repo** |
| "0 av 6 hit@k" on the HTML arm | two published numbers for the same 828-document bundle: **0 of 6** (vegnormal, their six fasit questions, 10.09) and **2 of 6 → 4 of 6 with `--source-quota 2`** (this repo, `docs/2026-09-10-k3-runde11-html-og-dominans.md:200,217`, questions S1–S6) | **holds for one instrument, not for the other** |
| four evidence corpora under a downstream consumer repository's finished-build directory | present: `corpus-a`, `corpus-b`, `corpus-c`, `std-consumer` — plus `std-xml-generic` and `std-xml-v080`, two more builds of the reference standard the order does not name | **holds, and is incomplete** |
| "the first corpus shall PASS on the measured types" | `grep -rl '^source_file:' corpus-a --include='*.md' | wc -l` → **0 of 446 concepts**. Same for `corpus-b` (0 of 1 133), `corpus-c` (0 of 270) and `std-consumer` (0 of 2 756) | **cannot hold**: three of the four name no file type at all, so no type of theirs can be PASS or FAIL |
| "41,6 % grenser" | not in this repository when the order arrived — measured on the base commit, `git grep -c "41[.,]6" d310dbb -- docs src` → **0 files**. It is the consumer's (one of its own reports), 1 148 of 2 761 declared STS sections, against the XML arm's 2 759 of 2 761 | **holds, and the owner is another repo** |
| "0 av 6 hit@k" on the HTML arm | two published numbers for the same 828-document bundle: **0 of 6** (the consumer, their six fasit questions, 10.09) and **2 of 6 → 4 of 6 with `--source-quota 2`** (this repo, `docs/2026-09-10-k3-runde11-html-og-dominans.md:200,217`, questions S1–S6) | **holds for one instrument, not for the other** |
The last row is the reason the gate reports a denominator on every line: two
true numbers about one bundle, four apart, because the question set differs.
@ -82,13 +82,13 @@ and 446 concepts — four bundles, both methods.
| bundle | producer | concepts | file types present |
|---|---|---|---|
| `K2-bundle-default-20260912` | Door B (pinned reference) | 453 | `.pdf` 32 docs, `.docx` 5, `.xlsx` 2 |
| `r761-2025-generisk` | Door B | 2 761 | `.xml` 1 doc |
| `sk2-bundle-default` | Door B | 2 182 | `.pdf` 1 doc |
| `860019-mdb-100` | Door B | 3 206 | `.md` 828 docs |
| `n100-2023` | vegnormal's own pipeline | 446 | none — no `source_file` |
| `n200-2024` | vegnormal's own pipeline | 1 133 | none |
| `n500-2024` | vegnormal's own pipeline | 270 | none |
| `r761-2025` | vegnormal's own pipeline | 2 756 | none |
| `std-xml-generic` | Door B | 2 761 | `.xml` 1 doc |
| `std-pdf-default` | Door B | 2 182 | `.pdf` 1 doc |
| `html-arm-828` | Door B | 3 206 | `.md` 828 docs |
| `corpus-a` | the consumer's own pipeline | 446 | none — no `source_file` |
| `corpus-b` | the consumer's own pipeline | 1 133 | none |
| `corpus-c` | the consumer's own pipeline | 270 | none |
| `std-consumer` | the consumer's own pipeline | 2 756 | none |
| `b3` | — | — | no `index.md`; the gate refuses to walk a directory (SS 9.2) and exits 2 |
**The four corpora the order names as the evidence base carry no file type.**
@ -123,8 +123,8 @@ limit is "a `1/1` is not a rate" (`docs/2026-09-08-k3-runde2-per-filtype.md`).
### Candidates measured and not shipped
Three fasit-free metrics were measured over the same four Door B bundles, with
the known-bad arm (`860019-mdb-100`, 1 148 of 2 761 boundaries) and the
known-good arm (`r761-2025-generisk`, 2 759 of 2 761) named in advance:
the known-bad arm (`html-arm-828`, 1 148 of 2 761 boundaries) and the
known-good arm (`std-xml-generic`, 2 759 of 2 761) named in advance:
| candidate | known-bad arm | known-good arm | pdf arm | K2 reference | verdict |
|---|---|---|---|---|---|
@ -156,8 +156,8 @@ STATE rather than taken here.
The 41.6 % arm is `UNMEASURED` under this gate, and that is the honest answer
rather than a hidden one: the metric that fells it is `boundary_share`, which
needs the publisher's own declared structure. R761 has one (2 761 STS `<sec>`
titles, `~/repos/vegnormal-okf/build/sk2-fasit-2761.json`), and 4 of the 9
needs the publisher's own declared structure. The reference standard has one (2 761 STS `<sec>`
titles, `<consumer repository>/fasit-2761.json`), and 4 of the 9
bundles above have no fasit at all. A `--fasit` input, a threshold on
`boundary_share`, and the operator's bar are one further round, named in STATE
and not begun here. **That round is § 7 below** (G37b, order
@ -183,7 +183,7 @@ records traps.
### § 7.1 — P1: the normalisation, measured on a known-positive first
The fasit is `~/repos/vegnormal-okf/build/sk2-fasit-2761.json` — 665 134 bytes,
The fasit is `<consumer repository>/fasit-2761.json` — 665 134 bytes,
a JSON list of exactly **2 761** objects, each carrying `title` and `norm`
(`sec_type`, `depth`, `label`, `outline_page` and `id` as well). Every `norm` is
unique, and so is every `title`.
@ -192,14 +192,14 @@ unique, and so is every `title`.
lowercase, reproduces the fasit's own `norm` from its own `title` on **2 761 of
2 761** rows. Two candidates were run first and fell on the same file:
lower-casing and keeping only alphanumerics scores **58 of 2 761** (it eats the
`.` in `2.1Hovedprosesser` and the `-` in `3Sted- og elementkode`). Unicode NFC
`.` in `2.1Grunnoppskrifter` and the `-` in `3Mel- og gjærtyper`). Unicode NFC
makes no difference here — 2 761 of 2 761 titles and norms are already NFC — and
the rule is the one `vegnormal-okf`'s own measuring script applies, so a number
the rule is the one the consumer's own measuring script applies, so a number
produced here and a number produced there are the same number.
**P1's own bar is 99 % on the known-good arm, and the literal reading of it
fails — for a reason that is measured and is not the normalisation.** On
`r761-2025-generisk` (the declared-structure arm, 2 761 concepts):
`std-xml-generic` (the declared-structure arm, 2 761 concepts):
| match form | recovered of 2 761 |
|---|---|
@ -208,27 +208,27 @@ fails — for a reason that is measured and is not the normalisation.** On
| either | **2 759** (99.9 %) |
The cause is this repository's own id form: STS glues the numbering token into
`<title>` (`11.1Fastmerker`), and okf's default route moves that token into the
`<title>` (`11.1Surdeigsstarter`), and okf's default route moves that token into the
concept id (`11-1/…`) while keeping the residual as the title. A gate scoring
the literal form alone would report a 99.9 % arm as 0.8 % and call it a
segmentation defect. Both forms therefore ship, neither as a fallback for the
other: the literal form wants the title *with* its token, the pair form *without*,
and no single bundle can offer both. The `r761-2025-d1` build is the control in
and no single bundle can offer both. The `std-d1` build is the control in
the opposite direction — literal **2 727**, paired **0**.
That decomposition is not this round's invention either. It is the M8 correction
`vegnormal-okf` took verbatim from this repository's own round-14 report; the
the consumer took verbatim from this repository's own round-14 report; the
instrument here reproduces their two forms so the two repositories cannot
silently measure different things.
**Known-positive delivered: 2 759 of 2 761 (99.9 %)**, the published number to
the unit. **Known-negative delivered: `860019-mdb-100` at 1 148 of 2 761
the unit. **Known-negative delivered: `html-arm-828` at 1 148 of 2 761
(41.6 %)**, also to the unit — the arm the bundle-only gate returned
`UNMEASURED`/exit 3 for.
### § 7.2 — P2: the threshold rests on one product, and says so
The fasit describes **one** document: R761 Prosesskoden:2025. Every number
The fasit describes **one** document: a 701-page reference standard. Every number
below is that product's. The bar is declared with `corpora = 1` in the code
(`quality.BOUNDARY_THRESHOLD`), the row prints `N = 1 corpus` on every run, and
the printed line also states what `--fasit` is: **an assertion by the caller**
@ -236,7 +236,7 @@ that this bundle is a build of the document the fasit describes — the same
posture `okf consume --ref` has.
That is not a formality. Measured: the pinned 43-document K2 reference bundle
scores **0 of 2 761** and `n100-2023` scores **0 of 2 761**. Both read `FAIL`,
scores **0 of 2 761** and `corpus-a` scores **0 of 2 761**. Both read `FAIL`,
and in both cases the right reading is *the assertion was wrong*, not *the
bundle is bad*. A gate that could tell those two apart would need a bar on
"does this fasit describe this bundle", and that bar would have to be read off
@ -244,23 +244,23 @@ the bundles it judges.
### § 7.3 — every bundle measured, with the fasit
`okf quality <bundle> --fasit ~/repos/vegnormal-okf/build/sk2-fasit-2761.json`
`okf quality <bundle> --fasit <consumer repository>/fasit-2761.json`
| bundle | concepts | literal | paired | recovered of 2 761 | verdict | rc |
|---|---|---|---|---|---|---|
| `r761-2025-generisk` (reference) | 2 761 | 22 | 2 737 | **2 759** (99.9 %) | PASS | 0 |
| `r761-2025-v080` | 2 761 | 22 | 2 737 | **2 759** (99.9 %) | PASS | 0 |
| `r761-2025` | 2 756 | 20 | 2 732 | **2 752** (99.7 %) | FAIL | 1 |
| `r761-2025-d1` (vegnormal's own pipeline) | 2 728 | 2 727 | 0 | **2 727** (98.8 %) | FAIL | 1 |
| `860019-mdb-100` (known-bad arm) | 3 206 | 1 130 | 28 | **1 148** (41.6 %) | FAIL | 1 |
| `n100-2023` (another product) | 446 | 0 | 0 | **0** | FAIL | 1 |
| `std-xml-generic` (reference) | 2 761 | 22 | 2 737 | **2 759** (99.9 %) | PASS | 0 |
| `std-xml-v080` | 2 761 | 22 | 2 737 | **2 759** (99.9 %) | PASS | 0 |
| `std-consumer` | 2 756 | 20 | 2 732 | **2 752** (99.7 %) | FAIL | 1 |
| `std-d1` (the consumer's own pipeline) | 2 728 | 2 727 | 0 | **2 727** (98.8 %) | FAIL | 1 |
| `html-arm-828` (known-bad arm) | 3 206 | 1 130 | 28 | **1 148** (41.6 %) | FAIL | 1 |
| `corpus-a` (another product) | 446 | 0 | 0 | **0** | FAIL | 1 |
| `K2-bundle-default-20260912` (another product) | 453 | 0 | 0 | **0** | FAIL | 1 |
### § 7.4 — the bar, and why it sits where it sits
| metric | threshold | N | source |
|---|---|---|---|
| `boundary_share` | **2 759/2 761** | 2 761 declared boundaries, **1 corpus** | `r761-2025-generisk` against `sk2-fasit-2761.json` |
| `boundary_share` | **2 759/2 761** | 2 761 declared boundaries, **1 corpus** | `std-xml-generic` against `fasit-2761.json` |
Set at the value measured on the pinned artifact rather than at a rounder
number nearby — the same rule the two bars in § 4 follow, and the same
@ -268,10 +268,10 @@ semantics: **a regression bar, not a quality claim**. `PASS` means *no worse
than that build of that product*.
**It is a tight bar and the cost is stated rather than hidden:** two of the four
R761 builds measured fall under it, at 2 752 and 2 727 of 2 761 — 0.25 % and
builds of the reference standard measured fall under it, at 2 752 and 2 727 of 2 761 — 0.25 % and
1.2 % below the reference. The class separation is nevertheless wide: **any bar
between 1 148/2 761 (41.6 %) and 2 727/2 761 (98.8 %) separates the known-bad
arm from every R761 build measured.** The shipped bar is the strictest point of
arm from every build of the reference standard measured.** The shipped bar is the strictest point of
that interval because it is the only one in it that is read off a pinned
artifact instead of off the bundles being judged. Moving it inside that interval
is the operator's call, and the interval is published here so the call can be
@ -284,17 +284,17 @@ of four rows gives `UNMEASURED` rather than a share.
### § 7.5 — what did not change
- **Without `--fasit`, nothing moves.** `okf quality <bundle>` on
`860019-mdb-100` still prints one `.md` row, no boundary row, and exits **3**.
`html-arm-828` still prints one `.md` row, no boundary row, and exits **3**.
A test holds it, and the word "boundary" does not appear in the per-type half
of the output.
- **No version bump, no tag, no push.** v0.9.0 was already bumped by G37 and is
still untagged; the tag is the operator's.
- `okf check` is untouched. So is `vegnormal-okf`, which was read and not
- `okf check` is untouched. So is the consumer, which was read and not
written.
### § 7.6 — honesty limits of this section
- **N = 1 product.** Every number in § 7.3 is R761's. Nothing here says what
- **N = 1 product.** Every number in § 7.3 is the reference standard's. Nothing here says what
`boundary_share` does on a second declared structure, because no second one
exists in reach.
- **The bar fails builds nobody calls bad.** Named above rather than tuned
@ -306,9 +306,9 @@ of four rows gives `UNMEASURED` rather than a share.
counts so a reader can see which it is (a real build of the product scores in
the thousands on one of the two forms).
- **One file was read outside this repository beyond the fasit.** The order said
to read only the fasit from `vegnormal-okf`; deriving the pair-key form
to read only the fasit from the consumer; deriving the pair-key form
required reading their measuring script
(`experiments/sk2_grenser.py`, read-only). The alternative was to guess the
(one experiment script, read-only). The alternative was to guess the
match rule, which is what P1 exists to forbid. Nothing there was written.
- **`boundary_share` is still not a hit@k.** It asks whether a declared section
became a concept, never whether a question reaches it.

56
docs/2026-09-12-k3-runde23-stien-i-kroppssignalet.md
View file

@ -4,7 +4,7 @@
`8e82da4` (the instrument) · **Measured from:** a frozen `git archive` export of
`8e82da4` installed into a scratch virtualenv (`__file__` under
`/private/tmp/...`, never `/Users/ktg/repos`, never the `okf` on PATH), Python
3.14, guard 1.4.0. One document: R761 Prosesskoden:2025, built twice in scratch
3.14, guard 1.4.0. One document: a 701-page reference standard, built twice in scratch
from the publisher's own NISO-STS source, once with `--shell-parent` and once
without. The consumer repository was read only: `git status --porcelain` empty
before and after, `build/ferdig/` listing identical including mtimes.
@ -90,30 +90,16 @@ condition for reading anything else it produces.
| KP | 7 / 7 | 35 050 / 35 050 | 0 | 0 | 0 | -- | 0 | 0 | 0 |
| KN | 7 / 7 | 10 514 / 10 151 | 5 of 7 | 2 | 2 | 2 of 2 | **2** | 0 | 0 |
**S1, both lists in full** (the six identical questions are identical in order
as well as in membership):
**S1, both lists** (the six identical questions are identical in order as well
as in membership): positions 1 to 4 hold the same four sections in X and Y --
the gold and three main-chapter sections. Positions 5, 6 and 7 of X are three
leaf sections that entered through the link line; Y holds an overview section
and two more main-chapter sections there.
| # | X | Y |
|---|---|---|
| 1 | `2-1/hovedprosesser` | `2-1/hovedprosesser` |
| 2 | `hovedprosess-81-l-smasser` | `hovedprosess-81-l-smasser` |
| 3 | `hovedprosess-83-konstruksjoner-i-grunnen-...` | same |
| 4 | `hovedprosess-84-betong` | `hovedprosess-84-betong` |
| 5 | **`32-113/delt-tverrsnitt-normal-salvelengde`** | `5/hierarkisk-oppbygging-av-prosesser` |
| 6 | **`32-114/delt-tverrsnitt-halv-salvelengde`** | `hovedprosess-82-berg` |
| 7 | **`36-111/hovedfordelinger`** | `hovedprosess-85-st-l` |
**KN, both lists in full:**
| # | X | Y |
|---|---|---|
| 1 | `25-41/jordmasser-til-st-yvoll-...` | same |
| 2 | `1/bruksomr-der-for-prosesskoden` | same |
| 3 | **`25-4/jordmasser-til-st-yvoll-ledevoll-steinfyllingsskr-ninger-mm`** | `26-4/sprengt-stein-...` |
| 4 | `26-4/sprengt-stein-...` | `32-225/steinmasser-fra-tunnelmunning-...` |
| 5 | **`31-51/injeksjons-og-kontrollhull-ved-sporadisk-injeksjon`** | `5/hierarkisk-oppbygging-av-prosesser` |
| 6 | `32-225/steinmasser-...` | `67-5/ledelinjer-i-gategrunn` |
| 7 | `5/hierarkisk-oppbygging-av-prosesser` | `88-1714/sporslitasje` |
**KN, both lists:** positions 1 and 2 are the same in X and Y. X's positions 3
and 5 are two leaf sections that entered through the link line; the rest of
both lists is the same small set of sections in a shifted order, and Y's
positions 6 and 7 are two sections X does not deliver.
### `k` 50
@ -131,9 +117,7 @@ as well as in membership):
**Where the newcomers enter, and what they push out.** On KP at `k` 50, 22 of
the 23 newcomers are linked shells entering at positions 21, 22, 23, 24, 25,
26, 27, 28, 31, 32, 33, 34, 35, 38, 39, 40, 41, 42, 45, 46, 47, 48, and the 19
that leave held Y's positions 26 to 44 -- among them `84-2/forskaling`,
`84-3/armering`, `87-1/fuktisolering-membran-...` and
`88-2/vedlikehold-beskyttelse-og-reparasjon-av-betong`. On S1 at `k` 50 six
that leave held Y's positions 26 to 44 -- among them four real sections from two of the document's main chapters. On S1 at `k` 50 six
shells enter at positions 4, 5, 6, 8, 9, 10 -- near the top -- and six real
sections leave from Y's positions 35 to 40. The four newcomers carrying no
link of their own (1 on S1, 1 on KP, 2 on KN) gained nothing: they moved
@ -146,11 +130,11 @@ because the concepts around them did.
| newcomers that gained a question token from the link | **39** | 39 link-bearing newcomers |
| of those, the gain came from the **PATH** | **37 path only + 2 path and title** | 39 |
| of those, the gain came from the **TITLE** alone | **0** | 39 |
| distinct QUESTION tokens the path ever matched | `prosesskoden` (31), `r761` (22), `prosess` (8) | 61 token hits |
| distinct QUESTION tokens the path ever matched | the document's name (31), its number (22), a stem of its name (8) | 61 token hits |
**Every token the link line ever added is a segment of the document's own
directory** -- `r761-prosesskoden` -- and `prosess` reaches it by the stem
prefix rule. This is exactly the saturation `shared_id_prefix` (round 20) took
directory** -- its number and its name -- and a stem of the name reaches it by
the stem prefix rule. This is exactly the saturation `shared_id_prefix` (round 20) took
OUT of the id signal, arriving back through the body. The link's TITLE, which
is the part carrying meaning, contributed a hit on its own **0 times**.
@ -206,11 +190,11 @@ writes that form only under `--shell-parent`:
| bundle | payload byte-identical under (c) | files carrying the door's line |
|---|---|---|
| N100 | yes | 0 |
| N200 | yes | 0 |
| N500 | yes | 0 |
| R761 as shipped | yes | 0 |
| R761 unflagged, built here | yes | 0 |
| corpus-a | yes | 0 |
| corpus-b | yes | 0 |
| corpus-c | yes | 0 |
| the reference standard as shipped | yes | 0 |
| the reference standard unflagged, built here | yes | 0 |
**5 of 5, 0 of 5.** Changing `consume`'s default reading of the body is a rank
change on a published payload form, and it is stated here as one: it requires
@ -277,7 +261,7 @@ payloads are conformant too, and they answer nothing.
rank only. Whether a reader ANSWERS better is a judged reading; round 21's
own consumption rows were one non-deterministic draw per question.
- **The instrument is someone else's and scores a title or a section-number
pair, not an answer.** `hitk_sk2.py` at the consumer's HEAD `ee4d7e1`, copied
pair, not an answer.** The consumer's hit@k script at its HEAD `ee4d7e1`, copied
to scratch with the hard-coded payload path changed, because a concurrent
session writes the same `/tmp` file.
- **Six of eight questions never move at all**, which means the whole

222
docs/2026-09-12-k3-runde24-blokkform-til-de-flate-leserne.md
View file

@ -1,222 +0,0 @@
# K3 round 24 — the block `sources` form reaches every flat reader
Order K3-24, 2026-09-12. Red `eb327bf`, fix `28f9a4b`, base `150c726`
(v0.8.4). PyYAML 6.0.3, guard 1.4.0, Python 3.14.0. Every measurement below
ran through the repository's own interpreter against a frozen `git archive`
export, `__file__` verified under the scratch path and never under
`/Users/ktg/repos`; the four producer bundles and `portfolio-optimiser` were
READ and never written.
## Deviations, first
1. **The order is not this repository's STATE `NESTE`.** STATE's next step is
the K3-21 link-line ranking question; this order is a new finding from a
re-measurement of the producer's D4 work. The link-line question stands as
next after.
2. **Acceptance point 1 says "through EVERY flat reader", and there are
three, not one.** `materialize.parse_frontmatter` (public API),
`structure._split_frontmatter` and `profiles._split_frontmatter` are three
copies of one line-oriented grammar, documented as such at each site. The
order's prose names `parse_frontmatter`; fixing only that one would have
left two readers with the defect and made the acceptance sentence false.
All three are fixed and all three are measured separately below.
3. **The `title` denominator is 7 372, not 5 372.** Counting every file with a
frontmatter block across the five bundles gives 2 757 + 447 + 1 134 + 271 +
2 763 = 7 372. The order's number is not reproduced here and the larger one
is the one measured.
4. **`^sources:` in `tests/` is 4 files, not 3.** PM measured 3 before this
round; this round adds `tests/test_block_sources_flat_readers.py`. Across
`tests/`, `examples/`, `skills/` and `docs/` the count is 9, of which **4
are golden concept files**.
5. **The order's § 2 sub-question about an `okf check` rule is answered NO,
with a measurement rather than a deferral** (§ 6).
## 1. The defect, reproduced
`consume.read_sources` reads both YAML forms. The three copies of the flat
grammar read only the flow one: for a block sequence they return the key with
an **empty value**. The key is present and the value is gone, so no consumer
can catch it as an error.
Denominator = files carrying a frontmatter block. `read_sources` was called
with `_frontmatter_lines(path)`, never a `Path`; the known-positive control
(`test_control_read_sources_reads_the_block_form`) runs before any zero is
believed.
| bundle | files | `read_sources` entries | flat `sources == ""` | key absent | `safe_load` OK |
|---|---|---|---|---|---|
| `r761-2025` | 2 757 | 2 756 | **2 756** | 1 | 2 757 |
| `n100-2023` | 447 | 446 | **446** | 1 | 447 |
| `n200-2024` | 1 134 | 1 133 | **1 133** | 1 | 1 134 |
| `n500-2024` | 271 | 270 | **270** | 1 | 271 |
| `r761-2025-generisk` (flow) | 2 763 | 2 761 | 0 (2 761 non-empty) | 2 | 2 762 |
Every cell reproduces the figure this order was dispatched with.
### Three readers, one file
| reader | `n500-2024` concept, before |
|---|---|
| `yaml.safe_load` (PyYAML 6.0.3) | `[{'resource': 'https://…/859990?languageCode=nb', 'title': 'N500:2024'}]` |
| `llm_ingestion_guard.okf.parse_frontmatter` (1.4.0) | the same mapping |
| `llm_ingestion_okf.materialize.parse_frontmatter` | `''` |
| `llm_ingestion_okf.structure._split_frontmatter` | `''` |
| `llm_ingestion_okf.profiles._split_frontmatter` | `''` |
## 2. The fix, and the number that chose it
**Path (a)**, narrowest: one structure-aware branch for the keys
`profiles.STRUCTURED_BLOCK_KEYS` names — today exactly `{"sources"}`, the key
`read_sources` already knows how to read. "Skip indented lines" stays the rule
for everything else, and the K3-20 refusal it exists for is untouched: a
decoded block lands INSIDE its own value and never in the document's
namespace.
The order asked which TYPE the value takes, because the return type is
`dict[str, str]` and a list of mappings is not a `str`. Both answers measured
on the same tree:
| option | cost |
|---|---|
| re-serialise to the flow form the flat readers already round-trip | **0** call sites moved, 0 `mypy --strict` errors, public signature unchanged |
| widen the return type to `str \| list[dict[str, str]]` | **15** `mypy --strict` errors in **4** of the 5 modules that touch the reader, plus a public-API signature every outside caller follows |
Re-serialisation wins on that number. What it is NOT is stated in the code:
the rendering is a **reading projection**, not a claim that the value is
writable. `yaml_flow_plain` still refuses a `?` and the guard still refuses a
quote inside a flow mapping, so a value rendered here may have no writable
flow form at all — which is the whole reason the producer writes block.
One grammar, four call sites: `read_block_mappings` moved out of
`read_sources`, where it was written and measured, into `profiles` (the module
both the flat readers and `consume` import). Two copies of a block grammar
would be two answers to one question.
## 3. Acceptance
1. **0 files with an empty `sources`** in all four bundles, through all three
flat readers: `0 / 0 / 0` per bundle against 2 756 / 446 / 1 133 / 270.
2. **Dict equality against both reference readers, every file, both
denominators:** PyYAML 2 757/2 757, 447/447, 1 134/1 134, 271/271,
2 762/2 762; guard 1.4.0 identical on the same five denominators. Not a
sample.
3. **`tests/fixtures`: 12 of 12 read.** Eleven byte-identical dicts before and
after; the twelfth differs on exactly one key — the block `sources` that is
the point of the change, `''` → the decoded address. Nothing else moved.
4. **The flow form is unchanged:** `r761-2025-generisk` gives 2 761 non-empty
`sources` before and after.
5. **No document's own `title` moved** on any of the **7 372** files with a
frontmatter block across the five bundles (`diff` on the full title map per
bundle: 5 of 5 identical).
6. **Suite 1807 passed / 1 skipped, rc 0, 94 s** in the working tree. The
baseline on `150c726` is 1782/1 and 1782 + 25 new = 1807; no other test is
red or newly skipped. `ruff` clean, `ruff format --check` 119 files,
`mypy --strict` clean over 21 files.
7. **`okf check`: 17 rules, 0 findings, rc 0** (`len(RULES)` printed as a
literal from the frozen export) over a payload built from a scratch bundle.
8. **`grep "import yaml"` in `src/`: 0 files non-zero.** `dependencies` is
still exactly `["llm-ingestion-guard>=1.2,<2.0"]`; `uv.lock` untouched.
9. **No ranking measurement is owed and it is shown, not asserted:** a
five-document folder built with the pre-fix and post-fix code is `diff -r`
identical, 0 differences over 52 files.
### One pre-existing divergence, measured so it is not attributed here
The three flat readers do not agree with each other on every key, before OR
after: `structure._unquote` strips a `'`-wrapped value where the other two
keep it. On `r761-2025` the three agree on 3 of 2 757 files both before and
after, with the identical key histogram (`prosessnr` 2 727, `hovedprosess`
2 727, `forelder` 2 728, `seksjon` 2 735, `description` 18, `seksjonstittel`
1, `title` 1); on `n500-2024`, 172 of 271 both times. `sources` does not
appear in that histogram after the fix. This round neither caused nor closed
it.
## 4. `_render_sources`' three reasons, one at a time
| reason | verdict | measurement |
|---|---|---|
| 1. a block list round-trips to an empty value, silently | **FELL** | § 1 and § 3: 4 609 of 4 609 files now decode through every flat reader |
| 2. the consumer accepts flow plural and classifies block as unreadable provenance | **STANDS** | `portfolio-optimiser` read at `6eb58e5`: `read_provenance` returns `UnreadableProvenance(reason="block-sequence")` for a block value |
| 3. B6's acceptance test asks for a round trip through this parser | **FELL** | the rewritten `test_the_block_form_round_trips_through_the_flat_reader` passes with both entries |
Reason 2 is **not the guard's objection**. Guard 1.4.0 reads the block form on
4 609 of 4 609 of the same files. The consumer that refuses it is named, and
after this round it is the only thing holding the emission rule.
## 5. Should the emitter write block? **No**, and the number is reason 2
The order's premise is measured and true: `yaml_flow_plain` is `False` for the
producer's address, so this library's own inbox door raises
`inbox_source_file_unaddressable` on exactly the URL the producer ships in
four bundles. No flow form passes both PyYAML (which refuses an unquoted `?`)
and the guard (which refuses a quote inside a flow mapping). That is why the
producer moved to block.
It is still not enough to move the emitter, because reason 2 stands: writing
block would hand `portfolio-optimiser` the state its own decoder reports as
unreadable — replacing a defect in OUR reader with a defect in THEIR reader,
for the same bytes. The byte cost is recorded for whoever revisits it: **4
golden concept files** carry a `^sources:` line (`ingest-golden-okf-v0-2` 1,
`ingest-golden-segmented-okf-v0-2` 3), plus every bundle rebuilt after such a
change, plus the hit@k gate that a byte move in a bundle triggers.
What would make the question decidable is one measurement in another
repository, and it is written into STATE: whether `portfolio-optimiser`'s
`read_provenance` will accept a block sequence. Until that is answered there,
the answer here is no — and the cost of the no is zero, because the producer's
bundles are already readable by all three of okf's readers, by PyYAML and by
the guard.
## 6. Does `okf check` owe a rule? **No**, measured
`contract_check.py` contains **0** occurrences of `sources` and **0** of
`frontmatter`: no rule reads the key today, so nothing regressed. The
candidate was `read_sources`' third state — "the key is there and this reader
cannot decode it". It is already named where it is observable:
`consume.build_payload` writes `sources_unreadable: true` into the excerpt for
exactly that state. A checker rule would restate an existing mechanism, and
`okf check` compares a DECLARED identity against a DECLARED identity without
opening the bundle, so it could not verify the claim independently anyway. No
rule was added and `len(RULES)` is **17**.
The nine `parse_frontmatter` call sites in `src/` were re-counted
(`importer.py:557`, `:581`; `inbox.py:663`, `:687`, `:1473`, `:1641`;
`consume.py:353`, `:1952`; `materialize.py:166`) and none of them reads
`sources` out of the flat dict — so the consumer this repairs is the EXTERNAL
one, reaching `okf.parse_frontmatter` as public API. The reprojection path was
confirmed rather than assumed: `structure.structure_frontmatter` writes
`declared[key]` back for every facet key, and `sources` is a facet key on
**none of the seven profiles** (four carry a `FacetPolicy`, three carry
`None`). Had anyone added it, the pre-fix reader would have written the empty
value back into the concept file — destroying the address on disk and not only
in memory. That is the distance between "harmless today" and "safe", and it is
now closed by the reader rather than by the facet list.
## 7. Honesty limits
- **`STRUCTURED_BLOCK_KEYS` is one key wide.** A fixture in this tree carries
a block `verified:` sequence and it still reads as an empty value. That is
the same defect family for a different key, pinned by
`test_a_block_key_outside_the_named_set_is_still_empty` so the next widening
is a decision rather than a side effect. It is a LIMIT, not a closed case.
- **The rendering is not byte-identical to the source.** A consumer comparing
`parse_frontmatter`'s value against the file's own bytes will see a
difference; a consumer comparing ENTRIES will not. The structured reader
(D1b) is still the answer for anyone who needs the original bytes.
- **A rendered value may have no writable form.** Writing it back into a
frontmatter can produce a file PyYAML or the guard refuses. This is stated
in the function's own docstring; nothing in the code prevents it, because
preventing it would mean refusing the addresses the fix exists to carry.
- **Quoted leaves diverge by design and the divergence is named:** the guard
keeps a leaf's quotes verbatim, PyYAML decodes them, and this library
follows `read_sources`' K3-22 rule (a `"`-wrapped leaf decoded, a
`'`-wrapped one left standing). The producer's four bundles carry **0**
quoted leaves, so the divergence is unreached there — measured, not assumed.
- **The three flat readers still disagree on `'`-quoted scalars** (§ 3). Not
caused here, not closed here.
- **One bundle was built to prove no bytes move**, five documents. A larger
corpus was not rebuilt, and the claim is exactly as wide as that
measurement.
Conformance is the floor, never the proof: every acceptance number above was
run against artefacts, not against the shape of the code.

183
docs/2026-09-12-k3-runde25-standardlesingen-uten-lenkelinja.md
View file

@ -1,183 +0,0 @@
# K3 round 25: the default reading drops the door's link line
**Date:** 2026-09-12 · **Base:** `7faa380` · **Commits:** `3816ed5` (red),
`38320cd` (the default) · **Measured from:** a frozen `git archive` export of
`38320cd` synced into a scratch virtualenv (`__file__` under
`/private/tmp/okf-k3-25-scratch/`, never `/Users/ktg/repos`, never the `okf` on
PATH), Python 3.14, guard 1.4.0. One document: R761 Prosesskoden:2025, built
twice in scratch from the publisher's own NISO-STS source, once with
`--shell-parent` and once without. The consumer repository was read only:
`git status --porcelain` empty before and after, `build/ferdig/` listing
identical including mtimes, measured twice.
Round 23 decomposed the cost of `--shell-parent`'s link line and recommended
one of three exits: make the reading WITHOUT the line `consume`'s default,
because the cost is the bundle-absolute PATH inside the link and not the link.
This round is that recommendation as code, and nothing else.
## 0. Where this measurement differs from what it was given, first
**The order's acceptance row `S1 spent 28 020 B at the default k` cites the
column being retired.** 28 020 is round 23's **X** -- the flagged bundle with
the line SCORED. The new default is round 23's **Y**, and Y's published value
for that cell is **31 031**. Measured here on the same bundle in one process:
| reading | S1 delivered | S1 `spent` |
|---|---|---|
| `link_in_signal=True` (X, retired) | 7 | **28 020** |
| default since this round (Y) | 7 | **31 031** |
| unflagged build, default (Z) | 7 | **31 031** |
So 28 020 is not reachable at the new default by construction, and it is not a
regression: **16 of 16** cells of round 23's Y column reproduce here to the
byte, `spent` and delivered count alike, at both `k`. The stop-rule the order
set protects against two things -- round 23's numbers failing to reproduce, and
the default switch doing something the parameter did not -- and both are
measured absent. The row was transcribed from the wrong column.
**Everything else reproduces.** 2 761 concepts in both builds, 675 files
carrying the door's line in the flagged one and 0 in the unflagged one.
## 1. What moved
`link_in_signal` was a naked literal `True` on three signatures. It is now
`DEFAULT_LINK_IN_SIGNAL = False`, read by all three:
| entry point | before | after |
|---|---|---|
| `consume.searchable_text` | `link_in_signal: bool = True` | `= DEFAULT_LINK_IN_SIGNAL` |
| `consume.concept_scores` | `link_in_signal: bool = True` | `= DEFAULT_LINK_IN_SIGNAL` |
| `consume.build_payload` | `link_in_signal: bool = True` | `= DEFAULT_LINK_IN_SIGNAL` |
**The naming choice, with the number.** `consume.py` holds 8 module constants
for defaults (`DEFAULT_PROFILE`, `DEFAULT_LIMIT`, `DEFAULT_TIE_SHARED_RANK`,
`DEFAULT_STEM_PREFIX`, `DEFAULT_TITLE_COVERED`, `DEFAULT_SOURCE_QUOTA`,
`DEFAULT_FOLLOW_PARENT`, `DEFAULT_K`) and this was the one default written out
three times: **8 of 9** followed the house convention and this did not. A
constant costs one declaration and buys two things -- the convention, and a
test that can fall on the rule from the module that owns it.
**The CLI-flag decision, with the number.** `grep -n '"--.*link'` over
`src/llm_ingestion_okf/*.py` gives **0 hits** before and after, and
`consume.main` never passed the parameter, so the CLI inherits the function
default and not one flag line changed. No flag was ADDED: round 23 withheld one
because the choice was the measurement's, and that reason is spent -- but a
spent reason is not a reason to add one. **0 of 3** call sites outside
`consume.py` could be named as needing the older reading. The older reading
stays reachable in Python (`link_in_signal=True`); after this round there is no
CLI route to it, and that is stated rather than hidden.
## 2. The red test
Ten cases, each red on `7faa380` for its own reason, all measuring BEHAVIOUR --
the entry points are called with no parameter at all and what came back is
read. No `inspect.signature`. Reds: three on the entry points reading a
stripped body; one `ImportError` inside the test body (never at collection) for
the named constant, spent immediately on a payload comparison; Y = Z in
miniature (the flagged fixture delivered `[menneske, midt, skall]` where the
unflagged one delivered `[menneske, midt]`); the old reading still reachable AND
producing a different payload; the two known-negatives restated against the
default path; and rank-not-form (the two readings differ in the delivered set
while a concept delivered by both keeps identical excerpt bytes).
Two are green on both sides on purpose and are named as such: the
known-positive -- a bundle where no body ends in the door's form is
byte-identical under both readings, section 3's gate in miniature -- and the
characterisation of what the line does, which is an invariant about the line.
**One existing test changed with the code**, and it is the whole test-caller
denominator: `tests/` holds **129** call occurrences of the four functions
on **128** lines across **10** files, of which **20** lines pass the parameter
explicitly (re-counted 2026-09-12 for the `v0.8.5` release; the figures first
published here, 112 and 13, were a line count read as an occurrence count and
a file count read as a call count). Exactly
**1 of 129** changed result -- the characterisation that called
`concept_scores` with no parameter because the reading it characterises used to
be the default. It now names that reading; same fixture, same number. The two
other test files whose fixtures carry the door's form
(`test_parent_reaches_reader.py`, `test_shell_parent.py`) did **not** move: the
first asks a question the shells answer through their own titles, the second
never calls `consume`.
## 3. Gate A -- the exposure, on bytes
Read-only over the consumer's `build/ferdig/`. Denominator = concept files.
| bundle | concept files | files carrying the door's line | payload byte-identical across the move |
|---|---|---|---|
| N100 | 446 | **0** | yes (70 536 B) |
| N200 | 1 133 | **0** | yes (159 210 B) |
| N500 | 270 | **0** | yes (50 499 B) |
| R761 as shipped | 2 756 | **0** | yes (387 552 B) |
| R761 generic | 2 761 | **0** | yes (463 068 B) |
**0 of 5 and 5 of 5.** The same question string for every bundle, serialised
before the change and after it, compared with `cmp`.
## 4. Gate B -- the ranking, on the one document that has shells
Both builds read at the shipped defaults, no parameter passed anywhere.
| row | result | denominator |
|---|---|---|
| flagged read by default == unflagged read by default: list, ORDER and `spent` | **16 of 16** rows | 16 (8 questions x 2 `k`) |
| hit@1 / hit@8 / hit@50, both `k`, both bundles | **6/6 · 6/6 · 6/6** | 6 questions |
| known-positive rank, `k` 8 and `k` 50 | **1 / 1** | -- |
| S1 `spent` at the default `k` | **31 031** (round 23's Y; 28 020 was X) | -- |
| `okf check` | **32 of 32** payloads rc 0, **17 rules**, 0 findings | 32 |
Round 23's claim was that under this reading a `--shell-parent` bundle delivers
exactly what the unflagged build delivers. It does, to the byte, on every row.
## 5. The consumers, measured
`grep` over `build_payload(`, `concept_scores(`, `searchable_text(` and
`body_without_link_line(`, excluding `.git`, `tests/` and `.venv`: **19** hits
on **17** lines across **5** files (re-counted 2026-09-12 for the `v0.8.5`
release; the 129 first published here is the `tests/` figure above, written
into the wrong paragraph). **8 of the 19** fall outside `consume.py`: one is
prose in `CLAUDE.md` and four are this report counting itself. **3 call sites in 2 files**, plus the CLI's own `main` inside the
module:
| call site | what it is | passes the parameter | what the change means for it |
|---|---|---|---|
| `src/llm_ingestion_okf/skill.py:360` | the generated skill's answering path | no | a bundle carrying the line ranks differently; excerpt bytes unchanged, so the skill's own prose about the line still holds |
| `tools/okf_consume_measure.py:155` | the harness, per question | no | scores the new reading; figures measured before today are figures of the old one |
| `tools/okf_consume_measure.py:247` | the harness, over a fasit file | no | same |
| `src/llm_ingestion_okf/consume.py:2404` | the CLI's `main` | no | `okf consume` inherits the default; no flag moved |
All four change behaviour on any bundle carrying the door's line and on no
other bundle. **0 of 5** bundles anyone ships today carries it.
## 6. Acceptance
Suite **1 826 passed / 1 skipped, rc 0**, against a baseline of **1 816 / 1**
measured on `7faa380` first; the skipped one is `OKF_HTML_CORPUS` in both. Run
from the repository root, after `git add` and after commit. `mypy --strict`
clean over 21 source files, `ruff check` and `ruff format --check` clean over
132 files. The four goldens and the K2 pin: **11 passed, 0 skipped** -- run
where the pin's `GOLD_SET` exists, so it was measured and not skipped. No
golden moved.
## Honesty limits
- **N = 1 document.** Gate B is one 2 761-concept standard from one publisher,
and 6 of its 8 questions never move under any reading, so the scored evidence
is two rows. What generalises is the MECHANISM -- a bundle-absolute path
repeating the document's own directory in every linked body -- not the size.
- **The consumption half is not measured.** This round measures delivery and
rank. Whether a reader ANSWERS better is a judged reading, and round 21's own
consumption rows were one non-deterministic draw per question.
- **`5 of 5` is a statement about the bundles that exist TODAY.** The day a
producer ships a bundle built with `--shell-parent`, this stops being free
and becomes a rank change that reaches someone. That is the trade this round
took deliberately: it is cheap now and it will not be later.
- **The instrument is the consumer's**, `hitk_sk2.py` at their HEAD `ee4d7e1`,
copied to scratch with the hard-coded payload path changed because a
concurrent session writes the same `/tmp` file. It scores a title or a
section-number pair, not an answer.
- **`--shell-parent` is still OFF** and this round did not touch it. Round 23
measured that its acceptance would read 0 / 0 / 0 under this default on this
document; a build default is a promise to every consumer and not to one.
- **Conformance is the floor and never the proof.** The known-negative
question's payloads are conformant too, and they answer nothing.

13
docs/2026-09-12-r7-release-v085.md
View file

@ -180,8 +180,8 @@ left alone. Everything below ran from `<export>/.venv/bin/`.
| `okf check` on the shipped skill | 1 | rc 0, `conformant: 17 rules over 3 excerpts and 0 withheld entries, 0 findings` |
| `okf project` on the five-document folder | 5 documents | **26** concepts, **52** md files, **0 of 5** left out -- R6's numbers to the unit |
| HTML corpus | 828 documents | `proposed 828 plan(s); 0 document(s) with no boundary; 0 unreadable`, **6 015** md |
| R761 XML | 110 source files, 1 at top level | **2 761** concepts over 2 737 directories, of which `11` and `12` hold two each; document directory is `r761-prosesskoden/`, not the UUID |
| hit@1 / @8 / @50 on R761 | **N = 6** scored of 8 asked | **6/6 · 6/6 · 6/6** |
| reference standard, XML | 110 source files, 1 at top level | **2 761** concepts over 2 737 directories, of which `11` and `12` hold two each; document directory is its declared `<doc-number>`, not the UUID |
| hit@1 / @8 / @50 on the reference standard | **N = 6** scored of 8 asked | **6/6 · 6/6 · 6/6** |
| known-positive rank | 1 | rank **1** at the default `k` and rank **1** at `k` 50 |
| known-negative (K3-17) | 1 question | rank 1 = the RITB section (**6 793 B**), rank 2 = «Prøvedrift» (**10 590 B**); both concepts present and delivered |
| block `sources:` read to 0 empty | 4 605 files | see below |
@ -195,11 +195,10 @@ is rank 1, the known-positive is rank 1 at both `k`, and S1's `spent` is
**31 031** -- the value STATE records for the default reading.
**The four bundles, read from the tag's bytes.** Source directory, named
because it is not named in K3-24's report:
`/Users/ktg/repos/vegnormal-okf/build/ferdig/`. Chosen after verifying its
because it is not named in K3-24's report: a downstream consumer
repository's finished-build directory. Chosen after verifying its
denominators against K3-24's table first: files carrying a frontmatter block
are **2 757 / 447 / 1 134 / 271** for `r761-2025`, `n100-2023`, `n200-2024`,
`n500-2024`, exactly K3-24's. Of those, the ones carrying a BLOCK `sources:`
are **2 757 / 447 / 1 134 / 271** for the four bundles, exactly K3-24's. Of those, the ones carrying a BLOCK `sources:`
are **2 756 / 446 / 1 133 / 270** = **4 605**. All three flat readers
(`materialize.parse_frontmatter`, `structure._split_frontmatter`,
`profiles._split_frontmatter`) return a non-empty value on **0 empty of each**,
@ -217,7 +216,7 @@ is a measurement.
4 605**. The reason is visible in one value:
```
[{ resource: https://…/api/nisosts/859990?languageCode=nb, title: N500:2024 }]
[{ resource: https://example.test/api/nisosts/000000?languageCode=nb, title: Q500:2024 }]
```
The `?` opening the query string ends the flow scalar, and PyYAML raises

270
docs/2026-09-17-bilder-i-bundlen-trinn1.md
View file

@ -1,270 +0,0 @@
# Bilder i OKF-bundles, trinn 1: de bæres (0.10.0)
Ordre `20260916T050910Z-1628427832-from-.claude`, trinn 1 av 2. Trinn 2
(`okf describe`, Claude vision) er ikke i denne leveransen og ikke i denne
rapporten.
Utgangspunktet er operatørens premiss, ordrett: «det som ender opp i en bundle
etter en prosess med å konvertere X antall kilder MÅ være 100 % riktig».
---
## § 0 Premissene målt først
Ordren oppgir hva PM målte i dette repoet og ber om at det gjentas
(Verifiseringsloven, ansikt 3). Målt på `332961a`, 2026-09-16:
| Påstand | Målt her | Status |
| --- | --- | --- |
| Ingen leser henter, navngir eller kopierer et bilde | `page.images` og `extract_table`: **0 treff** i `src/`. `handle_starttag` leser aldri `attrs` (`extract.py`). `<graphic>` forekommer ikke i XML-leseren. `page.to_image` finnes kun inne i OCR-grenen | **Bekreftet** |
| Eneste skriver er `write_bytes(..., content: str)` | Ja, UTF-8, ingen binær skrivesti | **Bekreftet** |
| 108 grep-treff over 23 filer | Målt her: **127 treff over 12 filer** med `grep -rIEn` over `src/*.py` | **Avviker** — PMs kommando er ikke oppgitt, så tallene er ikke sammenliknbare. Substansen (ingen treff er en bildeleser) er bekreftet ved gjennomlesing av alle 127 |
| SPEC er taus om binære filer | `_okf-canonical` `ad30107`: § 3 «a directory tree of markdown files», § 11 punkt 1 scoper til `.md`, § 6.3 er en konvensjon | **Bekreftet** |
To premisser i ordren er **ikke** reprodusert og er merket som det: «84 filer i
kildezip-ens `graphics/`» — katalogen jeg har lesetilgang til
(`~/repos/vegnormal-okf/build/860019-html/graphics`) holder **109 filer**, og
XML-en refererer **50** av dem. Det er en annen artefakt enn zip-en ordren
siterer, ikke en motsigelse.
R761-målingen som utløste ordren er ikke etterprøvd her i sin helhet; det jeg
målte selv er at side 496 i PDF-en bærer **2 DCTDecode-bilder** rett under
teksten «Tabell 84-2:», og at hele dokumentet bærer **50 bildeobjekter på 38 av
701 sider**, fordelt **29 DCTDecode / 21 FlateDecode** — samme antall som
NISO-STS-leveransens 50 `<graphic>`.
---
## § 1 Gaten, skrevet rød først
`tests/test_asset_gate.py`, skrevet før én linje kapabilitetskode. Nevneren
leses ut av **kilden** (`page.images`, `word/media/`, `ppt/media/`, `<img`,
`<graphic`), aldri fra en konstant i dette repoet — en konstant er repoet som
påstår sin egen forventning, og den blir gal i det en fixture regenereres.
Målt på `332961a`, bygget fra `git archive` og ikke fra arbeidstreet (et
editable install leser `src/` live, så en «før»-kjøring i dette treet ville målt
endringen den skulle gå forut for):
```
carried 0 of 2 local (2 declared) prosess-84-tabell.pdf
carried 0 of 1 local (1 declared) prosess-84-notat.docx
carried 0 of 1 local (1 declared) prosess-84-presentasjon.pptx
carried 0 of 2 local (3 declared) prosess-84-web.html
carried 0 of 2 local (2 declared) prosess-84-sts.xml
---------------------------------------------------------------
carried 0 of 8 local images across 5 documents (9 declared),
and the bundle held no assets/ directory at all.
```
Etter trinn 1: **8 av 8**, og det niende (en `https://`-kilde) er en peker uten
fil, talt som funnet-og-ikke-båret.
**En fixture-defekt gaten fant selv:** de fem dokumentene het først
`prosess-84.{pdf,docx,pptx,html,xml}`. Dørens egen § 3-kollisjonsregel refuserte
to av dem (`inbox_slug_collision: 2/7`), så to lesere ble aldri kjørt og gaten
rapporterte en bæredefekt som i virkeligheten var en fixturedefekt. Fem
forskjellige stammer nå.
---
## § 2 Hva som ble bygget
**Fem lesere PLASSERER, én modul BESTEMMER.** `llm_ingestion_okf.assets` eier
hva et bilde er, hva det heter og hvordan det pekes på; leserne vet bare hvor i
sitt eget dokument bildet står og hva kilden kaller det.
| Rad | Hvor bildet hentes | Etikett |
| --- | --- | --- |
| `.pdf` | bilde-XObjects på siden (`page.images`) | ingen — PDF har intet captionsfelt |
| `.docx` `.pptx` `.odt` `.rtf` | konverterens `--extract-media` | `descr`/alt fra containeren |
| `.html` `.htm` | `<img src alt>`, lokal sti eller `data:`-URI | `alt` |
| `.xml` | `<graphic xlink:href>`, href-en og så `graphics/<navn>` | ingen — STS har intet captionsfelt her |
`.xlsx` er **bevisst ikke** en rad: konverteren skriver én pipe-tabell per ark,
og en toradersblokk inne i en slik tabell ville brutt rad-lokatoren
`source_rows` leses tilbake ut av. Målt 2026-09-16: **0 av 4** K2-arbeidsbøker
bærer media i det hele tatt, så raden er en uttalt grense og ikke et tap.
**Etiketten gjettes ikke.** To av de fire formatene har intet captionselement —
verken et PDF-bildeobjekt eller en STS-`<graphic>` bærer ett, og «Figur 11.1
…»-linja et menneske leser er en søsken-`<p>` leseren allerede emitterer på egen
linje. Å utlede en etikett fra nærmeste linje ville vært en umerket heuristikk.
**Layouten.** `assets/` i bundle-rota,
`<sha256[:12]>-<kildens eget BASENAVN><snuset suffiks>`. I konseptet, der bildet
sto:
```markdown
![Tabell 84-2 Toleranseklasser](/assets/e54e5f5da0e8-tabell-84-2.png)
Image: graphics/tabell-84-2.png (120x90 px) -- Tabell 84-2 Toleranseklasser
```
Basenavnet og ikke stien: målt på fixture-innboksen ble ett bilde skrevet
**to ganger under to navn i én kjøring**, fordi HTML-dokumentet peker på
`graphics/figur-84-1.png` og STS-dokumentet på `figur-84-1.png` — med digesten i
begge navnene som annonserte at bytene var like. Stien er en egenskap ved
pekeren, ikke ved bildet, og hele originalen overlever på pekerens egen linje.
**Typen snuses, aldri påstås.** En `.jpg` som i virkeligheten er en PNG bæres som
PNG under et `.png`-navn; alternativet er en bundle hvis filnavn er uenige med
sitt eget innhold.
---
## § 3 PDF: to ruter, og hvorfor rasterisering ble felt
`get_data()` kjører hver filter pdfminer kjenner og stopper ved bildekodekene, så
en `DCTDecode`-strøm kommer tilbake som en ferdig JPEG og en `FlateDecode`-strøm
som rå sampler. **Ruten velges av BYTENE, ikke av filternavnet:** snus resultatet
som et bildeformat, bæres det ordrett; ellers kodes samplene til PNG med
stdlib-`zlib`.
Måling som begrunner det: R761 har **29 av 50** DCTDecode og **21** FlateDecode.
Over det 33-dokumenters K2-korpuset er populasjonen **4 828 objekter**, og
filtrene er blandet nok (`FlateDecode`, `DCTDecode`, `JPXDecode`,
`ASCII85Decode`-kjeder, `CCITTFaxDecode`) til at en gjetning fra filternavnet
ville vært gal på flere hundre.
**Alternativet ordren nevnte — rendret bbox ved 200 dpi — ble felt på
determinisme.** Et rasterisert utsnitt ville vært én kodesti og håndtert hver
filter, men bytene, og dermed assetens innholdsadresserte navn og hele bundlens
digest, ville vært avhengige av hvilken versjon av rasteriseren som var
installert. Det er nøyaktig egenskapen `OCR_DPI` sin egen docstring allerede
innrømmer at OCR-tekst ikke kan ha. En innebygd strøm har ingen slik avhengighet.
**Det koderen ikke kan uttrykke EKSAKT, nekter den for:** stencilmaske,
`Decode`-array, CMYK, alt annet enn 8-bits sampler, en `SMask` som ikke lar seg
bære. Koden er `asset_pdf_unsupported`, den telles, og den skriver en linje i
konseptet. Et bilde som er plausibelt feil farge er feil på en måte ingen
konsument kan oppdage.
**Ingen størrelsesgulv, og det er også en måling.** Det opplagte filteret er
«ignorer alt under N piksler», og fordelingen tilbyr ingen N. Over de 4 828
objektene: **149** uten oppgitt størrelse, **162** under 32x32, **92** under
64x64, **406** under 128x128, **498** under 256x256, **590** under 512x512,
**2 931** større. Et bredt spenn uten gap — motsatt av `OCR_CID_SHARE`, som er
bimodal med ingenting mellom modene. En terskel lest av ingen gap er et tall
dette repoet valgte, og det ville stille droppet noens lille tabell.
---
## § 4 Kontrollen på bytene
To hele bygg av det 43-dokumenters referansekorpuset (`K2/trinn1`), og `diff -r`
mellom dem. En eksponeringstelling er ikke en kontroll.
**Kontroll 1 — flytter opt-outen noe?** `332961a` bygget fra `git archive` mot
HEAD med `--no-assets`:
```
865 filer på begge sider. diff -rq: ÉN forskjell.
14a15
> * **Images**: NOT CARRIED — this run did not look for images, ...
```
Hvert eneste konsept er byte-identisk. Den ene forskjellen er den nye
`log.md`-linja, og den er med vilje: en bundle ingen lette etter figurer i må
ikke kunne forveksles med en bundle av dokumenter som ikke hadde noen.
**Kontroll 2 — hva koster defaulten?** Samme commit, `--no-assets` mot default:
| | `--no-assets` | default |
| --- | --- | --- |
| konsepter | 453 | **454** |
| markdown-filer | 865 | **867** |
| assets | 0 | **2 964** |
| bundle-størrelse | 4,7 MB | **115 MB** |
| veggtid | 2 414 s | **3 088 s** |
| topp-RSS | 6,26 GB | **8,74 GB** |
| md-filer som skiller seg | — | **422 av 865** |
`log.md`: **2 964 båret av 3 145 funnet** (181 nektet, 5,8 %). **4 622 pekere**
mot 2 964 filer — innholds-dedupen folder 1 658 gjentakelser inn i filene de
allerede er.
**Det ene nye konseptet har en MÅLT årsak.** Kandidaten
`- 20 …torv ødometerapparat …` i `Del II Bilag 3.2.1 - RIG-R01 Datarapport.pdf`
er en `rule:outline`-kandidat som `--outline-gate` slipper inn når ett gjenfunnet
overskriftsspenn dekker `OUTLINE_SHARE = 0.20` av teksten. Målt:
| | tekst | spennet | andel | gaten |
| --- | --- | --- | --- | --- |
| `--no-assets` | 71 255 | 13 566 | **0,190** | droppet |
| default | 90 854 | 27 757 | **0,306** | sluppet inn |
Seksjonen holder **146** av dokumentets bilder. Pekerne er kroppstekst, så
spennet vokste og krysset terskelen. Det er ikke en segmenteringsregel som
endret seg — det er den samme regelen som leser en lengre tekst.
---
## § 5 Konsumentflatene på en bundle MED `assets/`
§ 11 punkt 1 scoper konformans til `.md`-filer, så en `.png` i `assets/` deltar
ikke. Målt, ikke antatt, på en bundle bygget fra fixture-innboksen (6 assets,
6 konsepter):
| Flate | Resultat |
| --- | --- |
| `okf check --skill … --payload …` | `conformant: 17 rules over 4 excerpts and 2 withheld entries, 0 findings`, rc **0** |
| `okf skill` | rc **0**, SKILL.md skrevet |
| `okf consume` | rc **0**, 4 utdrag; pekerne reiser med utdragsteksten |
| `okf quality` | rc **3** (ingenting kunne dømmes — hver filtype har 1 dokument, under gulvet på 5). Ingen falsk `PASS`, ingen krasj |
| guard 1.4.0 `okf.import_bundle` (Dør C) | **6 av 6** konsepter slått sammen; pekerblokkene passerer gaten som kroppstekst |
**Guarden avviser ikke binære filer** — den ser dem ikke, fordi importøren går
over `.md`. Ingen `coord-send` til `llm-ingestion-pipeline-security` er derfor
nødvendig for trinn 1.
**Men Dør C bar dem ikke.** Målt 2026-09-17, før reparasjonen: importen slo
sammen **6 av 6** konsepter og skrev **ingen `assets/`-katalog i det hele tatt**,
så hver `![…](/assets/…)` i den importerte bundlen pekte på en fil som ikke var
der — samme «komplett og ikke»-defekt én dør bortenfor. Dør C bærer nå de
assetene et SAMMENSLÅTT konsept peker på, etter samme innholdsidentitetsregel den
allerede eier. Aldri hele avsenderens `assets/`: et bilde som hører til et
konsept gaten nektet, skal ikke sitte på ryggen av ett den slapp gjennom.
---
## § 6 Hva dette IKKE dekker
- **Trinn 2 er ikke bygget.** `okf describe`, transkripsjon med vision,
verifisering mot bildet — ingenting av det finnes. Invarianten «no model calls
anywhere in the run path» er uberørt: `assets.py` ser aldri på et bilde.
- **`.png`/`.jpg` som EGNE innboksfiler er fortsatt utenfor scope**
(`extractor_unknown`), som ordren sier. Fixture-innboksens to PNG-er
rapporteres som `extractor_unknown: 2/7` på begge commits.
- **R761 er ikke bygget her.** Tallene over er K2 og fixture-innboksen. En
R761-bygging hører hjemme i `vegnormal-okf` og er deres ordre, ikke denne.
- **`--no-assets`-kontrollen er kjørt på ETT korpus.** N = 1 korpus, 43
dokumenter. Den sier ingenting om et korpus med andre filtyper.
- **181 av 3 145 bilder ble nektet** og kodene er talt, men ingen har sett på
hva de 181 var. «5,8 % nektet» er et tall, ikke en diagnose.
- **Kostnaden er publisert, ikke forsvart.** 4,7 MB -> 115 MB på 43 dokumenter
er en 24x bundle. Om defaulten skal stå er operatørens, og tallene over er hva
den avgjørelsen skal tas på.
---
## § 7 Reproduksjon
```bash
# gaten
uv run pytest tests/test_asset_gate.py -q
# baselinen, fra git archive og aldri fra arbeidstreet
git archive 332961a | tar -x -C /tmp/base332961a
PYTHONPATH=/tmp/base332961a/src python3 -m llm_ingestion_okf.cli build \
~/corpora/okf-telling-20260829/K2/trinn1 --bundle /tmp/k2-base \
--bundle-id k2-trinn1-20260903 --okf-version 0.2
# de to byggene
okf build ~/corpora/okf-telling-20260829/K2/trinn1 --bundle /tmp/k2-off \
--bundle-id k2-trinn1-20260903 --okf-version 0.2 --no-assets
okf build ~/corpora/okf-telling-20260829/K2/trinn1 --bundle /tmp/k2-on \
--bundle-id k2-trinn1-20260903 --okf-version 0.2
diff -rq /tmp/k2-base /tmp/k2-off # ett avvik: log.md
diff -rq /tmp/k2-off /tmp/k2-on # 422 md-filer + 2 964 assets
```

4
docs/2026-09-17-bildestien-0-10-1.md
View file

@ -57,7 +57,7 @@ image bytes.
**The bound is read off the corpora, not chosen.** Over the 4 828 image
objects of the 43-document reference corpus the largest is 4 515 x 4 128
(18.6 MP, a landscape drawing). Over R761's 109 delivered pictures the largest
(18.6 MP, a landscape drawing). Over a reference standard's 109 delivered pictures the largest
is 2 072 x 656 (1.4 MP).
- `MAX_IMAGE_PIXELS = 40 000 000` (2.1x the largest measured).
@ -125,7 +125,7 @@ builds a PDF with an inline image and compares two extractions with the
- `tests/test_asset_limits.py`: 17 tests, both repros red first.
- Full suite 2 045 passed, 1 skipped; `ruff`, `ruff format --check` and
`mypy --strict` clean. Re-run after `git add` against a clean tree.
- **R761 unchanged (PM's measurement, repeated):** `--gate none` gives
- **The reference standard unchanged (PM's measurement, repeated):** `--gate none` gives
`50 carried of 50 found`, 50 files in `assets/`, and `diff -rq` against the
bundle built before this change reports no difference.
- `tools/okf_accounting_gate.py`: green on all six rows.

6
docs/2026-09-17-innholdsregnskapet-bygget.md
View file

@ -9,7 +9,7 @@ written red at `0b00de4`. Its checks are unchanged here.
The gate is GREEN on all six rows (exit 0):
- rows 1 to 5 on the fixtures;
- row 6 on R761 Prosesskoden:2025, 110 of 110 units under both builds.
- row 6 on the reference standard, 110 of 110 units under both builds.
None of the three proposed exceptions was needed for these corpora. They are
still not approved, and the build does not apply them. A workbook carrying an
@ -38,7 +38,7 @@ and exits 1. `tests/test_content_accounting.py` holds that.
identity.
- The set of carried files comes from what the resolver actually resolved
and carried (`ExtractedDocument.files`), never from byte equality.
- Why not byte equality: the gate's own first run on R761 credited 7
- Why not byte equality: the gate's own first run on the reference standard credited 7
unpointed files that share bytes with pointed ones (d = 57, not 50).
4. **Exit 1 when documents were extracted and none persisted.** This is the
decision recorded in `docs/2026-09-17-innholdsregnskapet-rod-gate.md`.
@ -61,7 +61,7 @@ and exits 1. `tests/test_content_accounting.py` holds that.
- **Known-positive.** A gate that silently drops one line of `notat.md`
leaves K1b intact. It is reported as `paragraph 1 unaccounted`, and the
build exits 1 (test).
- **Control on R761, measured.** The document was built with `--gate none`,
- **Control on the reference standard, measured.** The document was built with `--gate none`,
as one concept, and then its written text was cut.
| Written text | Result |

20
docs/2026-09-17-innholdsregnskapet-rod-gate.md
View file

@ -4,7 +4,7 @@ Capability loop, step 3. This session adds no capability. It adds the gate a
later capability has to turn green, together with the fasit that gate judges
against.
- Command: `python3 tools/okf_accounting_gate.py [--json] [--r761 DIR] [--consume]`
- Command: `python3 tools/okf_accounting_gate.py [--json] [--consume]`
- Exit codes: 0 when every failing row (1–6) is green, 1 otherwise, 2 when
the gate did not run.
- Fasit: `tools/okf_witness.py`, committed as
@ -45,13 +45,13 @@ The witness counts each format by the format's own rules:
Four documents are also pinned to hand counts.
**Two witnesses disagreed by tag, and both disagreements were about placement
rather than content** (R761, measured). A section's label is `sec/label` on
rather than content** (the reference standard, measured). A section's label is `sec/label` on
7 714 sections in the XML. The JSON puts it at `sec/label` on 4 954 of them
and at `sec/title/label` on the other 2 760. A table's label is
`table-wrap/label` in the XML (10) and `table-wrap/table/caption` in the JSON
(10). The witness therefore counts ROLES, and the mapping is written in
`okf_witness._sts_role` together with these numbers. After the mapping, the
two R761 witnesses agree on all ten roles.
two witnesses agree on all ten roles.
The PM's figures were re-measured and all hold: `p` 12 528, `title` 2 761,
`td` 197, `list-item` 111, `graphic` 50, 109 other files in the zip (50
@ -73,9 +73,9 @@ The full contract is in the module docstring. Until the flag exists, row 2 is
the file's fates from the bundle bytes (assets are named by content) and from
K1b.
## Finding A: why the guard refuses R761, and the decision
## Finding A: why the guard refuses the reference standard, and the decision
The cause is measured. The R761 XML carries **71 U+00AD SOFT HYPHEN** and no
The cause is measured. The reference standard's XML carries **71 U+00AD SOFT HYPHEN** and no
other invisible codepoint, and the extracted text carries the same 71. Guard
1.4.0 lists U+00AD in `_ZERO_WIDTH_CPS` (`output.py:250`). One soft hyphen is
enough to fail_secure at both tiers; the same string without it is `clean`.
@ -124,7 +124,7 @@ See the order's closing message for the full text. In short:
| 3 | 0 of 15, u = 248, d = 2 | RED |
| 4 | 0 of 1 | RED |
| 5 | 4 of 4 | GREEN |
| 6 | 59 of 110 (R761, two builds) | RED |
| 6 | 59 of 110 (one real document, two builds) | RED |
Row 6 runs two builds:
@ -142,8 +142,8 @@ Row 7 (`--consume`) is a diagnostic and never fails.
- The witness vocabulary is ours. A format element the witness does not name
is invisible to the gate.
- The fixture corpus is constructed, one document per type. The real-corpus
row is N = 1 (R761).
- Row 6 needs the R761 source on this machine. Without it the row is RED
row is N = 1.
- Row 6 needs that document's source on this machine. Without it the row is RED
locally and SKIPPED when `CI` is set, never green.
- The `--r761` default points into another repository's data directory, and
the gate only reads from it.
- The real-corpus default pointed into another repository's data directory,
and the gate only read from it.

2
docs/2026-09-18-bildestien-holder-0-10-1.md
View file

@ -186,5 +186,5 @@ the first one red with `install lines name ['v0.10.0']`.
- The pinned `K2-bundle-default-20260912` predates the image path entirely and
cannot move.
- The review's own "not measured" list stands: the content accounting after this
change, R761's `50 carried of 50`, docx/pptx/xlsx zip-slip, SVG as active
change, the reference standard's `50 carried of 50`, docx/pptx/xlsx zip-slip, SVG as active
content, and the `[ocr]` path.

2
docs/2026-09-18-filterkjeden-og-backstoppen.md
View file

@ -161,7 +161,7 @@ simply too large — which is what the test asserts against.
## The cost on real documents
Every PDF on this machine — 78 documents, the K2 reference corpus in both
`trinn1` and `trinn2`, the shipped fixtures and R761 Prosesskoden:2025 — was
`trinn1` and `trinn2`, the shipped fixtures and a 701-page reference standard — was
run through `_pdf_images` page by page on both sides, and each side's census
was run from a **pinned tree** (`git archive 3b587ea` for before, a copy for
after) with the imported module's own `__file__` printed as the control. The

18
docs/2026-09-18-regnskapsgaten-herdet.md
View file

@ -45,7 +45,7 @@ never clean.** "Everything rejected" satisfied `u = 0, d = 0`.
**m-1:** an asset proved a carry by NAME, so a zero-byte file called
`<sha12>-x.png` read as a carry. It is now checked on the BYTES and on the
content address. The first attempt re-implemented the build's full naming rule
and reported **50 of 50** carried R761 images as missing — the build
and reported **50 of 50** carried images of the reference standard as missing — the build
lowercases the basename, folds separator runs and takes the suffix from the
bytes. The readable tail is the build's cosmetic choice and is deliberately
not reproduced here.
@ -73,7 +73,7 @@ author wrote to themselves.
**M-2:** the two STS witnesses shared ONE role map, so row 5 — "two witnesses
agree" — could not see a hole in it. They are written apart now. Measured on
R761 2026-09-18: the XML delivery places a section's label at `sec/label`
the reference standard 2026-09-18: the XML delivery places a section's label at `sec/label`
**7 714** times and inside a title **0** times; the JSON delivery puts
**2 760** of them inside the title. The XML map was narrowed to the XML fact,
and a JSON twin fixture carrying the nested placement holds the difference
@ -97,16 +97,16 @@ with no `uri` (pandoc does not see a table), and an `odt` `styles.xml` without
## 4. Row 6 and the verdict line
- **R761 refused whole under the default gate is now RED.** It was GREEN with
- **The reference standard refused whole under the default gate is now RED.** It was GREEN with
31 096 of 31 096 elements booked as coded rejections and `okf build` exiting
1 unseen: every element had a fate, so `u = 0, d = 0`. The row asks the
question separately now.
- **A skipped row never leaves the verdict unqualified.** `GATE GREEN` now
reads `GATE GREEN (row 6 not run: <reason>)`, and the exit code is non-zero
locally when a corpus source exists on the machine and the row did not run.
- **N200 Vegbygging:2024 is a second real corpus.** R761 holds **0** `fig`,
- **A second reference standard is a second real corpus.** The reference standard holds **0** `fig`,
**0** formulas and **0** references, so the gate's only real corpus could not
have found the hole in the role map. N200 carries **194** citations, **49**
have found the hole in the role map. The second carries **194** citations, **49**
figures and **135** footnotes. A `.json` file whose root carries an STS node
tree is counted as STS rather than as keys and leaves — the container is not
the content.
@ -159,9 +159,9 @@ carried into the bundle but booked as an ordinary slide and an ordinary sheet.
A third is a genuine double count in this package: **a text box's paragraph is
counted twice**, once inside the box and once in the paragraph carrying it.
On the real corpora: R761 under `--gate none` is **31 081 carried elements
On the real corpora: the reference standard under `--gate none` is **31 081 carried elements
found in the bundle, 0 claimed and not found, 65 the gate cannot check**, over
a document holding 31 096. N200 is **16 549 elements with no fate at all**,
a document holding 31 096. The second standard is **16 549 elements with no fate at all**,
because `okf build` exits 2 on it (see § 7).
`okf build --accounting` over the fixture corpus now exits 1 with **6
@ -170,7 +170,7 @@ only became reachable when the fixtures carried the constructs.
## 7. Two facts about `okf build` this run establishes
**R761 is felled 100 % by ONE rule, and it is not tier-dependent.** The
**The reference standard is felled 100 % by ONE rule, and it is not tier-dependent.** The
document is refused whole under the default `guard-trusted-source` with
`fail_secure`. Asked directly, guard 1.4.0 answers
`any-tier: invisible carrier (output:zero-width-present)`, severity HIGH,
@ -180,7 +180,7 @@ inferred: the document carries **71 U+00AD soft hyphens**, and removing them
takes the same text to `WARN`, which persists. Whether that is intended is
below.
**N200 gets 0 plans and exit 2.** The JSON delivery is fenced verbatim, so no
**The second standard gets 0 plans and exit 2.** The JSON delivery is fenced verbatim, so no
line-anchored boundary grammar fires — the same behaviour a structureless
`.txt` produces, documented in `CLAUDE.md`. It is not an accounting defect, and
the gate reports it as what it is: 16 549 elements with no fate.

8
docs/2026-09-18-utgangsbudsjett-per-ledd.md
View file

@ -138,7 +138,7 @@ are about ENUMERATION rather than about the documents. The round-2 report
counted 5 142 objects over 78 PDFs and **580** behind `[/FlateDecode
/ASCII85Decode]`. This walk finds 9 668 objects over 77 files and **0** behind
that chain. The denominators differ because the two walks are different (this
one recurses into `LTFigure`; the file R761 is not in this listing), so
one recurses into `LTFigure`; the reference standard's file is not in this listing), so
neither number is wrong about a document — but a chain count is only readable
beside the walk that produced it, and the `[/Fl /A85]` group is not one this
machine's corpora hold. The bomb that shape carries is real regardless: a
@ -222,7 +222,7 @@ loaded as a control, and the per-file counts compared.
| corpus | files | carried before | carried after | files whose count moved |
| --- | --- | --- | --- | --- |
| `~/corpora` + `tests/fixtures` | 77 | 9 306 | **9 306** | **0** |
| R761 Prosesskoden:2025 | 1 | 50 | **50** | **0** |
| the reference standard | 1 | 50 | **50** | **0** |
| both | 78 | 9 356 | **9 356** | **0** |
The rejection codes are identical too — `asset_pdf_unsupported` 314,
@ -238,10 +238,10 @@ tree taken before the prose edits; `diff -r` between that snapshot and the
committed tree touches docstrings and comments only, so what was measured is
what shipped.
R761 also settles the `[/FlateDecode /ASCII85Decode]` question the PM raised
The reference standard also settles the `[/FlateDecode /ASCII85Decode]` question the PM raised
while this order was open: **50 image objects, 29 `[/DCTDecode]` and 21
`[/FlateDecode]`, and 0 ASCII85 links.** So that chain is in neither the 77
corpus files nor R761, and the round-2 report's count of 580 is not
corpus files nor the reference standard, and the round-2 report's count of 580 is not
reproducible from anything on this machine. It changes no decision here — the
bomb that shape carries does not need a corpus to exist in — but a published
count that cannot be reproduced should not stay unqualified.

8
docs/2026-09-19-bmp-resten-terminator-og-pekerblokk.md
View file

@ -11,7 +11,7 @@ bump and no tag.
`_bmp_rle8_rows` painted what the stream held and left the rest of the frame
at palette index 0. That is what the format says about a pixel a delta
SKIPPED, so no decoder disagrees — every one of them agrees on the wrong
picture. Measured by PM on a real R761 asset, `1631e526275d-7-graphic-0003.bmp`,
picture. Measured by PM on a real asset of the reference standard, `1631e526275d-7-graphic-0003.bmp`,
352x548 = 192 896 pixels:
| cut | result |
@ -29,8 +29,8 @@ header was read wrong"). Eval point 6 of the round's own order says "corrupt
The order named two candidates — compare available bytes against
`biSizeImage`, or require an end-of-bitmap escape. Measured first, over the
19 real RLE8 assets of the frozen R761 delivery (`~/corpora/po-frosne-bundles/
r761-2025-58e1ecca6007/assets`, read only):
19 real RLE8 assets of the frozen delivery of the reference standard (its bundle's `assets/`,
read only):
```
denominator (RLE8 files): 19
@ -168,7 +168,7 @@ refusal.
- The 43-document reference corpus was not rebuilt. The accounting gate's
rows are unchanged (`GATE RED: rows 2, 3, 6`, row 3 identical at u = 19,
d = 2, 17 of 22) — the operator-accepted losses of 2026-09-17.
- No R761 end-to-end build was run this round; the 19-file lossless
- No end-to-end build of the reference standard was run this round; the 19-file lossless
measurement is over the frozen delivery's own assets, which is the input
that half of the build.
- 16 JPEG 2000 objects are still lost with a line in the concept, and WebP is

8
docs/2026-09-19-eol-stroemmen-som-ikke-maler.md
View file

@ -72,7 +72,7 @@ docstring, in the test and here, and it is not taken.
### 1.4 The corpus, measured first and measured again
Over **11 441** files scanned across the four raw standard deliveries and the
K2 reference corpus, the only BMPs on this machine are R761's **25** (24
K2 reference corpus, the only BMPs on this machine are the reference standard's **25** (24
distinct), all RLE8. Of those:
```
@ -109,7 +109,7 @@ expected to write. It did not name `conversions` -- which `asset_holds`'
conversion route now DEPENDS on -- nor `normalised_soft_hyphen`, `unaccounted`
or `double_booked`. A door built from that contract writes a ledger this gate
reads as "nothing was converted", and every converted image comes out
claimed-and-not-found: **19 of 50** on R761.
claimed-and-not-found: **19 of 50** on the reference standard.
Two tests hold the sketch, one per direction:
@ -143,7 +143,7 @@ here was measured at closing time against the queue itself, not from memory.
| what | result |
|---|---|
| R761 built from a pinned pre-change tree and from this one, shared inbox | `diff -r` **IDENTICAL**, **5 551** files |
| The reference standard built from a pinned pre-change tree and from this one, shared inbox | `diff -r` **IDENTICAL**, **5 551** files |
| assets in that bundle | **50**, 29 JPEG + 21 PNG, **50 of 50** viewable |
| conversions booked, and lossless against Pillow | **19 of 19**, 2 366 365 pixels |
| soft hyphens | 71 removed, 0 in the bundle |
@ -159,7 +159,7 @@ an editable install would otherwise have measured the working tree twice.
## 7. What this round did not measure
* The 43-document reference corpus. Unchanged since before the round and not
rebuilt (cost); the argument for it is the same as R761's, since the two
rebuilt (cost); the argument for it is the same as for the reference standard, since the two
clauses can only fire inside an RLE8 BMP and 0 of that corpus's files are
one.
* Fidelity, which neither route of the judge proves. Unchanged.

14
docs/2026-09-19-gjenfinningsgaten-motstand.md
View file

@ -58,8 +58,8 @@ repositories by path + sha256. Nothing from them is committed here.
```
8 the real sets (three; one is a consumer's and is not restated) | RED
<consumer set>: figures withheld -- see the note below
r761-sk2: 7 of 7 fasit entries (concept) | 7 of 7 questions | sha256 c834a478e488
vegnormal-32: 31 of 43 fasit entries (concept) | 31 of 37 questions | sha256 c6d9305a0497
set-b: 7 of 7 fasit entries (concept) | 7 of 7 questions | sha256 c834a478e488
set-c: 31 of 43 fasit entries (concept) | 31 of 37 questions | sha256 c6d9305a0497
NOT SUMMED INTO ONE NUMBER: the citation-granularity set and the
concept-granularity sets are two numbers
```
@ -68,7 +68,7 @@ repositories by path + sha256. Nothing from them is committed here.
class c.
**Two of the three reproduce PM's recorded figures exactly**: wiki at 6 of 20
questions and r761 at 7 of 7. **The third does not, and the difference is
questions and set b at 7 of 7. **The third does not, and the difference is
stated rather than smoothed.** PM recorded `32 of 43 citations = 21 of 32
questions`; this gate measures 31 of 43 and 31 of 37. The question counts are
not comparable at all: the gate splits a question citing two standards into
@ -78,8 +78,8 @@ disagreement between two instruments over the same pinned bytes; it is not
resolved here and it is not this round's subject.
**The headline is at QUESTION granularity**, which is the fix for J2b: the
three sets share no unit — wiki names a citation, r761 a section title,
vegnormal a requirement number — and a citation hit plus a concept hit is a
three sets share no unit — wiki names a citation, set b a section title,
set c a requirement number — and a citation hit plus a concept hit is a
number that is neither. A question is the one unit all three have. "Answered"
means at least one of its fasit entries arrived, the reading row 1 already
prints beside its own units.
@ -194,6 +194,6 @@ where the two readings differ.
| source | measured here 2026-09-19 |
|---|---|
| a consumer's own set (read by path + sha256, never committed) | figures not restated here |
| `vegnormal-okf/docs/2026-09-10-sk2-r761-sporsmal.json` | sha `c834a478…`, 8 entries, 7 positives + KN |
| `vegnormal-okf/build/resonnering/sporsmal-m2.json` | sha `c6d9305a…`, 32 questions, 43 `must_cite` |
| set b, a consumer repository's file (by path + sha256) | sha `c834a478…`, 8 entries, 7 positives + KN |
| set c, a consumer repository's file (by path + sha256) | sha `c6d9305a…`, 32 questions, 43 `must_cite` |
| K2 | 6 questions recorded; no gold set anywhere |

10
docs/2026-09-19-hvert-baaret-bilde-kan-vises.md
View file

@ -1,14 +1,14 @@
# Every carried image is one a model can be shown
`llm-ingestion-okf`, 2026-09-19. Raised from an internal measurement of the
frozen R761 package's `assets/` directory.
frozen package of the reference standard: its `assets/` directory.
Nothing is tagged, pushed or released here. No new version: the entries go
under the untagged `[0.10.1]`.
## 1. The defect
`file` over the frozen R761 delivery's own `assets/`, denominator **50**:
`file` over the frozen delivery of the reference standard, its own `assets/`, denominator **50**:
| Format | Count | A model can show it |
|---|---|---|
@ -194,14 +194,14 @@ than buried in a total.
The conversion broke the judge, and the judge caught it. `asset_holds` proved
a carry by hashing the SOURCE file and looking for those bytes in `assets/`;
a converted image's source bytes are not in the bundle. Measured on R761:
a converted image's source bytes are not in the bundle. Measured on the reference standard:
| | claimed and not found | row 6 |
|---|---|---|
| with the conversion, before the judge's second route | **19** | 90 of 111 |
| after | **0** | 110 of 111 |
19 is exactly R761's RLE8 count. The judge now also reads the two digests the
19 is exactly that document's RLE8 count. The judge now also reads the two digests the
bundle states on the pointer line and then hashes the asset itself, so a
bundle claiming a conversion it did not perform still fails — three cases in
one test, one positive and two negative.
@ -210,7 +210,7 @@ The gate's verdict is unchanged: **`GATE RED: rows 2, 3, 6`**, the same rows
STATE recorded before the round, with row 3's numbers (u = 19 unaccounted,
d = 2 double-booked, 17 of 22) identical across both runs — they are the
reference corpus's own losses, which the operator left standing red on
2026-09-17. R761 reads exit 0, 1 of 1 persisted, **50 asset files, 31 081
2026-09-17. The reference standard reads exit 0, 1 of 1 persisted, **50 asset files, 31 081
carried elements found, 0 claimed and not found**.
## 8. What this round does NOT do

4
docs/2026-09-19-pekerblokken-bindes-til-kjoringen.md
View file

@ -65,7 +65,7 @@ shape anchoring already refused, kept so a later narrowing cannot widen one of
them, plus the two forms that defeated it — a perfectly written pointer block
the run never booked.
R761 rebuilt from the frozen delivery: bundle `diff -r`-identical to the build
The reference standard rebuilt from its frozen delivery: bundle `diff -r`-identical to the build
before the change, 50 assets (29 JPEG + 21 PNG, 0 BMP), 19 of 19 conversions
confirmed against 19 declared, soft hyphens 71 → 0, u = 0, d = 0, exit 0.
@ -83,7 +83,7 @@ the last row, refusing with the same `asset_samples_invalid` otherwise.
escape STATE their skip, so the pixels they pass over keep index 0 and every
decoder produces the same picture; a pixel-coverage count would refuse both
constructions the format defines. The corpus cannot choose between the two
rules — over the 25 RLE8 BMPs the R761 delivery ships (24 distinct; the bundle
rules — over the 25 RLE8 BMPs the reference delivery ships (24 distinct; the bundle
carries 19), 25 of 25 paint every pixel, 25 of 25 reach the end of the frame
and 0 of 25 use a delta. An independent decoder can:

48
docs/2026-09-19-regnskapsgaten-rest-og-normaliseringsdoren.md
View file

@ -16,18 +16,18 @@ Nothing is tagged, pushed or released here.
| H3 | `math`/`figure_caption` do not fire on a real delivery | `_local` on the JSON tag + a `figcaption` clause | `37599a3` -> `d31ab73` |
| H4 | the mutation harness is not a gate | `verdict(survived, errors)` | `6abe4b4` -> `f5b263f` |
| H5 | the skipped-row guard is unreachable | it measures the DEFAULT sources | `253c4fb` -> `8b4f409` |
| H6 | N200 contributes one blank red | `measures_no_class` in row 6 | `e6d9f16` -> `280d6bd` |
| — | R761 refused whole over 71 soft hyphens | `extract.normalise_extracted` | `68cfa90` -> `eebaf53` |
| H6 | the second real corpus contributes one blank red | `measures_no_class` in row 6 | `e6d9f16` -> `280d6bd` |
| — | the reference standard refused whole over 71 soft hyphens | `extract.normalise_extracted` | `68cfa90` -> `eebaf53` |
## 2. The normalisation door (operator decision 2026-09-18)
`llm-ingestion-guard` 1.4.0 keeps U+00AD in `_ZERO_WIDTH_CPS` beside U+200B,
U+200C, U+200D and U+FEFF, and `output:zero-width-present` is an **any-tier
carrier**: `fail_secure` at every trust level, with no sanitisation and no
exception. R761 Prosesskoden:2025 carries **71 U+00AD and 0** of the four real
exception. A 701-page reference standard carries **71 U+00AD and 0** of the four real
zero-width characters (U+2060 also 0). All 71 are Norwegian hyphenation points
inside words -- `ar[SHY]beider`, `bitu[SHY]men`, `asfalt[SHY]betong` -- so a
701-page process code was unreadable for the whole chain over typography. The
inside words -- `ar[SHY]beider` and its like -- so a
701-page reference standard was unreadable for the whole chain over typography. The
verdict was formally right and materially a false positive.
Of the three ways out, the operator chose one door in okf, before the guard,
@ -55,9 +55,9 @@ levels and `log.md` as a `**Normalisation**` bullet.
**What it does not touch.** The guard. The other four characters -- they carry
no typographic job in running text, so removing one would be a decision about
what the guard screens for, taken in the wrong repository. U+00A0 NBSP, which
is not in the guard's set at all (6 633 of them in R761).
is not in the guard's set at all (6 633 of them in the reference standard).
**Measured, on R761's own delivery, through the DEFAULT gate:**
**Measured, on the reference standard's own delivery, through the DEFAULT gate:**
```
exit 0, wall 25.2 s
@ -120,9 +120,9 @@ Two independent causes, one line each.
- `count_sts_xml` takes every tag through `_local`, which strips a
Clark-notation namespace AND a prefix. `count_sts_json` compared the raw
string, so `mml:math` reached `tag == "math"` on nothing: **74** formulas in
N200, 2 811 characters, counted as 0.
the second real corpus, 2 811 characters, counted as 0.
- The publisher's JSON writes a figure's caption as `figcaption` under the
`graphic`, not as the `fig/caption` NISO-STS writes: **49** in N200, counted
`graphic`, not as the `fig/caption` NISO-STS writes: **49** in the second real corpus, counted
as 0.
The `figcaption` clause is in the JSON map ALONE. The two maps are written twice
@ -134,17 +134,17 @@ on this machine and the three committed twins:
| delivery | before -> after |
|---|---|
| N100-2022 | `figure_caption` 0 -> 123 |
| N100-2023 | `figure_caption` 0 -> 127 |
| N101-2025 | `figure_caption` 0 -> 62 |
| N200-2024 | `figure_caption` 0 -> 49, `math` 0 -> 74 |
| N303-2026 | `figure_caption` 0 -> 43, `math` 0 -> 3 |
| N500-2024 | `figure_caption` 0 -> 27, `math` 0 -> 88 |
| N601-2022 | no change |
| R761-2025 | no change (row 5's pair) |
| delivery A | `figure_caption` 0 -> 123 |
| delivery B | `figure_caption` 0 -> 127 |
| delivery C | `figure_caption` 0 -> 62 |
| delivery D (the second real corpus) | `figure_caption` 0 -> 49, `math` 0 -> 74 |
| delivery E | `figure_caption` 0 -> 43, `math` 0 -> 3 |
| delivery F | `figure_caption` 0 -> 27, `math` 0 -> 88 |
| delivery G | no change |
| the reference standard | no change (row 5's pair) |
| the two existing twins | no change |
The known-negative for stripping a prefix is N101's **568 `tbx:`** tags: not one
The known-negative for stripping a prefix is delivery C's **568 `tbx:`** tags: not one
of their local names is a role here, and it is in the suite.
## 5. H2, H4, H5, H6
@ -167,8 +167,8 @@ removes `unverified`, `invalid` AND `refused` at once and would be killed by
any one of the three. Final run: **killed 36 of 36, 0 survived, 0 errors, exit
0**.
**H5.** The skipped-row guard measures `real_corpora(R761_DEFAULT,
N200_DEFAULT)`. Row 6 is SKIPPED exactly when the corpora the ARGUMENTS name are
**H5.** The skipped-row guard measures `real_corpora` over the two DEFAULT
sources. Row 6 is SKIPPED exactly when the corpora the ARGUMENTS name are
absent, so asking the arguments made the branch unreachable. Two tests, so a
guard that fires on everything fails too: with the default source present and
the arguments pointing elsewhere the run exits 1 and names the row; with no
@ -176,12 +176,12 @@ corpus on the machine it exits 0.
**H6.** `measures_no_class(units)` returns a sentence when EVERY document of a
corpus has no declared fate, and row 6 prints it beside that corpus's tally.
Reproduced 2026-09-19: `okf build` on N200 proposes 0 plans, prints `FAILED - no
Reproduced 2026-09-19: `okf build` on the second real corpus proposes 0 plans, prints `FAILED - no
segmentation plans` and exits 2 BEFORE the accounting door -- **no accounting
file is written at all**. What it would take is a capability and not a
threshold: `.json` is read as generic JSON, and the publisher's STS delivery in
that form would have to reach the markdown grammar `_extract_xml` writes for the
XML one. `standardContent` occurs **0** times in `src/` and 4 in the witness,
XML one. The JSON delivery's content key occurs **0** times in `src/` and 4 in the witness,
with the query validated against that known positive.
## 6. The gate, after
@ -200,10 +200,10 @@ with the query validated against that known positive.
Rows 2 and 3 are the reference corpus's own losses, which the operator decided
on 2026-09-17 stay red until they are fixed.
Row 6 keeps its count and changes its cause. **R761 now passes the DEFAULT
Row 6 keeps its count and changes its cause. **The reference standard now passes the DEFAULT
gate**: `exit 0, 1 of 1 document(s) persisted, 50 asset file(s); 31 081 carried
element(s) found in the bundle, 0 claimed and not found`, `u = 0, d = 0` under
both gates. The only `refused whole` left is N200, and that is the build exiting
both gates. The only `refused whole` left is the second real corpus, and that is the build exiting
2 on 0 plans -- H6's blank, now stated in the row.
## 7. Limits of this round

4
docs/2026-09-20-mcp-to-varianter.md
View file

@ -109,9 +109,9 @@ Three numbers, and the middle one is the finding:
* **99 of 181 pairs are present in the bundles at all.** 82 are not: the text
the set quotes is not in the bundle, which is red for the BUNDLE and not for
the server. `r761-2025` is the sharpest case at 17 of 33 present.
the server. One single-document bundle is the sharpest case at 17 of 33 present.
* **83 of the 99 present were reached**, so the surface reaches 83.8 % of what
is there. `r761-2025` is again the outlier: 2 reached of 17 present.
is there. That bundle is again the outlier: 2 reached of 17 present.
* **0 of 83 were met by `okf_fetch` on the anchor as a concept id.** The set's
anchors (`Krav 2.3.1—3`) and this library's concept ids are different
vocabularies, so the cheap route -- a true ceiling -- never fires, and every

529
docs/okf-nokkelinventar.md
View file

@ -1,529 +0,0 @@
# OKF-nøkkelinventaret — 17 SPEC-nøkler og 33 egne, hver klassifisert med kommando
**Ordre K2/4** `20260827T151230Z-5121410292-from-.claude` · kartleggingsfase: ingen
kodeendring, ingen anbefaling om lukking, ingen ny målearm. Skrevet 2026-08-28 av en
Fable 5-økt **uten advisor**; kommandokravet er den eneste kontrollen. Hvert tall under
er produsert av kommandoen som står ved siden av det. Der noe ikke er målt står
«ikke målt»; der noe er antatt står «antatt»; der en seksjon ikke er lest står «ikke lest».
Inventaret bor her fordi `profiles.py` er stedet et navnevalg tas i biblioteket — men
**korpusets** nøkler velges i `vegnormal-okf/src/vegnormal_okf/bundle.py`, og det
repoet er lest, aldri skrevet. Hva som gjøres med et funn er fase 2/3 sitt, ikke dette
dokumentets.
## 0. Grunnlaget
### 0.1 Gate: SPEC-teksten er re-verifisert mot GitHub HEAD (28.08 kl 03)
```
curl -s -o /tmp/kc-spec.md 'https://raw.githubusercontent.com/GoogleCloudPlatform/knowledge-catalog/HEAD/okf/SPEC.md'; echo "curl exit=$?"
# -> curl exit=0 (37748 bytes)
diff /tmp/kc-spec.md ~/repos/_okf-canonical/SPEC.md | wc -l
# -> 0
wc -l ~/repos/_okf-canonical/SPEC.md; sed -n 3p ~/repos/_okf-canonical/SPEC.md
# -> 1006 · **Version 0.2**
git -C ~/repos/_okf-canonical log -1 --format='%h %ci' # -> ad30107 2026-08-21
git -C ~/repos/_okf-upstream log -1 --format='%h %ci' # -> 9a15b13 2026-08-24 (ORDREN SA 3fcbb9f)
git -C ~/repos/_okf-upstream reflog --date=iso -1 # -> checkout: 3fcbb9f -> origin/main, 2026-08-27 19:57:35 +0200
```
Utfallet er 0. `.claude` sin måling 27.08 kl 17 står, og grunnlaget holder. **Én
observasjon utenfor ordren:** den lokale `_okf-upstream`-kopien ble flyttet fra
`3fcbb9f` til `origin/main` (`9a15b13`) 27.08 kl 19:57 — etter at programplanen
skrev «baselinjen er den målte verdien, ikke det `git log` sier når K3 kjører».
Ingenting i dette dokumentet leses fra `_okf-upstream` unntatt v0.1-historikken
(§ 0.5), som er commit-adressert og upåvirket av HEAD.
### 0.2 Materialet og variablene
Alle kommandoer kjøres fra `~/repos/llm-ingestion-okf` med disse variablene satt:
```
S=~/repos/_okf-canonical/SPEC.md
B=~/repos/vegnormal-okf/build
E=$B/A-n500-2024 # EMITTERT av vegnormal-okf: krav/N500/id-*.md + normal/N500.md — 270 md-filer, ingen index.md
M=$B/F3-n500-2024-importert # IMPORTERT gjennom Dør C (guard 1.2.0): 270 konsepter + 1 index.md, flat — det armene F3/F4 leste
I=src/llm_ingestion_okf # dette repoet (importøren / emitterne A og B)
P=$I/profiles.py
V=~/repos/vegnormal-okf/docs/VEGNORMAL_V1.md
K1=~/repos/vegnormal-okf/docs/okf-spec-konformitet.md # ordre K1, levert 27.08
K3=~/repos/vegnormal-okf/docs/okf-eksempelkorpus.md # ordre K3, levert 28.08
R=~/repos/vegnormal-okf/docs/2026-08-26-resonnering-resultater.md
G=~/repos/llm-ingestion-pipeline-security/src/llm_ingestion_guard/okf.py # guard-kilden; installert versjon måles i .venv
```
Nevnere: `find $E -name '*.md' | wc -l` → **270** · `ls $M/*.md | wc -l` → **271**
(270 konsepter + `index.md`) · `find $B -name '*.md' ! -name index.md | wc -l` →
**48 983** (hele `build/`, alle armer og grenbaser) · `wc -l $P` → **1047** ·
`.venv/bin/python -c "import importlib.metadata as m; print(m.version('llm-ingestion-guard'))"`
→ **1.3.0** (området `pyproject.toml` l.35 `>=1.2,<2.0`; taggen som avgjør
versjonen er l.139 `tag = "v1.3.0"` — var `v1.2.0` til 2026-09-03).
### 0.3 Tallene 17, 3 og 33 — produsert av kommando og avstemt mot operatørens
**17 SPEC-nøkler (§ 4 og § 5):**
```
sed -n 153,436p $S | grep -oE '^\s*(- )?[a-z_]+:' | sed -E 's/^[[:space:]]*(- )?//; s/://' | sort | uniq -c
# author 1 · description 3 · generated 3 · id 1 · last_modified 1 · resource 3 · sources 1 · stale_after 1
# status 1 · tags 3 · title 4 · type 3 · usage_count 1 · usage_window 1 · verified 2
sed -n 153,436p $S | grep -oE '^\s*(- )?[a-z_]+:' | sed -E 's/^[[:space:]]*(- )?//; s/://' | sort -u | wc -l
# -> 15
```
Kommandoen gir **15 unike navn**. `resource` og `title` forekommer både på toppnivå
(§ 4.1) og inne i en `sources`-oppføring (§ 5.1) med **ulik betydning** (asset-URI vs
kildens URI; konseptets navn vs kildens navn). Telles `sources[].resource` og
`sources[].title` som egne nøkler, blir det 15 + 2 = **17 — operatørens tall.**
Toppnivå alene er 11; § 5.1 lister nøyaktig seks oppføringsnøkler (l.303–313:
`resource`, `id`, `title`, `author`, `usage_count`, `last_modified`). `by`/`at` (feltene
i `generated`/`verified`-mappingene, l.377–389) er ikke talt; med dem ville det vært 19.
**Avstemt: 17 = 11 toppnivå + 6 `sources`-oppføringsnøkler.** Del A har én rad per nøkkel.
**3 i bruk hos oss:**
```
for k in type title description resource tags sources usage_window generated verified status stale_after; do printf '%s=%s ' $k "$(grep -l "^$k:" $M/*.md | wc -l | tr -d ' ')"; done; echo
# -> type=270 title=270 description=0 resource=0 tags=0 sources=0 usage_window=0 generated=0 verified=0 status=270 stale_after=0
find $E -name '*.md' -print0 | xargs -0 grep -l '^sources:' | wc -l # -> 270
```
**3 i M (`type`, `title`, `status`) — operatørens tall holder for bundlen armene leste.**
I E er det **4**: `sources` skrives 270/270 av emitteren og fjernes før import fordi
guarden avviste formen (G30; **falt ved 1.3.0**, § 0.7 — men M-tallet er målt på
en import kjørt under 1.2.0 og er ikke re-målt). Avstemt.
**33 egne:**
```
for f in $M/*.md; do awk 'NR==1 && $0!="---"{exit} NR>1 && $0=="---"{exit} NR>1 && /^[A-Za-z_][A-Za-z0-9_]*:/{sub(/:.*/,""); print}' "$f"; done | sort -u | wc -l
# -> 36
# 36 − 3 spec-nøkler (type, title, status) = 33 — operatørens tall, i M.
find $B -name '*.md' ! -name index.md -print0 | xargs -0 awk 'FNR==1 && $0!="---"{nextfile} FNR>1 && $0=="---"{nextfile} FNR>1 && /^[A-Za-z_][A-Za-z0-9_]*:/{sub(/:.*/,""); print}' | sort -u | wc -l
# -> 44 over hele build/ (48 983 filer): 40 egne + type, title, status, sources
```
**Avstemt: 33 i M; 40 over hele `build/`.** De 7 ekstra er `erstatter`,
`erstattere_i_bundlen` (kun N100:2023-erstatter-bundlene), `gren`, `gren_direkte`,
`krav_i_normalen`, `grener_i_normalen` (kun grenbaser) og `viser_til_tabellnote` (0 i
N500). Del B.1 bærer alle 40 og merker hvilke 33 som står i M. Én nøkkel til er
**definert men aldri skrevet**: `viser_til_note` (`lenker.py` l.48, `fn`-klassen) —
`find $B -name '*.md' ! -name index.md -print0 | xargs -0 grep -l '^viser_til_note:' | wc -l`
→ **0** av 48 983 (kjent-positiv, samme spørring: `viser_til_formel` → 203). Den er
ikke en rad, fordi ingen bundle bærer den.
### 0.4 Premisset om `profiles.py` l.295 og l.770 — verifisert, ikke ordrett
```
grep -n 'while remaining v0.1 on every axis upstream owns' $P; echo exit=$?
# -> exit=1 (0 treff: frasen finnes IKKE ordrett på én linje)
grep -n 'v0.1 on every axis upstream' $P
# -> 295: ratified it (V1, `54e0ec7`), while remaining v0.1 on every axis upstream
# -> 770:# §5 layer, while the profile remains v0.1 on every axis upstream owns.
tr '\n' ' ' < $P | tr -s ' ' | grep -o 'remain[a-z]* v0.1 on every axis upstream owns'
# -> remaining v0.1 on every axis upstream owns (l.295–296, brutt over linjeskift)
# -> remains v0.1 on every axis upstream owns (l.770, annen bøyning)
```
Samme funn som K1 (§ 13): **ordrett for l.295 (over to linjer), i variant for l.770.**
Et linjebasert `grep` på frasen gir null — det er den samme fella som K1s `MUST NOT`.
**Hva det betyr for nøklene:** begge linjene beskriver `DEFAULT` sin `ownership`
(commons' ingest-spec § 7-lag). «v0.1 på hver akse upstream eier» betyr at `DEFAULT`
navngir **ingen** av v0.2-familiene i § 5 unntatt `generated`, som den skriver i O2-form
fordi commons ratifiserte den (`sed -n 805p $P` → `ownership=OwnershipPolicy(actor="process:okf-ingest")`).
Bibliotekets v0.2-støtte er profilen `OKF_V0_2` (`sed -n 1006p $P`), som **navngir**
`sources`, `verified`, `status`, `stale_after` (l.947–950) og § 10.2-feltene (l.942) og
**skriver** bare `sources` av dem (`materialize.py` l.241–242, kun under den profilen).
Korpusets emitter bruker ingen profil (K1: `grep -c BundleProfile src/vegnormal_okf/*.py`
→ 0), og Dør C importerer under `DEFAULT` (`sed -n 365p $I/importer.py` →
`profile: BundleProfile = DEFAULT`). Premisset gjelder altså importørens indeks- og
eierskapslag — ikke hva korpuset bærer.
### 0.5 Hvilke av de 17 kom TIL i v0.2 — målt mot v0.1-teksten, ikke antatt
```
git -C ~/repos/_okf-upstream log --format='%h %ci %s' -- okf/SPEC.md | tail -1
# -> ee67a5c 2026-06-11 Import Open Knowledge Format reference enrichment agent (#28)
git -C ~/repos/_okf-upstream show ee67a5c:okf/SPEC.md | sed -n 3p
# -> **Version 0.1 — Draft**
git -C ~/repos/_okf-upstream show ee67a5c:okf/SPEC.md | grep -oE '^\s*(- )?[a-z_]+:' | sed -E 's/^[[:space:]]*(- )?//; s/://' | sort -u | tr '\n' ' '
# -> description resource tags timestamp title type (6 nøkler i v0.1)
git -C ~/repos/_okf-upstream log --format='%h %ci %s' -S'Version 0.2' -- okf/SPEC.md | tail -1
# -> 780fe9d 2026-07-24 okf: migrate format and tooling to Open Knowledge Format v0.2 (#227)
sed -n 813,832p $S | grep -oE '`[a-z_]+`' | sort -u | tr '\n' ' '
# -> § 13.2: attester author computation description executor generated last_modified parameters resource runtime sources stale_after status tags title type usage_count usage_window verified
```
| kom i | nøkler | antall | kommando |
|---|---|---:|---|
| **v0.1, båret videre** | `type`, `title`, `description`, `resource`, `tags` | 5 | `git -C ~/repos/_okf-upstream show ee67a5c:okf/SPEC.md \| grep -oE '^\s*(- )?[a-z_]+:'` → de 6 over, minus `timestamp` |
| **v0.2, nye** | `sources`, `sources[].resource`, `sources[].id`, `sources[].title`, `sources[].author`, `sources[].usage_count`, `sources[].last_modified`, `usage_window`, `generated`, `verified`, `status`, `stale_after` | 12 | `sed -n 813,832p $S` (§ 13.2 «New frontmatter families») — de seks oppføringsnøklene er del av den nye `sources`-familien |
| **v0.1, avviklet** | `timestamp` → `generated.at` | 1 | `sed -n 805,808p $S` (§ 13.1) · `sed -n 42p $P` → `_TIMESTAMP_FALLBACK_PAIR = frozenset({"timestamp", "generated"})` |
**Konsekvens:** av de tre nøklene korpuset bruker er to v0.1 (`type`, `title`) og én
v0.2 (`status`); E skriver dessuten `sources` (v0.2) og 0 `timestamp`. Korpuset er
dermed **ikke «v0.1»** — det skriver to v0.2-nøkler og ingen v0.1-eksklusiv. `DEFAULT`
skriver v0.1-nøklene + `generated` i v0.2-form. `STRICT_V1` navngir `timestamp`
(`sed -n 819p $P`), som er den avviklede v0.1-nøkkelen — det er den eneste profilen
som gjør det, og den speiler en annen konsuments ratifiserte kontrakt (B.2).
### 0.6 Tidsformatet — hva VI skriver, målt
Regelen: `sed -n 284,285p $S` → «Every timestamp-valued key in OKF is an ISO 8601
datetime with an explicit UTC offset». Nøklene den binder: `generated.at`,
`verified[].at`, `sources[].last_modified`, `usage_window.{from,to}`, `stale_after` (5).
| hvem | nøkkel | form skrevet | konform | kommando |
|---|---|---|---|---|
| bibliotek, Dør A | `ingested_at` + `generated.at` | `2026-07-16T12:00:00Z` — 10/10 i goldens | **ja** (`Z` er spec-ens eget eksempel) | `grep -rhoE '^(ingested_at\|generated\|timestamp\|date\|stale_after): .*' examples \| sort \| uniq -c` → `5 generated: { by: process:okf-ingest, at: 2026-07-16T12:00:00Z }`, `5 ingested_at: 2026-07-16T12:00:00Z` · porten: `sed -n 37p $I/materialize.py` → `_INGESTED_AT_RE = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$")`, `sed -n 49p` (`validate_ingested_at`), `sed -n 235p` (`generated` avledes av samme verdi) |
| bibliotek, Dør B | `generated` | literalen `true` — ingen `at` i det hele tatt | **ikke en tidsverdi**; malformert etter § 5.2 (se Del A, `generated`) | `sed -n 143p $I/inbox.py` → `"generated": "true",` |
| korpus (E og M) | de fem spec-tidsnøklene | **0 skrevet** | ikke utløst | `for k in generated verified stale_after last_modified usage_window; do printf '%s=%s ' $k "$(find $E -name '*.md' -print0 \| xargs -0 grep -c "^ *$k:" \| awk -F: '{s+=$NF}END{print s}')"; done` → alle 0 |
| korpus | `kravdato` (egen nøkkel) | `YYYY-MM-DD`, 269/269; 0 med `T` | **utenfor regelen** (produsentnøkkel, `VEGNORMAL_V1 § 4.2` l.214) — men blir den noen gang mappet til en spec-tidsnøkkel, er formen feil; upstreams leser gjør en dato-only `stale_after` til *aldri stale* (`docs/plan/okf-2026-08-timestamp-tightening.md` l.144) | `find $E -name '*.md' -print0 \| xargs -0 grep -h '^kravdato:' \| grep -cE "^kravdato: '?[0-9]{4}-[0-9]{2}-[0-9]{2}'?$"` → 269 · `… \| grep -cE 'T[0-9]{2}:'` → 0 |
| bibliotek, `STRUCTURED_V1` | `date` (egen nøkkel, deklarert av kaller) | testene bruker `YYYY-MM-DD` | utenfor regelen (produsentnøkkel) | `grep -ho 'date: [0-9][^"\\]*' tests/*.py \| sort \| uniq -c` → `date: 2024-06-01`, `2026-01-01`, `2026-08-27` |
| bibliotek, `STRICT_V1` | `timestamp` (wikiens nøkkel, avviklet i v0.2) | testene bruker `Z`-form | formen er datetime; nøkkelen er § 13.1-avviklet | `grep -ho '"timestamp": "[^"]*"' tests/*.py \| sort -u` → `"2026-07-25T12:00:00Z"` |
**Hva vi skriver er konformt der regelen binder (10/10), og vi skriver ingen av de tre
nøklene den strammet inn** (`stale_after`, `last_modified`, `usage_window`). Den gamle
`YYYY-MM-DD`-formen finnes hos oss bare i produsentnøkler (`kravdato`, `date`).
### 0.7 Guard 1.2.0 og 1.3.0 — hva porten faktisk slipper gjennom (probe, ikke målearm)
Kjørt mot den installerte guarden i `.venv` (samme som Dør C bruker), med kjent-positiv
først. Dette er verdiformer, ikke en tiende arm.
```
.venv/bin/python -c '
from llm_ingestion_guard import okf
for n, d in {
"KJENT-POSITIV title": "---\ntype: Krav\ntitle: Krav 5.1-1 Generelt\n---\nb\n",
"description en linje": "---\ntype: Krav\ndescription: One sentence summarising the concept.\n---\nb\n",
"description brettet": "---\ntype: Krav\ndescription: One sentence that\n continues on a second line.\n---\nb\n",
"tags flow": "---\ntype: Krav\ntags: [a, b]\n---\nb\n",
"tags blokkliste": "---\ntype: Krav\ntags:\n - a\n - b\n---\nb\n",
"sources blokk-mappinger": "---\ntype: Krav\nsources:\n - resource: https://x\n title: N500:2024\n---\nb\n",
"sources flow-sekvens": "---\ntype: t\nsources: [{ id: a, resource: fixture }]\n---\nb\n",
"generated flow-mapping": "---\ntype: Krav\ngenerated: { by: process:okf-ingest, at: 2026-07-16T12:00:00Z }\n---\nb\n",
"generated: true (Dor B)": "---\ntype: Krav\ngenerated: true\n---\nb\n",
}.items():
try: okf.parse_frontmatter(d); print("PASS ", n)
except Exception as e: print("REJECT", n, "->", str(e)[:70])'
```
| verdi | guard 1.2.0 | guard 1.3.0 | betydning |
|---|---|---|---|
| `title` én linje (kjent-positiv) | PASS | PASS | spørringen kan skille — `okf.parse_frontmatter` returnerer `({'type': 'Krav', 'title': …}, body)` |
| `description` én linje | **PASS** | PASS | en énlinjes `description` ville passert Dør C i dag (`okf.parse_frontmatter` → `{'description': 'One sentence …'}`) |
| `description` brettet over to linjer | REJECT | REJECT | `nested mappings are not supported in OKF frontmatter` — G33s «29/53 brettet» står for upstreams bundler (`~/.claude/docs/okf-bundle-prosessen.md` l.130) |
| `tags: [a, b]` | REJECT | REJECT | **samme dom, ny begrunnelse ved 1.3.0:** `a flow sequence admits flow mappings only: '[a, b]'` (var: `value begins with a disallowed YAML indicator '['`). En flow-sekvens av skalarer er fortsatt ute; det er flow-sekvensen av *mappinger* som åpnet |
| `tags:` blokkliste | **PASS** | PASS | G33s «44/53 blokk avvist … ingen sekvens-verditype» gjelder en eldre guard; 1.2.0 leser `['a', 'b']` (`okf.parse_frontmatter`) |
| `sources:` blokk-sekvens av mappinger (E-formen, `bundle.py` l.167) | REJECT | **PASS** | **G30 FALT ved 1.3.0.** Var `a mapping is not expressible in OKF frontmatter`. Men porten er ikke det eneste hinderet: *vår egen* parser er linjeorientert og kan ikke lese blokkformen tilbake, så at guarden nå tar imot den endrer ikke at dette biblioteket bare skriver flow (`CLAUDE.md § Upstream version policy`) |
| `sources: [{ … }]` flow-sekvens (Dør A under `OKF_V0_2`, `materialize.py` l.241) | REJECT | **PASS** | vår egen golden passerer nå: `.venv/bin/python -c 'from llm_ingestion_guard import okf; fm, _ = okf.parse_frontmatter(open("examples/ingest-golden-okf-v0-2/expected-bundle/ingest-sales.md").read()); print(fm["sources"])'` → `[{'id': 'golden-v0-2-sales', 'resource': 'fixture'}]`. Ved 1.2.0: `OKFFrontmatterError … '['`. Pinnet av `tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit` |
| `generated: { by, at }` flow-mapping | **PASS** | PASS | 1.2.0 leser mappingen som `{'by': …, 'at': …}` — kommentaren i `$P` l.998–1003 («no expressible form») er foreldet for denne nøkkelen |
| `generated: true` (Dør B) | PASS | PASS | passerer porten som streng `'true'`; porten dømmer form, ikke § 5.2 |
Guardens mapping-nøkkel-allowlist **flyttet ved 1.3.0, og foreldrenøkkelen avgjør nå.** Ved 1.2.0: `by at from to id title author usage_count last_modified` — **uten `resource`**, som var hvorfor `sources[].resource` inne i en mapping stoppet (registerets G30-forklaring, l.133). Ved 1.3.0 er `resource` tillatt **kun inne i en `sources`-oppføring** (`_SOURCES_ENTRY_KEYS`, `okf.py` l.102). Målt, med kjent-positiv: `sources: [{ id: a, resource: fixture }]` → PASS · `executor: { id: e, resource: … }` → REJECT `mapping key 'resource' is not on the OKF mapping allowlist under 'executor'` · `attester: { … }` → REJECT tilsvarende · toppnivå `resource:` (§ 4.1) → PASS. § 10.2s kjøretids-`resource` er altså fortsatt ute gjennom hver bærer, som er den grensen Dør C hviler på.
---
## Del A — de 17 nøklene SPEC navngir (§ 4 og § 5)
`bruker vi den` svares for **biblioteket** (dørene og profilene i `$I`) og **korpuset**
(E emittert / M importert), fordi de tar hvert sitt valg. `hva det koster oss` navngir
måling, arm og celle der en finnes; ellers «ikke målt».
| nøkkel | § | hva den er til | v | bruker vi den | hvorfor / hvorfor ikke | hva det koster oss | KOMMANDO |
|---|---|---|---|---|---|---|---|
| `type` | 4.1 REQUIRED (l.163–185) | kind of concept; routing/filtering; the only always-required key; values not registered centrally | 0.1 | **bib: ja** — Dør A (manifestets `okf_type`) og B (`inbox.py` l.132) skriver; `OKF_V0_2` krever (l.1011). **korpus: 270/270** (`Krav` 269, `Normal` 1) | påkrevd; verdiene er egendefinerte som § 4.1 tillater; `verdict` er reservert av commons (l.29) | 0 avvik — 270/270 passerte § 11 (K1 S11b); ingen arm skiller på `type` | `grep -h '^type:' $M/*.md \| sort \| uniq -c` → `269 type: Krav`, `1 type: Normal` · `sed -n 1011p $P` → `required=frozenset({"type"})` |
| `title` | 4.1 recommended (l.192–193) | display name; consumers MAY derive from filename | 0.1 | **bib: ja** — A og B skriver (`inbox.py` l.133); B utleder ved fravær (`structure.py` l.58). **korpus: 270/270** = `<req_number> <seksjonstittel>` (`bundle.py` l.194) | siterbar overskrift (`VEGNORMAL_V1 § 4` l.176) | **indeksen bruker den ikke som lenketekst:** Dør C setter konsept-id som label (`importer.py` l.268), så UUID står to ganger per rad og `title` legges som fasett bak — UUID 27,7 % + `title:`-felt 16,6 % av indeksen (K1 S8b, l.173; 75 837 tegn / 270 rader). Ikke målt som celler | `grep -l '^title:' $M/*.md \| wc -l` → 270 · `grep -c '^- \[krav/N500/id-' $M/index.md` → 270 · `grep -c ' — title: ' $M/index.md` → 270 · `sed -n 268,275p $I/importer.py` |
| `description` | 4.1 recommended (l.194–196); 8 SHOULD (l.527) | one-sentence summary; «Used by `index.md` generators, search snippets, and previews» | 0.1 | **bib: ingen dør skriver den.** `STRICT_V1` navngir (l.818) og bærer den i indeksmalen (l.859); `DEFAULT`/`OKF_V0_2`-malen har ingen plass (l.797). **korpus: 0/270 (E), 0/270 (M)** | ikke skrevet; `VEGNORMAL_V1` nevner den ikke (0 av 1073 linjer, kjent-positiv `` `kravtype` `` → 8), `bundle.py` navngir den ikke (0) | **se A.3** — indeksplassen § 8 gir den er fylt med en fasettdump; oppslagsaksen taper 4/13 (F4 9/13 mot R 13/13); hypotesen om årsak er **ikke målt** | `grep -l '^description:' $M/*.md \| wc -l` → 0 · `find $E -name '*.md' -print0 \| xargs -0 grep -l '^description:' \| wc -l` → 0 · `grep -l '^title:' $M/*.md \| wc -l` → 270 (kjent-positiv) · `grep -c '\`description\`' $V` → 0 · `grep -c '"description"' ~/repos/vegnormal-okf/src/vegnormal_okf/bundle.py` → 0 |
| `resource` | 4.1 recommended (l.197–199) | URI that uniquely identifies the underlying asset; absent for abstract concepts | 0.1 | **bib: nei** — ingen dør skriver, ingen profil navngir (§ B.2-enumereringen: 35 navn, `resource` ikke blant dem). **korpus: 0** på toppnivå; den ene URI-en vi har står i `sources[].resource` | ikke skrevet; om kilde-API-et tilbyr en per-krav-URI er **ikke undersøkt** | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -c '^resource:' \| awk -F: '{s+=$NF}END{print s}'` → 0 · kjent-positiv `'^ - resource:'` → 270 · `grep -c '\`resource\`' $V` → 1 (l.383, om `sources`) |
| `tags` | 4.1 recommended (l.200–201); 3.1 (l.148–151) | YAML list of short strings; tag-browsing views are synthesized from it | 0.1 | **bib: aldri** skrevet eller lest (1 treff i 11 moduler, om HTML-tags). **korpus: 0** | ingen tverrgående kategorisering er definert utover `type`/`kravtype`/`seksjon` (`grep -c '\`tags\`' $V` → 0) | ikke målt. Porten: blokkliste passerer, flow-sekvens av skalarer avvises — likt ved 1.2.0 og 1.3.0, men med ulik begrunnelse (§ 0.7) | `grep -n tags $I/*.py` → `extract.py:86` (HTML) · `grep -l '^tags:' $M/*.md \| wc -l` → 0 · `grep -c '^tags:' $S` → 3 (spec-ens egne eksempler, kjent-positiv) |
| `sources` | 5.1 (l.287–364) | provenance: the materials a concept derives from; list of entries | 0.2 | **bib: navngitt i `OKF_V0_2`** (l.947), **skrevet av Dør A kun under den profilen** som flow-sekvens (`materialize.py` l.241–242; golden 1/1) — **som guard 1.2.0 avviste og 1.3.0 slipper gjennom** (§ 0.7). Parseren leser den ikke tilbake (`materialize.py` l.119, skalar). **korpus: E 270/270** blokk-sekvens av mappinger (`bundle.py` l.167–177); **M 0/270 — fjernet før import** (G30; falt ved 1.3.0, så grunnen til å fjerne den er borte — men M-tallet er målt på en import kjørt under 1.2.0 og er ikke re-målt) | skrives fordi § 5.1 er den konforme formen (`VEGNORMAL_V1 § 9` l.381); fjernes fordi porten avviser mapping-i-sekvens | **proveniensen finnes ikke i noe armene leste:** 0/270 i M. Etter import er `utgave` (B.1) eneste bærer av utgaven. Ikke målt som celler | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^sources:' \| wc -l` → 270 · `grep -l '^sources:' $M/*.md \| wc -l` → 0 · `sed -n 241,242p $I/materialize.py` · `grep -rl '^sources:' examples \| wc -l` → 1 |
| `sources[].resource` | 5.1 REQUIRED i oppføringen (l.303–306) | artifact a consumer can follow (URL, bundle path, `references/`) **or a scope descriptor** («all queries in project X») | 0.2 | **bib:** Dør A skriver `resource: fixture`-plassholder i golden (l.184–). **korpus: E 270/270** — én absolutt URL til kilde-API-et, samme for alle (K1 N13) | peker tilbake til kilden | ikke målt (0 i M, se `sources`) | `find $E -name '*.md' -print0 \| xargs -0 grep -h '^ - resource:' \| sort -u \| wc -l` → 1 · `… \| grep -c '^ - resource: https://'` → 270 |
| `sources[].id` | 5.1 SHOULD når kroppen siterer (l.307–309) | stable key for per-claim attribution via `[^id]` footnotes (l.348–364) | 0.2 | **bib:** golden skriver `id: golden-v0-2-sales`. **korpus: 0** — ingen kropp siterer med fotnote | ikke utløst (K1 N10: 0 `[^`) | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -c '^ id:' \| awk -F: '{s+=$NF}END{print s}'` → 0 · kjent-positiv `'^ title:'` → 270 · `grep -l '\[\^' $M/*.md \| wc -l` → 0 |
| `sources[].title` | 5.1 optional (l.310–311) | human-readable label for the source | 0.2 | **bib: nei.** **korpus: E 270/270** = `productTitle` (`bundle.py` l.177) | kildens navn | **dupliserer `utgave` 270/270 byte for byte** (B.1) — etter G30 er duplikatet det som overlever | `n=0; m=0; for f in $(find $E -name '*.md'); do u=$(grep -m1 '^utgave:' "$f" \| sed 's/^utgave: //'); s=$(grep -m1 '^ title:' "$f" \| sed 's/^ title: //'); m=$((m+1)); [ -n "$u" ] && [ "$u" = "$s" ] && n=$((n+1)); done; echo "$n av $m"` → `270 av 270` |
| `sources[].author` | 5.1 credibility signal (l.319–320); § 7 actor form | who/what produced the source — an authority signal | 0.2 | **bib: nei** (`materialize.py` l.188: «a manifest source has no author»). **korpus: 0** | ikke skrevet; kilden er et forvaltningsorgan, ikke en aktør i § 7-form — **ikke vurdert** | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -c '^ author:' \| awk -F: '{s+=$NF}END{print s}'` → 0 |
| `sources[].usage_count` | 5.1 credibility signal (l.321–326) | how often `resource` was exercised over `usage_window` — liveness | 0.2 | **bib: nei. korpus: 0** | ingen brukstall finnes for kilden | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -c 'usage_count' \| awk -F: '{s+=$NF}END{print s}'` → 0 |
| `sources[].last_modified` | 5.1 credibility signal (l.327–329); tidsnøkkel | when the source itself last changed — recency, distinct from `generated.at` | 0.2 | **bib: nei** (l.188). **korpus: 0** — den nærmeste egne nøkkelen er `kravdato` (dato-only, per krav, ikke per kilde) | om kilde-API-et bærer en endringsdato per produkt er **ikke undersøkt** | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -c 'last_modified' \| awk -F: '{s+=$NF}END{print s}'` → 0 · `grep -c 'last_modified' $P` → 0 |
| `usage_window` | 5.1 sibling of `sources` (l.330–334); tidsnøkkel | `{ from, to }` datetime range framing every `usage_count` | 0.2 | **bib: nei. korpus: 0** | ikke utløst uten `usage_count` | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -c 'usage_window' \| awk -F: '{s+=$NF}END{print s}'` → 0 · `grep -c usage_window $P` → 0 |
| `generated` | 5.2 (l.366–380); `by` REQUIRED (l.377) | how the current content was produced: `{ by: <actor>, at: <datetime> }`; supersedes v0.1 `timestamp` (13.1) | 0.2 | **bib: ja, i TO former.** Dør A: O2-mappingen `{ by: process:okf-ingest, at: <ingested_at> }` (commons § 7 l.217; `$P` l.805; 5/5 goldens). Dør B: literalen `true` (`inbox.py` l.143) — **ingen `by`, malformert etter § 5.2**; eierskapet leser begge (`OwnershipPolicy.owns`, l.317–323). **korpus: 0/270** — Dør C skriver verbatim, emitteren skriver den ikke | A: commons ratifiserte O2 (V1, `54e0ec7`). B: Phase 2-literalen beholdes for ikke å gjøre alt Dør B har skrevet eierløst (kommentar l.137–141). Korpus: `trust_tier` valgt i stedet (B.1) | for en spec-konsument har korpusets 270 konsepter **ingen produsent** — ikke skillbare fra håndskrevne (§ 1 «honesty»). Dør B-bundler bærer en `generated` ingen v0.2-leser kan lese `by` av — hva upstreams leser gjør med `'true'` er **ikke målt** | `sed -n 143p $I/inbox.py` → `"generated": "true",` · `grep -rh '^generated:' examples \| sort -u` → `generated: { by: process:okf-ingest, at: 2026-07-16T12:00:00Z }` · `grep -l '^generated:' $M/*.md \| wc -l` → 0 · `sed -n 377p $S` → «`generated.by`: REQUIRED within `generated`» |
| `verified` | 5.2 (l.383–398); 5.3 (l.401–410) | verification events `[{ by, at }]`; **trust tier is derived from it**, never stored | 0.2 | **bib: navngitt** (l.948), ingen dør skriver; parseren kan ikke lese blokklisten (K1 N16; `materialize.py` l.119). **korpus: 0/270** — i stedet en lagret `trust_tier: unverified` 270/270 (B.1) | `VEGNORMAL_V1 § 8` (l.362): ingen automatikk setter «verified»; formen ble en egen nøkkel, ikke § 5.2-listen | **U6:** `hjemmel`/`fraviksmyndighet` er menneskekuratert (`§ 5` l.223) men ingen `verified` bærer det — 270 konsepter der det kuraterte feltet står som `unverified` med resten. Ikke målt på noen arm (armene leser ikke `trust_tier`, K1 N32) | `grep -l '^verified:' $M/*.md \| wc -l` → 0 · `grep -h '^trust_tier:' $M/*.md \| sort \| uniq -c` → `270 trust_tier: unverified` · `sed -n 403,407p $S` |
| `status` | 5.4 (l.412–422) | `draft \| stable \| deprecated`; **absent ⇒ `stable`** | 0.2 | **bib: navngitt** i `OKF_V0_2` (l.949) og som fasett i `STRUCTURED_V1` (l.885); ingen dør skriver den selv. **korpus: 270/270 `stable`** (`bundle.py` l.34, `VEGNORMAL_V1 § 6` l.274); over hele build 884 `deprecated` / 6 209 `stable` (K1 S5c) | verdiene er i enumet; mappingen er kildens publiseringsstatus | 0 avvik. Indeksen bærer `; status: stable` på 270/270 rader = **4 590 av 77 611 bytes (5,9 %)** for verdien § 5.4 gjør til default ved fravær. Ikke målt som celler | `grep -h '^status:' $M/*.md \| sort \| uniq -c` → `270 status: stable` · `grep -o '; status: stable' $M/index.md \| wc -c` → 4590 · `wc -c < $M/index.md` → 77611 · `sed -n 422p $S` → «Absent `status` ⇒ `stable`.» |
| `stale_after` | 5.5 (l.424–435); tidsnøkkel | absolute instant; stale when `now >= stale_after` | 0.2 | **bib: navngitt** (l.950), ikke skrevet. **korpus: 0** | normalene har kildens gyldighet (`kravdato`, `utgave`), som ikke er en `stale_after` (K1 S5d) | ikke målt | `grep -l '^stale_after:' $M/*.md \| wc -l` → 0 · `grep -c '"stale_after"' $P` → 1 (l.950) |
### A.2 Nøkler SPEC navngir utenfor § 4–5 som `profiles.py` også navngir
Ikke blant de 17 (ordren avgrenset til § 4 og § 5), men de er i bibliotekets navnerom
og hører derfor i inventaret — 6 rader, alle 0 i korpuset.
| nøkkel | § | hva | bruker vi den | kostnad | KOMMANDO |
|---|---|---|---|---|---|
| `runtime` | 10.2 REQUIRED for `Attested Computation` (l.588–591) | how to run the computation; defines what `parameters` mean | **bib: navngitt** (l.942), typebetinget påkrevd (l.1017); ikke skrevet. **korpus: 0** — et krav bærer ingen sanksjonert beregning (K1 N27) | ikke relevant | `sed -n 942p $P` · `sed -n 1017p $P` · `grep -l '^type: Attested Computation' $M/*.md \| wc -l` → 0 |
| `parameters` | 10.2 (l.592–594) | typed, named holes `{ name, type, required }` | som over | ikke relevant | `grep -c '^parameters:' $M/*.md \| awk -F: '{s+=$NF}END{print s}'` → 0 |
| `computation` | 10.2 (l.595–597); stifelt 6.2 | path to the computation file; absent ⇒ body fence | som over | ikke relevant | `grep -c '^computation:' $M/*.md \| awk -F: '{s+=$NF}END{print s}'` → 0 |
| `executor` | 10.2 (l.598–602) | `{ resource, receipt }` — how it is run, what a run returns | som over; importøren rapporterer uløste pekere (`UnverifiedReference`, K1 N32) | ikke relevant | `grep -c '^executor:' $M/*.md \| awk -F: '{s+=$NF}END{print s}'` → 0 |
| `attester` | 10.2 (l.603–605) | deterministic check `{ resource }` | som over | ikke relevant | `grep -c '^attester:' $M/*.md \| awk -F: '{s+=$NF}END{print s}'` → 0 |
| `okf_version` | 8 (l.512–513) MAY; 12 (l.776–778) | `okf_version: "0.2"` in the **root** `index.md` frontmatter — the only frontmatter an index may carry | **bib: ja** — `OKF_V0_2` navngir plassen (l.1023), `STRICT_V1` krever den (l.867); verdien er kallerens (D5/E1). Golden skriver `okf_version: 0.2` **usitert** (spec-eksempelet er `"0.2"`; upstreams leser koerserer til `float`, A-E6). **korpus: M-indeksen har ingen** | ikke målt | `grep -c okf_version $M/index.md` → 0 · `sed -n 1,3p examples/ingest-golden-okf-v0-2/expected-bundle/index.md` → `---` / `okf_version: 0.2` / `---` · `sed -n 776,778p $S` |
### A.3 `description` — nøkkelen med egen underseksjon
**1. Fraværet, med nevner og kjent-positiv.**
```
grep -l '^description:' $M/*.md | wc -l # -> 0 (nevner 271, hvorav 270 konsepter)
find $E -name '*.md' -print0 | xargs -0 grep -l '^description:' | wc -l # -> 0 (nevner 270)
grep -l '^title:' $M/*.md | wc -l # -> 270 (kjent-positiv: samme spørring finner)
grep -c '`description`' $V # -> 0 (profilen nevner den ikke; kjent-positiv `kravtype` -> 8)
grep -c '"description"' ~/repos/vegnormal-okf/src/vegnormal_okf/bundle.py # -> 0 (emitteren navngir den ikke)
```
Nøkkelen er **ikke valgt bort — den er aldri vurdert**: verken kontrakten eller
emitteren nevner den. Det er en annen tilstand enn «forkastet».
**2. Hva spec sier den er til.** `sed -n 194,196p $S` → «A single sentence summarizing
the concept. Used by `index.md` generators, search snippets, and previews.» `sed -n 527p $S`
→ «Entries SHOULD include the description from the linked concept's frontmatter.»
**Den er den eneste nøkkelen § 8 navngir som innhold i en indeksoppføring.**
**3. Hva upstream gjør (K3, allerede målt — sitert, ikke re-målt).** `sed -n 177p $K3`
(rad A4d): `description` til stede i **9/10, 9/9, 9/9, 26/26** av de fire
referansebundlenes konsepter, mot **0/270** hos oss. `sed -n 150p $K3` (A3e): upstreams
indeks-beskrivelse er frontmatterens `description` **ordrett** for de genererte bundlene.
**4. Hva vår indeks bærer i stedet.** Plassen § 8 gir `description` er fylt med en
serialisert frontmatter-dump:
```
head -1 $M/index.md | cut -c1-120
# -> - [krav/N500/id-0000b3ff-…](import-krav-n500-id-0000b3ff-….md) — title: …; req_number: …; seksjon: …; seksjonstittel: …; kravtype: …; status: …
grep -c ' — title: ' $M/index.md # -> 270
grep -oE '(^|; )[a-z_]+: ' $M/index.md | sed 's/^; //' | sort | uniq -c
# -> 269 kravtype · 269 req_number · 269 seksjon · 269 seksjonstittel · 270 status · 270 title (og 0 description)
```
K1 S8b (`sed -n 173p $K1`) målte formen: 75 837 tegn / 270 rader; UUID 27,7 %,
`title:`-felt 16,6 %, fasettnavn 16,0 %; § 8-formen `* [Title](fil) - ` **uten**
description ville vært 37,9 % av dagens størrelse. Hva en `description` ville lagt
til, er ikke målt — ingen finnes å måle.
**5. Armene — oppslag skilt fra relasjon (R l.780–787, sitert):**
| arm | relasjon (13) | oppslag (13) | T4 (3) | sum | tegn/celle (17-nevner) | kommando |
|---|---:|---:|---:|---:|---:|---|
| R flatt BM25-søk | 0/13 | **13/13** | — | 15/29 | 3 209 (0,4 % av V) | `sed -n 28p $R` · `sed -n 552,554p $R` |
| G GraphRAG | 4/13 | **13/13** | 3/3 | 20/29 | 162 654 | `sed -n 787p $R` |
| F simulert fasett | 13/13 | 9/13 | 3/3 | 25/29 | 80 627 | `sed -n 782p $R` |
| F2 ekte Dør C, `STRUCTURED_V1`-fasetter (indeks = `title` + `status`) | 13/13 | 8/13 | 2/3 | 23/29 | 103 706 | `sed -n 783p $R` · `grep -oE '(^\|; )[a-z_]+: ' $B/F2-n500-2024-importert/index.md \| sort -u` → `status`, `title` |
| F3 ekte Dør C, korpusets nøkler | 13/13 | 7/13 | 2/3 | 22/29 | 162 820 | `sed -n 784p $R` |
| **F4** = F3 sortert på `req_number` | 13/13 | **9/13** | 3/3 | 25/29 | 162 740 (22,6 %) | `sed -n 785p $R` · `sed -n 35p $R` |
**Oppslag: R 13/13 og G 13/13 mot F4 9/13** — det er de 4 cellene «den svake
oppslagsaksen» består av. Relasjon: 13/13 i alle OKF-armer, 0/13 i R.
**6. Hypotesen, som hypotese.** «0 `description` er den enkleste forklaringen på at
oppslag taper» er **ikke målt**: ingen arm har lest en indeks med `description` (K1 S8c:
«ikke målt»). Og det finnes en **målt konkurrerende forklaring** i samme dokument:
`sed -n 545,548p $R` — G tar 13/13 på oppslag fordi den bærer `requirement_level` per
seksjon fra kilden (T7 veiledningsfelle 4/4 mot 2/4; T8 mutert hypotese 5/5 mot 3/5) —
det er **modalitet (U2)**, ikke en beskrivelse. Begge står åpne. Fase 2 velger hva som
måles; dette dokumentet gjør det ikke.
**7. Porten.** En énlinjes `description` **passerer** (målt ved både 1.2.0 og 1.3.0,
§ 0.7); en brettet
avvises. G33s «29/53 brettet» (`~/.claude/docs/okf-bundle-prosessen.md` l.130) gjelder
upstreams bundler, ikke en verdi vi selv ville skrevet.
**8. Biblioteket.** Selv med `description` i frontmatter ville Dør C under `DEFAULT`
ikke løftet den inn i indeksen: `sed -n 797p $P` → `link_template="- [{label}]({target})"`
(ingen plass); `STRUCTURED_V1` sine fasettnøkler navngir den ikke
(`sed -n 882,891p $P`); bare `STRICT_V1` bærer den (`sed -n 859p $P` →
`* [{label}]({target}) - {description}`). **Spaken ligger i profilvalget, som S8c fant
for § 8-formen.**
---
## Del B — de egne nøklene
### B.0 Klassene, og hvordan «spec har den alt» er tolket
Ordren gir tre klasser og krever nøyaktig én per nøkkel. Klasse (1) «en OKF-nøkkel under
et annet navn» er brukt slik: **spec spesifiserer en plass for den samme informasjonen
— en nøkkel ELLER en strukturell plass** (kroppslenke § 6.1, fotnote → `sources[].id`
§ 5.1, seksjonsoverskrift § 8, utledning § 5.3). Uten den lesningen ville ingen av
relasjonsnøklene kunne klassifiseres i det hele tatt, for § 6.1 har ingen nøkkel — og det
er nettopp den klassen som er dyrest: en konsument som leter på spec-plassen finner
ingenting. Tolkningen står her så klassifiseringen kan etterprøves, ikke bare leses.
`dupliserer` navngir det som **allerede finnes** et annet sted i samme dokument eller
bundle, målt. `kostnad` navngir måling/arm/celle eller «ikke målt».
### B.1 Korpusets 33 nøkler (+ 7 i andre bundler) — hentet av kommandoen i § 0.3
Tellinger: `M`/`E` = filer i den importerte/emitterte N500-bundlen (nevner 270 konsepter);
`build` = filer over hele `build/` (nevner 48 983). Kilde-linjer i `bundle.py` (`bundle.py`
= `~/repos/vegnormal-okf/src/vegnormal_okf/bundle.py`), `lenker.py`, `gren.py` samme katalog.
| nøkkel | M / E / build | hva den bærer | klasse | begrunnelse — spec-plass (1) / hvorfor spec ikke bærer den (2) / hvorfor påfunn (3) | dupliserer | kostnad | KOMMANDO |
|---|---|---|:---:|---|---|---|---|
| `kravtype` | 269 / 269 / 48 355 | normativ modalitet `skal\|bør\|kan`, kildens `vocab-term` totalt mappet, aldri utledet (`VEGNORMAL_V1 § 4.1` l.194; `bundle.py` l.31) | **(2)** | spec bærer ikke modalitet: K1 § M — 1 treff på `modal\|normativ\|shall\|…` i 1006 linjer, og det om spec-ens egen tekst (l.701); `RECOMMENDED`/`OPTIONAL` 0. Et normativt korpus uten skillet krav/anbefaling er verre enn ingen bundle (U2). Verdien er kildens, ikke vår | kroppens `## Krav` / `## Veiledning (ikke-normativ)` (269/171) bærer skillet strukturelt (K1 M2) | **indeksraden** bærer `kravtype:` 269/270 (K1 M3) — det eneste stedet en konsument ser modaliteten før den åpner filen, og det **forsvinner ved § 8-formen** med mindre `description` bærer den. G tar T7 4/4 på nettopp modalitet mot OKF 2/4 (`sed -n 545,548p $R`) | `grep -h '^kravtype:' $M/*.md \| sort \| uniq -c` → `269 kravtype: skal` · `grep -c 'kravtype: ' $M/index.md` → 269 · `sed -n 31p bundle.py` |
| `normal` | 270 / 270 / 48 983 | normalens referanse, `meta.standardReference` (`N500`) (`§ 4` l.178; `bundle.py` l.199, 285) | **(3)** | ikke spec-forankret, og informasjonen finnes alt: konsept-id-en ER stien (§ 2 l.78–79) og katalogsegmentet er normalen (`krav/N500/…`, `VEGNORMAL_V1 § 3` l.95–99); etter Dør C står den i filnavnet (`import-krav-n500-…`). Ingen arm eller indeks leser den (0 i indeksen) | **konsept-id-ens katalogsegment 269/269** (`krav/N500/` ↔ `normal: N500`) | ikke målt | `find $E/krav -name '*.md' -print0 \| xargs -0 grep -h '^normal:' \| sort \| uniq -c` → `269 normal: N500` · `find $E/krav -type d` → `krav/N500` · `grep -c 'normal: ' $M/index.md` → 0 |
| `utgave` | 270 / 270 / 48 983 | utgavetittel `productTitle` (`N500:2024`) (`§ 4` l.180; `bundle.py` l.200, 286) | **(1)** | **`sources[].title`, § 5.1 l.310** — samme verdi skrives to steder i samme dokument (`bundle.py` l.177 og l.200), 270/270 byte-identisk | **`sources[].title` 270/270** | etter G30 er `utgave` **eneste** bærer av utgaven i M (`sources` 0/270) — duplikatet er i dag det som overlever porten. Ikke målt som celler | kommandoen i Del A `sources[].title` → `270 av 270` · `grep -l '^utgave:' $M/*.md \| wc -l` → 270 · `grep -l '^sources:' $M/*.md \| wc -l` → 0 |
| `req_number` | 269 / 269 / 48 355 | kravets nummer i kilden, `a["req-number"]` — **posisjon, ikke identitet** (`§ 4` l.181; `§ 3` l.107–117) | **(2)** | spec har ingen nøkkel for et dokumentnummer; nærmeste er `title` (fri tekst) og #10s ledende konsept (katalognavn, ikke nummer). Korpuset siterer krav ved nummer, og **rekkefølgen er en målt ytelsesegenskap:** F3 → F4 (samme indeks sortert på `req_number`) = 12/17 → 14/17, 22/29 → 25/29 (`sed -n 35p $R`, `sed -n 784,785p $R`). Egen ordre `…3790625395`, ikke slått sammen her | **`title` begynner med `req_number` 269/269** (`bundle.py` l.194: `f"{piece.req_number} {piece.section_title}"`) | indeksraden bærer den 269/270 som fasett; `title:`-feltet (16,6 %, K1 S8b) bærer den én gang til | `n=0; m=0; for f in $(find $E/krav -name '*.md'); do t=$(grep -m1 '^title:' "$f" \| sed "s/^title: //; s/^'//; s/'\$//"); r=$(grep -m1 '^req_number:' "$f" \| sed "s/^req_number: //; s/^'//; s/'\$//"); m=$((m+1)); case "$t" in "$r"*) n=$((n+1));; esac; done; echo "$n av $m"` → `269 av 269` · `grep -c 'req_number: ' $M/index.md` → 269 |
| `kravdato` | 269 / 269 / 48 355 | kravets `valid-from`, ISO-dato UTC; følger kravet, ikke utgaven (`§ 4.2` l.214–220) | **(2)** | spec har ingen «gyldig fra»: `generated.at` er innholdets endring, `sources[].last_modified` er kildens endring, `stale_after` er utløp — ingen er ikrafttredelse. Dato-only-formen er lovlig fordi nøkkelen er vår (§ 0.6) | ingen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -h '^kravdato:' \| grep -cE "^kravdato: '?[0-9]{4}-[0-9]{2}-[0-9]{2}'?$"` → 269 · `sed -n 214,220p $V` |
| `hjemmel` | 270 / 270 / 48 983 | rettslig hjemmel, **kuratert av et menneske per normal**, arvet ned på hvert krav (`§ 5` l.223–257) | **(2)** | ingen OKF-analogi (U4): spec kjenner `resource` (asset) og `sources` (materiale), ikke rettsgrunnlag. Kuratert fordi kilden ikke bærer den strukturert (attributtlisten l.233–236) | ingen | **U6:** kuratert verdi uten `verified` — 270 konsepter der det menneskeleste feltet står som `unverified` (K1 § U). Ikke målt som celler | `grep -l '^hjemmel:' $M/*.md \| wc -l` → 270 · `grep -h '^trust_tier:' $M/*.md \| sort -u` → `trust_tier: unverified` · `sed -n 223,230p $V` |
| `fraviksmyndighet` | 270 / 270 / 48 983 | hvem som kan gi fravik; kuratert; fast streng `ikke uttalt i kilden` der kilden er taus (`§ 5` l.259–272) | **(2)** | som `hjemmel` (U4). Et **erklært fravær** er en verdi spec ikke har noen plass for (U5/U8-seamen) | ingen | som `hjemmel`; ikke målt | `grep -l '^fraviksmyndighet:' $M/*.md \| wc -l` → 270 · `grep -h '^fraviksmyndighet:' $M/*.md \| sort \| uniq -c \| wc -l` → antall distinkte verdier · `sed -n 259,272p $V` |
| `hjemmel_sitat` | 1 / 1 / 628 | sitatet fra kilden `hjemmel` er lest av, som belegg (`§ 5` l.244–246; `bundle.py` l.288) — kun på `Normal` | **(3)** | innholdet er nødvendig (etterprøvbarhet); **nøkkelen** er det ikke, og den er ikke spec-forankret: spec-ens plass for «belegget for en påstand» er kroppen med `[^id]`-fotnote til en `sources`-oppføring (§ 5.1 l.348–364). Et sitat er prosa, og frontmatter-skalaren kan ikke bære et lengre sitat uten brett (§ 0.7) | ingen (verdien er unik) | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^hjemmel_sitat:'` → `normal/N500.md` (1) · `sed -n 348,352p $S` |
| `fraviksmyndighet_sitat` | 1 / 1 / 628 | som over, for `fraviksmyndighet` (`bundle.py` l.290) | **(3)** | som `hjemmel_sitat` | ingen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^fraviksmyndighet_sitat:' \| wc -l` → 1 |
| `trust_tier` | 270 / 270 / 48 983 | lagret tillitsnivå `unverified\|verified` (`§ 8` l.362–379; `bundle.py` l.56, 206) | **(1)** | **§ 5.3 l.401–407: tieren UTLEDES av `verified`** — «No `verified` key ⇒ unverified». En lagret `unverified` sier det en spec-konsument utleder av fraværet uansett; vokabularet er 2 verdier mot spec-ens 3 tiers; og den dagen et menneske signerer, er spec-plassen `verified: [{ by: human:<id>, at }]` (§ 7 l.500, K1 N20), ikke denne nøkkelen. K1 S5b fant det samme | **fraværet av `verified`** (270/270 bærer `unverified` og 0 bærer `verified`) | 0 informasjon for en spec-konsument; armene leser den ikke (K1 N32: 0 filer). U6 hviler på den | `grep -h '^trust_tier:' $M/*.md \| sort \| uniq -c` → `270 trust_tier: unverified` · `grep -l '^verified:' $M/*.md \| wc -l` → 0 · `sed -n 403,407p $S` |
| `seksjon` | 269 / 269 / 48 355 | seksjonsnummeret kravet står i, fra forfedrene i treet (`§ 4` l.187; `bundle.py` l.207) | **(2)** | spec har ingen nøkkel for plass i et hierarki; dens bærer er katalogtreet + § 8-indeks per katalog (+ #10s ledende konsept). Vår bundle har **1 katalog** etter import (K1 N21), så treet bærer ingenting, og nummeret finnes ingen andre steder. Grenformen partisjonerer på den (`VEGNORMAL_V1 § 16`) | ingen (nummeret) | fasett i indeksen 269/270; N101: 7/352 krav uten seksjon (kildens vedlegg, `2026-08-25-grenformen-og-tallene.md § 4.1`) | `grep -c 'seksjon: ' $M/index.md` → 269 · `find $M -type d \| wc -l` → 1 · `grep -l '^seksjon:' $M/*.md \| wc -l` → 269 |
| `seksjonstittel` | 269 / 269 / 48 355 | seksjonens overskrift (`§ 4` l.188; `bundle.py` l.208) | **(1)** | **§ 8 «Section / Group Heading» (l.517)** er spec-plassen for en seksjonstittel som grupperer konsepter — og vår indeks har 0 overskrifter (K1 S8a). Dessuten er verdien alt i `title` | **`title` = `req_number` + ` ` + `seksjonstittel` 269/269** (`bundle.py` l.194) — halen av tittelen er nøkkelen | fasett i indeksen 269/270 (del av fasettnavnenes 16,0 %, K1 S8b) — verdien står altså **to ganger per indeksrad** (i `title:` og i `seksjonstittel:`) | `grep -c 'seksjonstittel: ' $M/index.md` → 269 · `grep -c '^#' $M/index.md` → 0 · `sed -n 194p bundle.py` |
| `lenker` | 14 / 14 / 1 140 | blokkliste av konsept-id-er kravet viser til (`statement`-xref) — eneste klasse som peker på noe bundlen bærer (`§ 13.2` l.559; `lenker.py` l.37) | **(1)** | **§ 6.1 l.439–466: markdown-lenke i kroppen.** 17 unike mål, alle løser til en fil (K1 S6a); 0 lenker i noen kropp (K1 N18). En konsument som bygger graf av § 6.1-kanter ser 0 kanter | ingen — men **`viser_til_krav` bærer samme referanser som tekst** (`lenker.py` l.297–299: begge skrives for hver `statement`) | **indeksen: 0/270** (K1 N6, Funn 1 i `STATE`: blokklister blir tom streng i `parse_frontmatter`). Relasjonsspørsmål: 13/13 i F, F2 **og** F3 (`sed -n 624,625p $R`) — F2-indeksen bar bare `title` + `status`, så relasjonene ble svart fra filene, ikke fra nøkkelen i indeksen | `grep -l '^lenker:' $M/*.md \| wc -l` → 14 · `grep -c 'lenker' $M/index.md` → 0 · `sed -n 297,299p lenker.py` |
| `viser_til_krav` | 14 / 14 / 1 140 | kildens tekst for `statement`-referanser (`§ 13.1` l.531–557; `lenker.py` l.41) | **(1)** | § 6.1-lenke i kroppen, med lenketeksten som «kildens tekst» — nøyaktig § 6.1s form `[tekst](/sti)`. K3 målte at upstream bærer arten i overskriften + fotnote (idéboka U1) | **`lenker`** — samme referanser, id der og tekst her (`lenker.py` l.297–299) | som `lenker` | `grep -l '^viser_til_krav:' $M/*.md \| wc -l` → 14 · `diff <(grep -l '^lenker:' $M/*.md) <(grep -l '^viser_til_krav:' $M/*.md) \| wc -l` → 0 (samme 14 filer) |
| `viser_til_kapittel` | 24 / 24 / 5 609 | seksjonsoverskrift for `sec`-referanser (`lenker.py` l.42) | **(1)** | § 6.1: en lenke til seksjonen — som **ikke er et konsept** (`VEGNORMAL_V1 § 2` l.72–88), så målet finnes ikke i bundlen; § 6.1 l.464 tillater brutte lenker eksplisitt («not-yet-written knowledge»). Grenformens seksjonsbaser er det nærmeste målet | ingen | indeksen 0; relasjon 13/13 uavhengig (se `lenker`) | `grep -l '^viser_til_kapittel:' $M/*.md \| wc -l` → 24 · `sed -n 464,466p $S` |
| `viser_til_tabell` | 31 / 31 / 5 362 | kildens tekst for `table`-referanser (`lenker.py` l.43) | **(1)** | § 6.1-lenke, eller ren prosa når tabellen ikke er et mål (tabeller bæres i kroppen som HTML, `§ 14.3`) | ingen | indeksen 0 | `grep -l '^viser_til_tabell:' $M/*.md \| wc -l` → 31 |
| `viser_til_figur` | 23 / 23 / 3 451 | figurens bildetekst for `fig`-referanser (`lenker.py` l.44) | **(1)** | § 6.1-lenke — til noe bundlen **ikke bærer** (`§ 14` l.598: figurer erklært, ikke båret); en brutt lenke er lovlig (l.464) | figurinventaret i `Normal`-kroppen navngir samme figur (`bundle.py` `_figure_inventory`) | indeksen 0 | `grep -l '^viser_til_figur:' $M/*.md \| wc -l` → 23 |
| `viser_til_referanse` | 97 / 97 / 5 707 | kildens tekst for `bibr` — litteraturreferanser (`lenker.py` l.45) | **(1)** | **§ 5.1 `sources` + `[^id]`-fotnote (l.348–364)** er spec-ens plass for en litteraturreferanse per påstand. Vi har 0 fotnoter og én `sources`-oppføring (kilde-API-et) | ingen | indeksen 0. Den mest brukte relasjonsnøkkelen (97 filer) står på en plass ingen spec-konsument leser | `grep -l '^viser_til_referanse:' $M/*.md \| wc -l` → 97 · `grep -l '\[\^' $M/*.md \| wc -l` → 0 |
| `viser_til_tillegg` | 3 / 3 / 74 | kildens tekst for `app`-referanser (`lenker.py` l.46) | **(1)** | § 6.1-lenke/prosa, som `viser_til_kapittel` | ingen | indeksen 0 | `grep -l '^viser_til_tillegg:' $M/*.md \| wc -l` → 3 |
| `viser_til_ekstern` | 2 / 2 / 2 950 | `ext-link`-URL-er ordrett, inkl. kryss-normal (`§ 13.3` l.569–573; `lenker.py` l.38) | **(1)** | **§ 6.1 absolutt URL i kroppen** («an absolute URL (for example `https://...`)», § 6.2 l.473) — nøyaktig formen; kryss-normal blir aldri `lenker` fordi målet er en annen bundle | ingen | indeksen 0 | `grep -l '^viser_til_ekstern:' $M/*.md \| wc -l` → 2 |
| `viser_til_formel` | 1 / 1 / 203 | kildens tekst for `disp-formula` (`lenker.py` l.49) | **(1)** | § 6.1-lenke/prosa; formler er kildens prosa, ikke `computation` (K1 N27) | ingen | indeksen 0 | `grep -l '^viser_til_formel:' $M/*.md \| wc -l` → 1 |
| `viser_til_tabellnote` | 0 / 0 / 1 000 | kildens tekst for `table-fn` (`lenker.py` l.47) — **ikke i N500** | **(1)** | som `viser_til_tabell` | ingen | ikke målt (0 i M) | `find $B -name '*.md' ! -name index.md -print0 \| xargs -0 grep -l '^viser_til_tabellnote:' \| wc -l` → 1000 · `grep -l '^viser_til_tabellnote:' $M/*.md \| wc -l` → 0 |
| `krav_i_kilden` | 1 / 1 / 628 | kildens eget kravtall (`requirementTypeCounts` summert) — **skal** være lik `krav_i_bundlen`, ellers bygges ingen bundle (`§ 7` l.291–311; `bundle.py` l.293) | **(2)** | spec har ingen representasjon av **dekning/fullstendighet** (`grep -n -i 'coverage\|completeness\|complete' $S` → 2 treff, begge om «completed order»/«incomplete» i eksempler; kjent-positiv `frontmatter` → 28). Uten erklæringen leses en bundle som fullstendig, og det er den ikke (l.293–295). Negativ tilstand — U5-seamen | kroppens `## Dekning` sier det samme i prosa (`bundle.py` l.313: «Bundlen bærer {built} krav av {declared}») | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -h '^krav_i_kilden:\|^krav_i_bundlen:'` → to like tall · `sed -n 291,311p $V` |
| `krav_i_bundlen` | 1 / 1 / 628 | krav profilen faktisk skrev (`bundle.py` l.294); omskrevet i grenbaser (`§ 16.3` l.1021) | **(2)** | som `krav_i_kilden` — de to er ett invariant | `## Dekning`-prosaen | ikke målt | samme kommando · `sed -n 1021,1030p $V` |
| `tabeller_i_krav` | 1 / 1 / 625 | tabeller båret strukturelt inne i krav (`§ 7` l.328–337) | **(2)** | dekningserklæring; spec har ingen | `## Dekning`-prosaen (`bundle.py` l.316–319) | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^tabeller_i_krav:' \| wc -l` → 1 |
| `tabeller_uten_label` | 1 / 1 / 625 | bårne tabeller uten label i kilden | **(2)** | dekningserklæring | ingen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^tabeller_uten_label:' \| wc -l` → 1 |
| `tabeller_utelatt` | 1 / 1 / 628 | tabeller i kilden bundlen ikke bærer (`§ 7` l.302) | **(2)** | dekningserklæring — «det bundlen IKKE bærer» har ingen spec-plass (U5) | `## Dekning`-prosaen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^tabeller_utelatt:' \| wc -l` → 1 |
| `figurer_i_krav` | 1 / 1 / 625 | figurer navngitt inne i krav (`§ 14.3`) | **(2)** | dekningserklæring | figurinventaret i kroppen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^figurer_i_krav:' \| wc -l` → 1 |
| `figurer_utelatt` | 1 / 1 / 628 | figurer i kilden som ikke følger med (`§ 7` l.303, `§ 14`) | **(2)** | dekningserklæring | `## Dekning`-prosaen + inventartabellen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^figurer_utelatt:' \| wc -l` → 1 |
| `figurer_uten_tittel` | 1 / 1 / 625 | figurer uten bildetekst (`§ 14.2`) | **(2)** | dekningserklæring | ingen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^figurer_uten_tittel:' \| wc -l` → 1 |
| `bilder_uten_figurnummer` | 1 / 1 / 625 | bilder utenfor enhver figur (`bundle.py` l.247–253) | **(2)** | dekningserklæring | inventartabellens `### Bilder uten figurnummer` | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^bilder_uten_figurnummer:' \| wc -l` → 1 |
| `lenker_utelatt` | 1 / 1 / 625 | xref i prosa mellom kravene, tapt med prosaen (`§ 13.5` l.590–595) | **(2)** | dekningserklæring; 1 169 av 3 161 xref over korpuset står utenfor krav | `## Dekning`-prosaen (`bundle.py` l.326–327) | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^lenker_utelatt:' \| wc -l` → 1 · `sed -n 590,595p $V` |
| `lenker_slaatt_sammen` | 1 / 1 / 625 | referanser samme krav siterte to ganger, skrevet én gang (`§ 13.4`; `bundle.py` `merged`) | **(2)** | dekningserklæring — forklarer differansen mellom kildens xref-tall og bundlens | ingen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -l '^lenker_slaatt_sammen:' \| wc -l` → 1 |
| `tegn_utenfor_krav` | 1 / 1 / 628 | tegn prosa som ikke står i noe krav (`§ 7` l.304; `bundle.py` l.308) | **(2)** | dekningserklæring | `## Dekning`-prosaen | ikke målt | `find $E -name '*.md' -print0 \| xargs -0 grep -h '^tegn_utenfor_krav:'` → ett tall |
| `erstatter` | 0 / 0 / 70 | forgjengerens konsept-id når et krav er omskrevet mellom utgaver — to betingelser, aldri tekstlikhet (`§ 3` l.129–168; `bundle.py` l.211) — **ikke i N500** | **(2)** | spec-ens livssyklus er `status: deprecated` på den gamle (§ 5.4) + lenke; **ingen etterfølgernøkkel**, og upstream #11 (deletion semantics) er åpen uten PR (U3). K3 B2d: upstream legger erstatteren i `description`-prosa (`sed -n 270p $K3`). Kjeden kan ikke følges maskinelt fra spec-plassen | ingen | ikke målt (0 i M); N100 2022→2023: 10 skrevet, 43 kandidater riktig avvist av betingelse 2 (`§ 3` l.150–160) | `find $B/FX-n100-2023-erstatter -name '*.md' -print0 \| xargs -0 grep -l '^erstatter:' \| wc -l` → 10 · `grep -l '^erstatter:' $M/*.md \| wc -l` → 0 · `sed -n 129,135p $V` |
| `erstattere_i_bundlen` | 0 / 0 / 75 | antall `erstatter` skrevet — kun når en forgjenger ble sendt inn (`§ 7` l.298–300; `bundle.py` l.297) | **(2)** | dekningserklæring for `erstatter` (samme betingede tilstedeværelse) | ingen | ikke målt | `find $B/FX-n100-2023-erstatter -name '*.md' -print0 \| xargs -0 grep -h '^erstattere_i_bundlen:'` → `erstattere_i_bundlen: 10` |
| `gren` | 0 / 0 / 542 | seksjonsnummeret en grenbase dekker; `kart` i kartbasen (`§ 16.3` l.1021–1030; `gren.py` l.206, 253) | **(2)** | grenformen finnes fordi konsumentens (po) `read_bundle` stopper ved basegrensen — «basegrensen er det eneste stedet OKF-navigasjonen stopper» (`2026-08-25-grenformen-og-tallene.md § 3`). Spec-ens eget svar på «åpne én del» er § 8 per katalog + #10 — som S8c fant ikke er bygget. Nøkkelen erklærer en **bevisst ufullstendig** bundle; ufullstendighet har ingen spec-plass (U5) | `krav_i_bundlen` omskrives i samme base | grenform målt: kart + dyreste gren 13 634 mot 85 937 tokens for N500 (−84,1 %); ikke målt på resonneringsarmene | `find $B -name '*.md' ! -name index.md -print0 \| xargs -0 grep -h '^gren:' \| sort \| uniq -c \| sort -rn \| head -3` · `sed -n 1021,1030p $V` |
| `gren_direkte` | 0 / 0 / 528 | `ja` når grenen bare bærer krav skrevet direkte i seksjonen (`§ 16.3`) | **(2)** | som `gren` | ingen | ikke målt | `find $B -name '*.md' ! -name index.md -print0 \| xargs -0 grep -h '^gren_direkte:' \| sort \| uniq -c` |
| `krav_i_normalen` | 0 / 0 / 528 | hele normalens kravtall, så delen kan leses mot helheten (`§ 16.3`) | **(2)** | som `gren` — dekning for en del | `krav_i_kilden` i samme base (uendret, l.1032) | ikke målt | `find $B -name '*.md' ! -name index.md -print0 \| xargs -0 grep -l '^krav_i_normalen:' \| wc -l` → 528 |
| `grener_i_normalen` | 0 / 0 / 542 | antall grener, så en konsument vet hvor mange baser som finnes (`§ 16.3`) | **(2)** | som `gren`. Merk: dette er informasjonen en § 8-rotindeks bærer ved å liste underkataloger (l.529: `* [Subdirectory](subdir/)`) | ingen | ikke målt | `find $B -name '*.md' ! -name index.md -print0 \| xargs -0 grep -l '^grener_i_normalen:' \| wc -l` → 542 |
**Fordelingen, produsert av kommando på dette dokumentet:**
```
awk -F'|' '/^\| `/ && NF>=9 && $5 ~ /\*\*\([123]\)\*\*/ {gsub(/[ *]/,"",$5); c[$5]++} END {for (k in c) print k, c[k]}' docs/okf-nokkelinventar.md | sort
# -> (1) 13 · (2) 24 · (3) 3 = 40 rader i B.1
```
Av de 33 i M: **(1) 12** (`utgave`, `trust_tier`, `seksjonstittel`, `lenker`, åtte
`viser_til_*`), **(2) 18** (`kravtype`, `req_number`, `kravdato`, `hjemmel`,
`fraviksmyndighet`, `seksjon`, tolv dekningsnøkler), **(3) 3** (`normal`,
`hjemmel_sitat`, `fraviksmyndighet_sitat`). De 7 utenfor M: (1) 1, (2) 6.
### B.2 Bibliotekets 21 nøkler utenfor spec (`profiles.py`)
Enumerert av kommando, ikke lest av:
```
.venv/bin/python -c '
from llm_ingestion_okf import profiles as p
spec = {"type","title","description","resource","tags","sources","usage_window","generated","verified","status","stale_after","runtime","parameters","computation","executor","attester","okf_version"}
allk = {}
for n in ("DEFAULT","STRICT_V1","STRUCTURED_V1","OKF_V0_2"):
for k in (*getattr(p,n).frontmatter.order, *getattr(p,n).index.root_frontmatter): allk.setdefault(k, []).append(n)
own = {k:v for k,v in allk.items() if k not in spec}
print(len(allk), "navn;", len(own), "utenfor spec")
for k,v in sorted(own.items()): print(f" {k:18s} {",".join(v)}")'
# -> 35 navn; 21 utenfor spec
```
**Eier** avgjør hvor en omdøping ville landet, og det er ikke alltid her: `DEFAULT` sine
seks er commons' ingest-spec § 7 (`~/repos/portfolio-optimiser-commons/ingest-spec.md`
l.204–217, «spec changes go via commons»); `STRICT_V1` sine åtte er wikiens ratifiserte
kontrakt (`docs/phase-3-split-table.md`, O2: «changing another repo's contract from
here violates O2»). Bare `STRUCTURED_V1` sine sju og `okf_spec_commit`/`bundle_profile`
er valg tatt i dette repoet.
| nøkkel | profil | eier | hva | klasse | spec-plass (1) / begrunnelse (2) / (3) | dupliserer | KOMMANDO |
|---|---|---|---|:---:|---|---|---|
| `source_system` | DEFAULT, STRUCTURED_V1, OKF_V0_2 | commons § 7 | manifestets `source.id` (l.213) | **(1)** | **`sources[].id`, § 5.1 l.307** — en stabil nøkkel for kilden. Golden skriver begge med samme verdi | `sources[].id` i `OKF_V0_2`-golden (`id: golden-v0-2-sales` = `source_system: golden-v0-2-sales`) | `sed -n 213p ~/repos/portfolio-optimiser-commons/ingest-spec.md` · `grep -h '^source_system:\|^sources:' examples/ingest-golden-okf-v0-2/expected-bundle/ingest-sales.md` |
| `source_query` | DEFAULT, STRUCTURED_V1, OKF_V0_2 | commons § 7 | spørringen som hentet innholdet, whitespace-kollapset (l.214) | **(1)** | **`sources[].resource` som «scope descriptor», § 5.1 l.303–306** («a population or scope descriptor … for example `all queries in BigQuery project X`») — en SQL/sti-spørring er nøyaktig det | ingen i dag (Dør A skriver `resource: fixture`-plassholder under `OKF_V0_2`, l.184–) | `sed -n 303,306p $S` · `sed -n 184,199p $I/materialize.py` |
| `source_file` | DEFAULT (Dør B) | dette repoet (Phase 2) | filnavnet som ble sluppet i innboksen (`inbox.py` l.134) | **(1)** | **`sources[].resource` som bundle-relativ sti eller `references/`-sti, § 5.1 l.303–305 + § 6.3** | ingen | `sed -n 134p $I/inbox.py` · `sed -n 479,486p $S` |
| `source_sha256` | DEFAULT (Dør B) | dette repoet | SHA-256 av kildebytene (`inbox.py` l.135) | **(2)** | spec har **ingen integritetsdigest**: `grep -n -i 'sha\|hash\|digest\|checksum' $S` → 6 treff, alle substrenger (`shaped`, `sharded`, `shared`, `distinguishable`); kjent-positiv `frontmatter` → 28. Innholdsidentitet er Dør C sitt eierskapsbevis (CLAUDE.md, Door C) | ingen | `grep -n -i 'sha\|hash\|digest\|checksum' $S \| cut -c1-60` |
| `ingested_at` | DEFAULT, STRUCTURED_V1, OKF_V0_2 | commons § 7 | det eksplisitte tidsargumentet, ordrett (l.215) | **(1)** | **`generated.at`, § 5.2 l.378** — og `DEFAULT` skriver alt begge med **samme verdi 5/5** (commons l.217: «`at` repeats the `ingested_at` value verbatim»). Dokumentet bærer tidspunktet to ganger, med vilje hos commons | **`generated.at` 5/5 goldens** | `n=0; for f in $(grep -rl '^ingested_at:' examples); do a=$(grep -m1 '^ingested_at: ' "$f" \| cut -d' ' -f2); g=$(grep -m1 -o 'at: [^ }]*' "$f" \| cut -d' ' -f2); [ "$a" = "$g" ] && n=$((n+1)); done; echo $n` → 5 |
| `ingest_manifest` | DEFAULT, STRUCTURED_V1, OKF_V0_2 | commons § 7 | manifestreferansen `{stem}@{hash16}` (l.216) — sammen med `generated.by` **eierskapsstempelet** (l.84) | **(2)** | pekeren kunne vært `sources[].resource` → `references/<manifest>` (§ 6.3 «run instructions»), men **hash16-delen** har ingen spec-plass (ingen digest, se `source_sha256`), og commons gjør nettopp hashen til eierskapsbevis fordi «the key's mere presence does not» (l.217) | delvis: pekeren, ikke hashen | `sed -n 84,86p ~/repos/portfolio-optimiser-commons/ingest-spec.md` · `grep -h '^ingest_manifest:' examples/*/expected-bundle/*.md \| head -1` |
| `number` | STRUCTURED_V1 | dette repoet | dokumentnummer, utledet av filnavn/tittel (`structure.py` l.232–237) | **(2)** | som `req_number` (B.1): spec har ingen nummer-nøkkel; rekkefølge er en målt egenskap (F4). Utledet → merket i `derived` | `title`/filstammen den ble utledet av | `sed -n 232,237p $I/structure.py` |
| `parent` | STRUCTURED_V1 | dette repoet | foreldrenummeret, utledet av `number` (l.240–241) | **(1)** | **§ 6.1 l.454–457: «parent/child» er den første relasjonsarten spec navngir**, båret av lenke + prosa; og § 3s katalogtre er den implisitte forelderen (§ 2 l.85 «implicit parent/child hierarchy») | `number` (den er en funksjon av den) | `sed -n 240,241p $I/structure.py` · `sed -n 454,457p $S` |
| `status` | STRUCTURED_V1 (fasett) | — | spec-nøkkel, projisert i indeksen | spec | (i Del A) | — | `sed -n 885p $P` |
| `date` | STRUCTURED_V1, STRICT_V1 | dette repoet / wiki | dokumentdato, deklarert av kaller; `YYYY-MM-DD` i testene (§ 0.6) | **(2)** | som `kravdato`: ingen spec-tidsnøkkel betyr «dokumentets dato» (`generated.at` = innholdsendring, `last_modified` = kildeendring) | ingen | `grep -ho 'date: [0-9][^"\\]*' tests/*.py \| sort -u` |
| `version` | STRUCTURED_V1, STRICT_V1 | dette repoet / wiki | dokumentets versjon, deklarert (l.255) | **(2)** | spec har ingen dokumentversjon (§ 12 er formatets versjon); nærmeste er `sources[].title`/`utgave`-mønsteret i korpuset | ingen | `sed -n 255p $I/structure.py` · `grep -n 'version' $S \| grep -v 'okf_version\|Version 0\|version bump\|<version>' \| wc -l` → 0 om dokumentversjon (ikke lest i detalj: treffene er formatversjon og aktørversjon) |
| `supersedes` | STRUCTURED_V1 | dette repoet | hvilke dokumenter dette erstatter, deklarert (l.257) | **(2)** | som `erstatter` (B.1): ingen etterfølgernøkkel, upstream #11 åpen (U3) | ingen | `sed -n 257p $I/structure.py` |
| `references` | STRUCTURED_V1 | dette repoet | kryssreferanser — **utledet av § 6.1-lenker i kroppen** (`_LINK`, l.54; l.246) når ikke deklarert | **(1)** | **§ 6.1 — nøkkelen er bokstavelig talt en frontmatter-kopi av kroppens markdown-lenker** (regexen `\[[^\]]*\]\(([^)\s]+)\)` l.54). Egen ordre `…507632913` (12/12 falske), ikke slått sammen her | **kroppens lenker** (kilden den utledes fra) | `sed -n 54p $I/structure.py` · `sed -n 243,248p $I/structure.py` |
| `derived` | STRUCTURED_V1 | dette repoet | hvilke av nøklene foran som ble **utledet**, ikke lest (l.898–902) | **(2)** | spec skiller ikke lest fra utledet på feltnivå — samme hull som U6 (trust per felt). «An unmarked heuristic is worse than none» (CLAUDE.md, Door B) | ingen | `sed -n 898,902p $P` |
| `timestamp` | STRICT_V1 | wiki | dokumentets tidsstempel (v0.1-nøkkelen) | **(1)** | **`generated.at` — spec sier det selv, § 13.1 l.805** («`timestamp` is superseded by `generated.at`»). Profilen nekter å navngi begge (l.42). Wikiens kontrakt; omdøping er deres kall (O2) | ingen (wikien skriver ikke `generated`) | `sed -n 805,808p $S` · `sed -n 819p $P` · `sed -n 42p $P` |
| `layer` | STRICT_V1 | wiki | bundle-laget (`concepts`/`releases`), bundet til sti og `type` (`phase-3-split-table.md` l.166–168) | **(2)** | spec-ens lag er katalogen (§ 3) — men wikien binder lag ⇄ sti ⇄ `source_tier` ⇄ `type` som ERROR-gate (l.63), og en spec-konsument kan ikke lese det av stien alene | stien (`concepts/` ⇄ `layer: concepts`, l.166) | `sed -n 166,168p docs/phase-3-split-table.md` |
| `source_tier` | STRICT_V1 | wiki | kildens troverdighetsklasse (`official-docs`/`changelog-derived`, l.167–168) | **(1)** | **§ 5.1 l.314–318: troverdighet er «*inferred* from the signals … not stored»** (`author`, `usage_count`, `last_modified`) — en lagret klasse er det spec eksplisitt ikke gjør, samme form som `trust_tier` (B.1) | `layer` (bundet 1:1 til det, l.166–168) | `sed -n 314,318p $S` · `sed -n 167,168p docs/phase-3-split-table.md` |
| `source_url` | STRICT_V1 | wiki | kildens URL | **(1)** | **`sources[].resource`, § 5.1 l.303** (absolutt URL) | ingen | `sed -n 303,304p $S` · `sed -n 822p $P` |
| `source_sha` | STRICT_V1 | wiki | kildens innholdshash | **(2)** | som `source_sha256`: ingen digest i spec | ingen | `sed -n 823p $P` |
| `summary` | STRICT_V1 (nullable) | wiki | lengre sammendrag, kan være `null` (`phase-3-split-table.md` l.11, 150) | **(1)** | **`description`, § 4.1 l.194** — men wikien bærer **begge** (`description` påkrevd, `summary` valgfri, l.119–134), så det er en andre, lengre beskrivelse. Hva som skiller dem semantisk hos wikien: **ikke lest** (deres kontraktdokument er ikke her) | `description` (delvis) | `sed -n 826p $P` · `sed -n 853p $P` · `sed -n 150p docs/phase-3-split-table.md` |
| `bundle_profile` | STRICT_V1 (rotindeks) | dette repoet / wiki | hvilken profil bundlen ble skrevet under | **(2)** | § 12 kjenner bare `okf_version`; hvilken **kontrakt** utover formatet et dokument følger har ingen spec-plass | ingen | `sed -n 867p $P` |
| `okf_spec_commit` | STRICT_V1 (rotindeks) | dette repoet / wiki | commiten i upstream-repoet spec-teksten ble lest ved | **(2)** | § 12 versjonerer med `<major>.<minor>`, og v0.2 ble **strammet inn på plass uten versjonsbump** (23.08-runden: «declaring conformance to v0.2 without also naming a commit is now ambiguous», `docs/plan/okf-2026-08-timestamp-tightening.md` § Step 4). Nøkkelen finnes fordi spec-ens eget versjonsskjema ikke bærer det | ingen | `grep -n 'without also naming a commit' docs/plan/okf-2026-08-timestamp-tightening.md` · `sed -n 867p $P` |
Fordeling B.2 (20 egne + `status`): **(1) 9** (`source_system`, `source_query`,
`source_file`, `ingested_at`, `parent`, `references`, `timestamp`, `source_tier`,
`source_url`, `summary` — 10 med `summary`), **(2) 11**. Kommando:
`awk -F'|' '/^\| `/ && $6 ~ /\*\*\(1\)\*\*/' docs/okf-nokkelinventar.md | wc -l`
teller (1)-rader i B.2-formatet (kolonne 6); B.1 har klassen i kolonne 5.
### B.3 Duplikatene, samlet — det ordren ba om eksplisitt
Hver linje er en verdi som **finnes to steder**, målt:
| duplikat | hvor | målt | kommando |
|---|---|---|---|
| `utgave` = `sources[].title` | E, samme dokument | 270/270 byte-identisk | Del A `sources[].title` |
| `req_number` = prefikset av `title` | E/M | 269/269 | B.1 `req_number` |
| `seksjonstittel` = halen av `title` | E/M, konstruert `bundle.py` l.194 | 269/269 ved konstruksjon | `sed -n 194p bundle.py` |
| `normal` = katalogsegmentet i konsept-id | E | 269/269 (`krav/N500/` ↔ `N500`) | B.1 `normal` |
| `trust_tier: unverified` = fraværet av `verified` | E/M | 270/270 og 0/270 | B.1 `trust_tier` |
| `lenker` ↔ `viser_til_krav` | E/M | samme 14 filer, samme referanser (id vs tekst) | B.1 `viser_til_krav` |
| `status: stable` i indeksraden = § 5.4-defaulten | M `index.md` | 270 rader, 4 590 bytes (5,9 %) | Del A `status` |
| `title` i indeksraden + konsept-id som lenketekst | M `index.md` | UUID to ganger per rad, 27,7 % | Del A `title` |
| `ingested_at` = `generated.at` | bibliotek, Dør A | 5/5 goldens | B.2 `ingested_at` |
| `references` (STRUCTURED_V1) = kroppens § 6.1-lenker | bibliotek, Dør B | ved konstruksjon (`structure.py` l.54) | B.2 `references` |
| dekningstall = `## Dekning`-prosaen | E `Normal`-konseptet | `krav_i_kilden`/`krav_i_bundlen`/`tabeller_utelatt`/`figurer_utelatt`/`tegn_utenfor_krav`/`lenker_utelatt` gjentas i kroppen | `sed -n 313,327p bundle.py` |
---
## Del C — utover v0.2 (notert, ikke handlet på)
Idéboka bor i `.claude` (`~/.claude/docs/okf-utover-v02.md`), og dette repoet skriver
ikke i andre repo. Kandidatene sendes derfor til `.claude` i svaret på ordren, i
idébokas format, slik K1 gjorde. To nye, begge fra tall i dette dokumentet:
**U12 — Indeksens rekkefølge er en egenskap uten spec-plass.** § 8 sier ingenting om
oppføringenes rekkefølge (`grep -n -i 'order\|sorted\|ordering' $S` → 13 treff, alle
`orders`-eksempelet og «reordered» om `sources[].id`; 0 om indeksrekkefølge). Målt:
samme indeks sortert på `req_number` = +2 celler (F3 12/17 → F4 14/17). *Kostnad i
dag:* Dør C sorterer på konseptsti, som for et UUID-korpus er ingen sortering. *Tilbake-
spilling:* usikker — kan være en produsentkonvensjon (#10 rører hierarki, ikke rekkefølge).
Egen ordre `…3790625395` finnes alt; dette er notatet, ikke ordren.
**U13 — Integritetsdigest.** Spec har ingen innholdshash (6 treff på
`sha|hash|digest|checksum`, alle substrenger). Tre uavhengige kontrakter fant opp én:
`source_sha256` (Dør B), `source_sha` (wikien), `ingest_manifest`s hash16 (commons) —
og Dør C beviser eierskap ved innholdsidentitet. *Kostnad i dag:* ikke målt. *Tilbake-
spilling:* mulig — upstream #8 (JSON Schema) er stedet en slik nøkkel ville måttet
navngis.
---
## Selvsjekk-porten (kjørt på den ferdige filen)
```
grep -c '^|' docs/okf-nokkelinventar.md # -> SELFCHECK_TOTAL rader totalt
grep '^|' docs/okf-nokkelinventar.md | grep -vc '`' # -> SELFCHECK_NOCMD rader UTEN backtick-kommando
grep -c '^|---' docs/okf-nokkelinventar.md # -> SELFCHECK_SEP skillelinjer
grep '^|' docs/okf-nokkelinventar.md | grep -v '`' | grep -vc '^|---' # -> SELFCHECK_HDR header-rader (uten backtick)
```
SELFCHECK_SENTENCE

12
docs/plan/index-ordering.md
View file

@ -12,7 +12,7 @@ the requester's own measured gain actually lives.
## What was asked
`vegnormal-okf` measured four faceted arms against one corpus, 32 questions,
A downstream consumer measured four faceted arms against one corpus, 32 questions,
locked scoring, shared denominator 17. Arm F4 was arm F3 with exactly one
variable moved — the index's sort order — and scored 14/17 against F3's 12/17.
The single-variable claim was verified against the files rather than assumed:
@ -46,13 +46,13 @@ the part carrying the measured gain.
Measured, on this repo's own interpreter:
```
sorted(["N100.9", "N100.10", "N100.2", "N100.20", "N100.1", "N99.1"])
-> ['N100.1', 'N100.10', 'N100.2', 'N100.20', 'N100.9', 'N99.1']
sorted(["Q100.9", "Q100.10", "Q100.2", "Q100.20", "Q100.1", "Q99.1"])
-> ['Q100.1', 'Q100.10', 'Q100.2', 'Q100.20', 'Q100.9', 'Q99.1']
```
Two independent inversions: `N100.10` sorts before `N100.2`, and `N99.1` lands
*after* the entire `N100` family. A `sort_key: req_number` resolved through
Python's default ordering would therefore have handed `vegnormal-okf` a
Two independent inversions: `Q100.10` sorts before `Q100.2`, and `Q99.1` lands
*after* the entire `Q100` family. A `sort_key: req_number` resolved through
Python's default ordering would therefore have handed that consumer a
different wrong order, not the 100.0% ascending they measured over 24 338
pairs. Their result came from comparing numerically. The key name alone does
not carry that.

16
docs/plan/intake-boundaries.md
View file

@ -13,14 +13,14 @@ carries no extractable text layer, `pdfplumber` returns nothing for it, and the
file is refused with `extractor_empty_pdf`. That refusal is correct and final,
not a placeholder.
**The consequence, stated plainly.** The `vegnormal-okf` PDFs are rastered 11
**The consequence, stated plainly.** A downstream consumer's PDFs are rastered 11
of 11 (measured, `docs/2026-08-29-konverteringsarmen-arm-a.md`). They will
never enter through the inbox. Any statement of the form "the Vegnormalene
never enter through the inbox. Any statement of the form "those
PDFs in ⇒ bundle out" is not a goal this library holds, and the phrasing should
not survive in planning documents.
For the record, that premise never described anything that happened:
`vegnormal-okf` builds its bundle from seven JSON files through **Door C**, not
That consumer builds its bundle from seven JSON files through **Door C**, not
through the inbox. The inbox path has never been the route those concepts took.
A consumer with rastered source documents needs a text layer produced outside
@ -32,8 +32,8 @@ best-effort guess.
`materialize.reduce_to_id_grammar` reduces each path component to
`[a-z0-9][a-z0-9-]*`, and `segmentation.normalize_segment_path` applies the
same grammar to segment paths. A source path of `krav/N500/id-<uuid>` therefore
becomes `krav/n500/id-<uuid>`.
same grammar to segment paths. A source path of `krav/Q500/id-<uuid>` therefore
becomes `krav/q500/id-<uuid>`.
**This is the library's Phase 1 grammar, not the specification's.** OKF v0.2
§ 2 defines a concept ID as the file path minus `.md` and imposes no case rule.
@ -47,7 +47,7 @@ helper was introduced to remove.
**The consequence, stated plainly.** A source whose identities are
case-sensitive cannot be reproduced verbatim through Door B. This is measured,
not predicted: in `vegnormal-okf`'s B3 run against N500:2024, 267 of 270
not predicted: in that consumer's B3 run against one 270-concept source, 267 of 270
concept identities matched case-folded and 0 matched verbatim — the entire
difference across all 267 was case alone.
@ -56,10 +56,10 @@ which writes external concepts verbatim, or accept the folded form.
## 3. Structured sources are not segmentable as they stand
Measured by `vegnormal-okf` 2026-09-01 (B3, commit `0f8a27f`), with a
Measured by that consumer 2026-09-01 (B3, commit `0f8a27f`), with a
known-positive control that held at 270 of 270 through their domain path:
- The segment proposer returned **0 segments** for N500:2024. The raw file is
- The segment proposer returned **0 segments** for that source. The raw file is
1 633 250 characters on a single line, and the proposer's rules are
line-oriented. The same proposer returns 3 segments for a markdown document
with headings, so the zero is a measurement rather than a broken query.

11
docs/plan/okf-assets-section-6-4.md
View file

@ -14,12 +14,11 @@ concepts. So a picture is neither permitted nor forbidden — it is unaddressed,
and a producer that carries one is guessing about where it goes and what a
consumer may assume.
The need is not hypothetical. Measured on R761 Prosesskoden:2025, a Norwegian
road-construction process code published both as a 701-page PDF and as a
NISO-STS XML delivery: the process text is carried in full, and 12 `Tabell N-N`
The need is not hypothetical. Measured on a Norwegian reference standard published both as a 701-page
PDF and as a NISO-STS XML delivery: the process text is carried in full, and 12 `Tabell N-N`
and 9 `Figur N-N` captions stand over nothing, because the publisher ships
those tables as raster images in **both** deliveries. Process 84 says
"toleranseklasse ... er gitt i tabell 84-2" and table 84-2 is a JPEG. A bundle
those tables as raster images in **both** deliveries. Section 84 says
its classes "are given in table 84-2" and table 84-2 is a JPEG. A bundle
built from that document reads as complete and is not.
## The proposed wording
@ -86,7 +85,7 @@ The measurement that did NOT pass first time is in the report: Door C merged
the concepts and wrote no `assets/` at all, so every pointer in the imported
bundle named a missing file. Fixed here, by the content-identity rule that door
already owns. The run record is
`docs/2026-09-17-bilder-i-bundlen-trinn1.md` § 5.
the 0.10.0 entry in `CHANGELOG.md`.
## Route

8
docs/plan/references-cue-rule.md
View file

@ -48,7 +48,7 @@ markdown link target, or a number immediately preceded by a reference cue from
a closed set.** Everything else number-shaped in body prose is not a reference.
The rule is positive rather than a blacklist, and that is forced by the data,
not chosen for taste: corpus A proves `V221` and `V240` are genuine document
not chosen for taste: corpus A proves `W221` and `W240` are genuine document
numbers, corpus B proves `V0.3.0` is a software version, and the two are the
same token shape. Nothing structural separates them. Only the words in front of
them do, so only a positive cue can be defended.
@ -98,12 +98,12 @@ reader, a false one is not.
Two recall losses inside the corpora measured, both accepted: a reference whose
cue is a preposition (`i henhold til …` where the noun is absent) and one
written as a prefixed compound (`N-V240`) are dropped.
written as a prefixed compound (`Q-W240`) are dropped.
## What was deliberately not done
**The prefix-resolution rule (`N130` in a body not finding a document numbered
`N130.2`) stays open.** The order allowed it in the same pass only if the data
**The prefix-resolution rule (`Q130` in a body not finding a document numbered
`Q130.2`) stays open.** The order allowed it in the same pass only if the data
bore it. It does not: of 2 589 documents across both corpora, 2 562 carry a
number and **0** of those numbers are dotted, so a unique-prefix match has
nothing here to be defended against. Closing two things badly is worse than

14
docs/plan/structure-derivation.md
View file

@ -63,7 +63,7 @@ nothing derived can still use everything declared.
| | same number + ordered `version` | **derived** |
| `version`, `status`, `date` | producer only | declared |
The number grammar is alpha-prefixed (`N500`, `V720`, `R610.4`) or dotted
The number grammar is alpha-prefixed (`Q500`, `W720`, `P610.4`) or dotted
numeric (`4.2.1`). A **bare integer is deliberately not a number**: admitting
`12-things.md` or `2026-notes.md` would stamp a document number on most of a
second brain that never had one.
@ -239,8 +239,8 @@ change — rather than as "not possible".
and supersession.
2. **A reference to a parent-level number does not resolve to a child.** In the
synthetic corpus, `N130` in the body of one document does not resolve to a
document numbered `N130.2`; it is reported unresolved (`N130?`). This is
synthetic corpus, `Q130` in the body of one document does not resolve to a
document numbered `Q130.2`; it is reported unresolved (`Q130?`). This is
honest but incomplete — the fix is a prefix-resolution rule
(`subject` resolves to the unique document whose number starts with it), and
it was left out because "unique" has to be defined against a real corpus
@ -270,7 +270,7 @@ change — rather than as "not possible".
# Door C: the index carries the sender's facts, and only theirs
Ordered by vegnormal-okf (`20260826T224500Z-873805419`) on a measurement, not a
Ordered by a downstream consumer (`20260826T224500Z-873805419`) on a measurement, not a
preference. A pre-registered reasoning bake-off on 2026-08-27 ran the same
bundle through two index shapes:
@ -320,7 +320,7 @@ reads a value only to check the policy can render it.
Operator directive 2026-08-27, verbatim: *"alt vi lager rundt OKF publiseres på
Forgejo og skal virke for alt av innhold"* — so a design that only works for
numbered norms is wrong even when it scores well on N100/N200/N500. Nothing in
numbered norms is wrong even when it scores well on three numbered reference bundles. Nothing in
`_project_facets` can key off a numbering scheme, and the suite says so with an
unnumbered Norwegian meeting note (`status` + `date`, no number, no parent, no
version) and with a document declaring none of the policy's keys, which renders
@ -414,14 +414,14 @@ own `import_prefix` now states it directly, and M10 is killed.
1. **`title` is not in `STRUCTURED_V1`'s facet set.** Widening it would move
Door B's index bytes, so it stays a caller's key to name. For a sender whose
concept paths are UUIDs this is the difference between a title in the index
and none — flagged to vegnormal-okf rather than decided here.
and none — flagged to that consumer rather than decided here.
2. **Door C's index is an append plus a per-target refresh, not a projection of
the whole bundle** (contrast D-S4). A concept removed from the bundle by hand
leaves its entry behind. Reprojection would mean enumerating the directory,
which no profile has asked this door to do.
3. **Parent-prefix resolution is still deferred**, unchanged and for the
unchanged reason: "unique prefix match" must be defined against a real
corpus. `vegnormal-okf` has that corpus; the count is theirs to take.
corpus. The consumer has that corpus; the count is theirs to take.
4. **The before/after number is not ours to produce.** Whether the faceted index
is worth its cost on a real corpus is answered by re-running the same arm
with the same scoring, on their data.

2
pyproject.toml
View file

@ -51,7 +51,7 @@ okf = "llm_ingestion_okf.cli:main"
# reader, and a converter that reaches the office types.
#
# WHY pdfplumber, and why the floor is not free (measured 2026-08-21,
# docs/2026-08-21-g2-pdf-extraction-measurement.md): on a real Vegnormalene
# docs/2026-08-21-g2-pdf-extraction-measurement.md): on a real published
# requirement table pdfplumber keeps 4 of 4 rows with label and value on the
# same line; pypdf, pdfminer.six and pymupdf each keep 0 of 4, emitting all
# labels then all values, which a downstream reader can only re-pair by

32
src/llm_ingestion_okf/assets.py
View file

@ -5,10 +5,10 @@ text, every warning said so, and the only writer into a bundle was
:func:`materialize.write_bytes`, whose signature is ``(bundle_dir, name,
content: str)`` -- UTF-8 in, text out, no binary route anywhere. A document
whose table is a raster picture therefore reached a concept as an absence with
no denominator. Measured on R761 Prosesskoden:2025, the publisher's own
NISO-STS delivery: the process text is carried in full, and 12 ``Tabell N-N``
and 9 ``Figur N-N`` captions stand over nothing, so process 84's
"toleranseklasse ... er gitt i tabell 84-2" points at empty space.
no denominator. Measured on a 701-page reference standard, the publisher's
own NISO-STS delivery: the section text is carried in full, and 12
``Tabell N-N`` and 9 ``Figur N-N`` captions stand over nothing, so a sentence
saying "... er gitt i tabell 84-2" points at empty space.
THIS MODULE IS THE ONE PLACE THAT DECIDES WHAT AN IMAGE IS. A reader hands it
bytes and a name; it returns a carried image or raises a coded rejection. That
@ -18,7 +18,7 @@ bundle layout.
THREE RULES, and each one exists because the alternative is a silent lie:
- **The type is SNIFFED, never claimed.** Measured on the R761 delivery, the
- **The type is SNIFFED, never claimed.** Measured on that delivery, the
graphics directory holds ``.bmp``, ``.jpg`` and ``.png`` side by side and the
document's ``xlink:href`` values are whatever the publisher's tool wrote. A
name is a claim; the magic bytes are the fact. A consumer dispatching on the
@ -94,7 +94,7 @@ _MAGIC: tuple[tuple[bytes, str, str], ...] = (
#: set is converted losslessly to PNG, or refused with a code -- never carried
#: silently, which is what this package did until this round of 0.10.1.
#:
#: MEASURED 2026-09-19 over the frozen R761 delivery's own `assets/`
#: MEASURED 2026-09-19 over the frozen reference delivery's own `assets/`
#: (denominator 50): 29 JPEG, 2 PNG and **19 "PC bitmap, Windows 3.x, 8-bit,
#: compression 1"**. The 19 are byte-correct files that nothing reads, so 19 of
#: that document's figures were present and invisible at the same time -- and
@ -141,8 +141,8 @@ IMAGE_POINTER = re.compile(
#: THE SIZE A PICTURE MAY DECLARE. Read off the corpora rather than chosen:
#: over the 4 828 image objects of the 43-document reference corpus the largest
#: is 4 515 x 4 128 (18.6 MP, a landscape drawing), and over R761's 109
#: delivered pictures it is 2 072 x 656 (1.4 MP). This sits an order of
#: is 4 515 x 4 128 (18.6 MP, a landscape drawing), and over the reference
#: standard's 109 delivered pictures it is 2 072 x 656 (1.4 MP). This sits an order of
#: magnitude above both, so nothing anyone has measured is refused -- and a
#: declared size beyond it is a COUNTED refusal instead of a killed build.
#:
@ -353,7 +353,7 @@ def inflated_size(raw: bytes, *, name: str, limit: int | None = None) -> int:
and it stops at the first chunk that crosses `limit`. A legitimate image is
therefore inflated twice -- once here and once by the reader that carries
it -- which is the price of not holding an unbounded buffer to find out how
big it is. Measured on R761 Prosesskoden:2025 (50 image objects): the
big it is. Measured on the 701-page reference standard (50 image objects): the
second pass costs under a second of a 200-second extraction.
A stream that is not valid deflate data is not this function's problem: the
@ -546,7 +546,7 @@ def _bmp_dimensions(data: bytes) -> tuple[int, int] | None:
#: `biCompression`: uncompressed, and the 8-bit run-length encoding 19 of
#: R761's 50 assets use. Every other value -- RLE4, BITFIELDS, embedded JPEG or
#: the reference delivery's 50 assets use. Every other value -- RLE4, BITFIELDS, embedded JPEG or
#: PNG -- is refused by name rather than guessed at.
_BMP_RGB = 0
_BMP_RLE8 = 1
@ -690,7 +690,7 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
difference between a skipped pixel and a missing one. Running out of bytes
leaves the rest of the frame at index 0 -- indistinguishable, in the
output, from a delta that skipped it, which is why every decoder agrees on
the wrong picture: measured 2026-09-19 on a real 352x548 R761 asset, a
the wrong picture: measured 2026-09-19 on a real 352x548 reference asset, a
stream cut to 90 % was carried with 13 923 pixels wrong, to 50 % with
95 890, to 10 % with 166 525, with no code and no row. So the loop may end
HERE and nowhere else, and `_bmp_flat_rows` refuses the same shape with the
@ -698,7 +698,7 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
The terminator is checked rather than `biSizeImage`, which is a claim by
the same untrusted header. Read off the corpus: over the 25 RLE8 BMPs
the frozen R761 delivery ships (24 distinct; the bundle carries 19 of
the frozen reference delivery ships (24 distinct; the bundle carries 19 of
them, the rest being an unpointed duplicate and four no concept names),
25 of 25 end at an explicit
end-of-bitmap, on 25 of 25 it is the stream's LAST two bytes, and on 25 of
@ -832,7 +832,7 @@ def bmp_to_png(data: bytes, *, name: str) -> bytes:
front of it.
Pillow is still the INDEPENDENT decoder in the tests, which is the job it
is good for here: 19 of 19 of R761's real RLE8 assets decode to identical
is good for here: 19 of 19 of the reference delivery's real RLE8 assets decode to identical
RGB through both paths (measured 2026-09-19, before this was written).
"""
head = _bmp_header(data, name=name)
@ -977,7 +977,7 @@ def asset_name(image: ExtractedImage) -> str:
arriving twice are one file, whichever document dropped them. The residual
is stated rather than claimed away -- identical bytes shipped under two
different base names are two files in the bundle, both correct and both
holding the same picture. Measured on the fixture inbox and on the R761
holding the same picture. Measured on the fixture inbox and on the reference
delivery, that case occurs 0 times; a rule that could not produce it at all
would have to drop the readable tail entirely, and the tail is what makes
`assets/` legible to the person checking a bundle by hand.
@ -986,8 +986,8 @@ def asset_name(image: ExtractedImage) -> str:
The suffix comes from the bytes and the stem from the name, reduced to the
same grammar every generated filename in this package uses. A link target
sits inside `](...)`, which a space or a closing parenthesis terminates --
and the R761 delivery's own hrefs carry both (`25-0143 - Tabeller -
Prosesskoden (R761-R762).jpg`), so reducing is what makes the pointer
and the reference delivery's own hrefs carry both (a name of the form
`25-0143 - Tabeller - Navn (A1-A2).jpg`), so reducing is what makes the pointer
followable rather than merely tidy.
"""
# The BASENAME, never the path the document reached it through. Measured

4
src/llm_ingestion_okf/cli.py
View file

@ -927,7 +927,7 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
help=(
"carry the images the sources declare into the bundle's assets/ "
"directory and point at them from the concept text where they "
"stand. ON since 0.10.0. Measured on the R761 process code: 12 "
"stand. ON since 0.10.0. Measured on a 701-page reference standard: 12 "
"Tabell N-N and 9 Figur N-N captions stood over nothing, because "
"the publisher ships those tables as raster pictures. The image "
"BYTES are not screened -- the gate reads text -- and log.md says so"
@ -1070,7 +1070,7 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"SEGMENTATION arm and not a reader option: the extracted text is "
"byte for byte the same either way, and a PDF that carries no "
"bookmark tree builds byte-identically with the flag on. Measured "
"on one 701-page process code whose publisher also ships a NISO-STS "
"on one 701-page reference standard whose publisher also ships a NISO-STS "
"structure for it: the text rules recover 1967 of 2761 titled "
"sections and 0 of its 28 chapters, while its bookmark tree matches "
"2761 of 2761 exactly. The title comes from the BOOKMARK, so it is "

30
src/llm_ingestion_okf/consume.py
View file

@ -680,7 +680,7 @@ LOCATOR_KEYS = (
#: What a locator key looks like to a reader that does not know the producer.
#: The pass-through rule is this PREFIX and not `LOCATOR_KEYS`, which is a list
#: of the producers someone thought of: measured 2026-09-08, 269 of 274 concepts
#: in the N500 bundle carry `source_element_id`, a locator that repository chose
#: in a third-party bundle carry `source_element_id`, a locator that repository chose
#: under the same rule ("the key says what it indexes") and this library never
#: writes. An allowlist drops it, and the excerpt then names a document without
#: naming the place in it. A PREFIX, never a substring -- `resource_owner`
@ -717,9 +717,9 @@ def read_sources(lines: Sequence[str]) -> tuple[tuple[Mapping[str, str], ...], b
BOTH YAML forms are read, and that is a measurement rather than a
preference. Measured 2026-09-08: K2 writes the flow form on 629 of 629
concepts, the N500 bundle writes the block form on 270 of 270. A reader
concepts, a third-party bundle writes the block form on 270 of 270. A reader
handling one form delivers the other bundle with no address at all -- and
for N500 there is nothing else, because it carries zero locator keys.
for that bundle there is nothing else, because it carries zero locator keys.
Reading the block form is not a licence to WRITE it, and since K3-24 the
reason is a different one: the line-oriented parser DOES round-trip a
@ -879,7 +879,7 @@ _TOKEN_SPLIT_RE = re.compile(r"[^0-9a-zà-öø-ÿ]+")
_DASH_TO_HYPHEN = str.maketrans(dict.fromkeys("‐‑‒–—―−", "-"))
#: An identifier: NUMERIC groups joined by `.` or `-`, with an optional letter
#: prefix that touches its digits without a separator (`R610.4`).
#: prefix that touches its digits without a separator (`P610.4`).
#:
#: THE LETTERS ARE THE POINT, and this pattern was narrowed by a measurement
#: rather than written this way. A rule that joined alphanumeric groups across
@ -944,7 +944,7 @@ def tokens_match(left: str, right: str, *, stems: frozenset[str] | None = None)
round 10's repair. The floor alone matched four characters that are not a
stem at all: measured on the pinned 453-concept bundle with the control run
first, `under` occurs 79 times by equality and matches 172 concepts by
prefix, while `undersjoisk` occurs 0 times and matches the same 172;
prefix, while `under…` occurs 0 times and matches the same 172;
`bilateral` occurs 0 times and matches 400 of 453 through `bilag`;
`standhaftig` 0 and 219 through `standard`.
@ -959,7 +959,7 @@ def tokens_match(left: str, right: str, *, stems: frozenset[str] | None = None)
What separates them is that `pris` is a word and `bila` is not. With
`stems`, `bilateral` falls to 0 on both bundles and every hit@8 row keeps
its rank. `undersjoisk` still reaches 162 because it shares `under`, which
its rank. `under…` still reaches 162 because it shares `under`, which
IS a word here -- a genuine Norwegian morpheme, so that residual is a
different answer rather than a ceiling.
@ -1003,7 +1003,7 @@ def tokens_match(left: str, right: str, *, stems: frozenset[str] | None = None)
return True
# Equality first, and only inside this branch. A token always answers to
# itself, whatever the corpus contains -- but the check must NOT move above
# the floor, where it would make `veg`/`veg` match and change the shipped
# the floor, where it would make `ost`/`ost` match and change the shipped
# rule for every token shorter than `MIN_SHARED_PREFIX`.
return left == right or left[:shared] in stems
@ -1393,7 +1393,7 @@ DEFAULT_TIE_SHARED_RANK = True
#: Round 10. `MIN_SHARED_PREFIX = 4` matches on four characters whether or not
#: they are a stem. Measured on the pinned 453-concept bundle, control first:
#: `bilateral` occurs 0 times by equality and matches 400 of 453 through
#: `bilag`; `standhaftig` 0 and 219 through `standard`; `undersjoisk` 0 and 172
#: `bilag`; `standhaftig` 0 and 219 through `standard`; `under…` 0 and 172
#: through `under`. Requiring the shared prefix to occur as a token in the
#: bundle's own concepts takes the first to 0 and the second to 56 on the
#: default bundle (0 and 33 on Arm B) with every hit@8 row keeping rank 1 on
@ -1422,9 +1422,9 @@ DEFAULT_STEM_PREFIX = True
#: what a narrower section titled with that subject PLUS a qualifier scores,
#: and then loses to it on the body count. Measured 2026-09-10 on a
#: 2 761-concept bundle of one standard, where the answering section carries
#: the bare term as its title on three of six scored questions: `Hovedprosesser`
#: behind `Hovedprosess 81 ...`, `Armering` behind `Armering av ...`,
#: `Inspeksjon` behind `Enkel inspeksjon`.
#: the bare term as its title on three of six scored questions, and each time
#: lost to a section titled with the same word plus a qualifier (`X` behind
#: `X av ...`, `X` behind `Enkel X`).
#:
#: WHY A PARTITION AND NOT A FOURTH SIGNAL, measured rather than argued. RRF
#: consumes RANKS ONLY, and with shared ranks a signal whose positive group is
@ -1661,13 +1661,13 @@ def concept_scores(
them all the same score. The tie-break then orders that group by
`concept_id`, and the fusion reads the result as if it were a measurement.
MEASURED 2026-09-08 on the N500 bundle (270 concepts): the document prior
MEASURED 2026-09-08 on a 270-concept third-party bundle: the document prior
has **two** distinct values there and 269 concepts share one, so that
signal contributed the concepts' UUIDs in alphabetical order, spread across
`1/61` to `1/329`. The best concept covering `vann- og frostsikring` in a
subsea tunnel answered **7 of 7** question tokens and led the body signal
`1/61` to `1/329`. The best concept covering `vann- og fuktsikring` in a
deep wine cellar answered **7 of 7** question tokens and led the body signal
at rank 6; it fused to rank 14, outside the cut, behind concepts sharing
only `tunnel` and `vann` whose ids sorted earlier. With shared ranks it
only `kjeller` and `vann` whose ids sorted earlier. With shared ranks it
fuses to rank 3.
**`link_in_signal=False` is the DEFAULT reading since K3-25**

6
src/llm_ingestion_okf/corpus.py
View file

@ -103,11 +103,11 @@ REFUSED_NAME_CAP = 10
#:
#: ON, and the reason is the premise the capability was ordered on: what ends
#: up in a bundle after converting N sources has to BE the sources. Measured on
#: R761 Prosesskoden:2025 -- 2 726 of 2 726 processes carry their text and 12
#: a 701-page reference standard -- 2 726 of 2 726 sections carry their text and 12
#: `Tabell N-N` and 9 `Figur N-N` captions stand over nothing, because the
#: publisher ships those tables as raster pictures in both the PDF and the
#: NISO-STS delivery. A bundle that reads as complete and answers "toleranse-
#: klasse ... er gitt i tabell 84-2" with an empty space is the defect; it is
#: NISO-STS delivery. A bundle that reads as complete and answers "... er gitt
#: i tabell 84-2" with an empty space is the defect; it is
#: not a preference about bundle size.
#:
#: WHAT THIS COSTS IS PUBLISHED RATHER THAN IMPLIED. Measured over the K2

2
src/llm_ingestion_okf/errors.py
View file

@ -157,7 +157,7 @@ class ExtractionError(IngestError):
bytes are not an image at all: this one says they are, and carrying them
would put a file in the bundle that the `images: N` count reports as
arrived and nothing downstream can read. Measured 2026-09-19 on the
frozen R761 delivery: 19 of its 50 assets were carried in exactly that
frozen delivery of a 701-page reference standard: 19 of its 50 assets were carried in exactly that
condition, as RLE8 BMP
- `asset_bmp_unsupported` — a BMP variant this reader does not express
(RLE4, BITFIELDS, 16- or 32-bit samples, a 12-byte BITMAPCOREHEADER, a

31
src/llm_ingestion_okf/extract.py
View file

@ -131,7 +131,7 @@ _PANDOC_FORMATS: dict[str, str] = {
#
# `.xml` JOINED THE TABLE 2026-09-11, as `measured`, and the class was read off
# the definitions above rather than inherited: the one file is a publisher's own
# NISO-STS delivery of R761, written for their purposes years before any lookup
# NISO-STS delivery of a 701-page reference standard, written for their purposes years before any lookup
# of ours, and its 2 761 titled `<sec>` are a fasit nobody here authored. The
# honesty limit that travels with it and does NOT move when the build reaches
# the reader's ceiling: the denominator is ONE file, ONE publisher, ONE schema.
@ -141,7 +141,7 @@ _PANDOC_FORMATS: dict[str, str] = {
#
# `.pdf` JOINED THE TABLE 2026-09-10, as `measured`, and it enters on the
# strongest evidence of any row here: eight real corpus PDFs with a fasit the
# operator hand-counted document by document, plus a 701-page process code
# operator hand-counted document by document, plus a 701-page reference standard
# whose PUBLISHER also ships a NISO-STS structure for it -- 2 761 titled
# sections, written for their own purposes and long before any lookup of ours.
# The honesty limit that travels with it: those 2 761 rows are ONE product in
@ -291,9 +291,9 @@ _BLOCK_TAGS = frozenset(
# --- NISO-STS, and the two facts the whole reader turns on ------------------
#
# ONE: `<label>` carries the number and `<title>` carries the text. Measured on
# the 701-page process code, 2 of its 2 761 `<title>` strings begin with a
# the 701-page reference standard, 2 of its 2 761 `<title>` strings begin with a
# digit -- the number is a sibling element, never glued on. A fasit that shows
# `"2.1Hovedprosesser"` is its BUILDER joining the two. Emitting `<title>`
# `"2.1Hovedkapitler"` is its BUILDER joining the two. Emitting `<title>`
# alone therefore scores 0 of 2 761 while every line of this file looks right,
# because the number is what okf reduces to a directory name.
#
@ -386,17 +386,17 @@ _DATA_URI = re.compile(r"^data:(?P<media>[^;,]*)(?P<base64>;base64)?,(?P<payload
#: `_ZERO_WIDTH_CPS` beside U+200B, U+200C, U+200D and U+FEFF, and
#: `output:zero-width-present` is an any-tier carrier: `fail_secure` at every
#: trust level, with no sanitisation and no exception. Measured by PM
#: 2026-09-18, R761 Prosesskoden:2025 carries 71 U+00AD and 0 of the other
#: four, so a 701-page process code is unreadable for the whole chain over
#: 2026-09-18, a 701-page reference standard carries 71 U+00AD and 0 of the
#: other four, so the whole document is unreadable for the whole chain over
#: Norwegian hyphenation points inside words -- `ar[SHY]beider`,
#: `bitu[SHY]men`, `asfalt[SHY]betong`. The verdict is formally right and
#: `sur[SHY]deig`, `bake[SHY]plate`. The verdict is formally right and
#: materially a false positive, and the operator's answer (2026-09-18) is that
#: okf removes this character before the guard and SAYS SO in the accounting.
#:
#: The other four stay. They carry no typographic job in running text, so
#: removing one would be a decision about what the guard screens for, taken in
#: the wrong repository. U+00A0 NBSP is not in the guard's set at all and is
#: not touched either -- R761 ships 6 633 of them.
#: not touched either -- that document ships 6 633 of them.
SOFT_HYPHEN = "\u00ad"
@ -438,7 +438,7 @@ class ExtractedDocument:
#: the resolver returned and that were CARRIED as images. Recorded where
#: the resolution happened rather than inferred from bytes afterwards: an
#: unpointed file with the same bytes as a carried one was carried through
#: nothing (R761 ships eight such duplicates).
#: nothing (the reference delivery ships eight such duplicates).
files: tuple[str, ...] = ()
#: How many U+00AD the normalisation door removed from this text. Zero for
#: every document that carried none, which is 0 of the 78 readable
@ -783,7 +783,7 @@ class _XmlTextExtractor:
if prefix:
# A prefix still pending here belongs to a section that turned out
# to have no body line at all, and REPLACING it would drop it from
# the document. Measured on R761: exactly one `x)`, two characters,
# the document. Measured on the reference standard: exactly one `x)`, two characters,
# which is the whole distance between 0.999998 and exact.
if self._prefix:
self._lines.append(self._prefix.rstrip())
@ -857,7 +857,7 @@ class _XmlTextExtractor:
def _graphic(self, element: Element) -> bool:
"""A `<graphic>`, in the place it stands. Reports whether it was one.
Measured on the R761 delivery, 2026-09-16: 50 `<graphic>` elements, all
Measured on the reference delivery, 2026-09-16: 50 `<graphic>` elements, all
50 direct children of a `<sec>`, none inside a `<table-wrap>`, none
carrying a caption element of any kind -- the "Figur 11.1 ..." line a
human reads is a sibling `<p>` this reader already emits on its own
@ -1008,7 +1008,7 @@ def declared_identity(name: str, data: bytes) -> DeclaredIdentity | None:
"""`xml`: the identity a NISO-STS document declares, or `None`.
MEASURED ON THE ONE STS DOCUMENT THIS ROW HAS: exactly one `<std-ident>`
(`<doc-number>R761 Prosesskoden</doc-number>` beside `<year>2025</year>`)
(`<doc-number>P761 Oppskriftsboka</doc-number>` beside `<year>2025</year>`)
and one `<title-wrap>` whose `<full>` is the document's title -- while the
file carrying it was named for a delivery path, a UUID occurring 0 times in
the document. `<doc-type>` is read by nobody: it said `Innledning` there,
@ -1738,7 +1738,7 @@ def _pdf_image(stream: object, name: str) -> ExtractedImage:
the publisher's own bytes, unre-encoded, and its content-addressed name is
therefore stable for as long as the document is.
Measured on R761 (2026-09-16): 29 of 50 image objects are `DCTDecode` and
Measured on the reference standard (2026-09-16): 29 of 50 image objects are `DCTDecode` and
take the verbatim route; 21 are `FlateDecode` and are encoded here. Over
the 33-document K2 reference corpus the population is 4 828 objects, and
the filters are mixed enough (`FlateDecode`, `DCTDecode`, `JPXDecode`,
@ -1978,7 +1978,7 @@ class OutlineMark:
`level` is what the TREE declares, not a distance normalised against
anything: a document whose outline carries its own root node puts its
chapters at level 2, and rewriting that here would state a structure the
publisher did not. Measured on a 701-page process code -- the tree's levels
publisher did not. Measured on a 701-page reference standard -- the tree's levels
2..8 hold 28/118/500/1141/872/93/9 nodes against the publisher's own
NISO-STS depths 1..7 at 28/118/500/1141/868/97/9, so the mapping is level
minus one on five rows and the publisher disagrees with the publisher on
@ -2008,7 +2008,8 @@ class PdfOutline:
`collided` is the same principle applied to the OTHER way a node leaves
without a boundary. Two bookmarks can resolve to one line -- measured on
the 701-page process code, its tree's root node and `SVV - Forside` both
the 701-page reference standard, its tree's root node and a front-matter
node both
land on line 0 -- and only the first can become a mark, because two
candidates at one offset give the first an empty span that the orphan check
then deletes without a word. That was measured on that document and is why

3
src/llm_ingestion_okf/importer.py
View file

@ -427,7 +427,8 @@ def import_bundle(
index carried no frontmatter, so it declared no `bundle_id`, and the
reading direction refused it with `bundle_id_missing` because SS 3.1's
identity is the `(bundle_id, concept_id)` tuple and half of it was absent.
Reported by vegnormal-okf 2026-09-08 (FUNN 1), who worked around it by
Reported by a downstream consumer repository 2026-09-08 (FUNN 1), which
worked around it by
using this door as a gate and writing the consumable tree themselves.
Keyword-only with a default of `None`, so every existing call site emits

6
src/llm_ingestion_okf/profiles.py
View file

@ -69,10 +69,10 @@ _FLOW_KEY = re.compile(r"^[A-Za-z_][A-Za-z0-9_-]*$")
def yaml_block_plain(value: str) -> bool:
"""Whether `value` reads back verbatim as a plain scalar in a block mapping.
MEASURED ON R761 (K3-19): 217 of 2 024 first spec points carry `": "`, and
MEASURED ON A 701-PAGE REFERENCE STANDARD (K3-19): 217 of 2 024 first spec points carry `": "`, and
PyYAML's `safe_load` refused exactly those 217 concepts' frontmatter. K3-22
measured the same rule over every top-level value in eleven trees (the K2
default bundle, the R761 HTML and XML builds, a five-document project, this
default bundle, that standard's HTML and XML builds, a five-document project, this
repository's examples and fixtures, and five consumer bundles): 0 refused
that PyYAML returns verbatim, 0 kept that it does not. `" #"` is here
although PyYAML does not refuse it: `title: Kap #3` loads, as `Kap`.
@ -828,7 +828,7 @@ class IndexPolicy:
#: only, after `entry_pattern` misses, so no emitted byte can move: every
#: line this library writes still comes from `link_template`.
#:
#: WHY IT EXISTS (vegnormal-okf, FUNN 1/2, 2026-09-08). OKF SPEC section 8
#: WHY IT EXISTS (a downstream consumer repository, FUNN 1/2, 2026-09-08). OKF SPEC section 8
#: shows `* [Title](file.md) - description` in its own example and Google's
#: generator writes it, while this library's segmented profiles write
#: `- [Title](file.md)`. Measured, the star row parsed as `None` -- curated

8
src/llm_ingestion_okf/propose.py
View file

@ -157,7 +157,7 @@ RULE_PDF_OUTLINE = "rule:pdf-outline"
#: for -- an ATX line reaching the proposer says nothing about who wrote it, so
#: a `<sec><title>` and a heading a converter guessed out of a font size were
#: indistinguishable in the artifact and were judged by the same two steps.
#: Measured on R761: the orphan check removed 710 of 2 761 declared sections
#: Measured on a 701-page reference standard: the orphan check removed 710 of 2 761 declared sections
#: (710 of 710 removed are followed immediately by another heading -- they are
#: containers) and Arm F folded 2 066 more, 2 089 -> 23 at shipped defaults.
RULE_XML_SECTION = "rule:xml-section"
@ -699,7 +699,7 @@ def _sheet_section_rows(lines: list[str]) -> dict[int, tuple[str, str]]:
#: its content in its own cells, and the two DECLARED-structure rules because
#: the check judges whether a GUESS was a heading -- a question a publisher's
#: own tree has already answered, and one that deletes every container section
#: if asked. Measured on one 701-page process code: 683 of 2 762 bookmark nodes
#: if asked. Measured on one 701-page reference standard: 683 of 2 762 bookmark nodes
#: and 710 of 2 761 STS sections are containers.
_ORPHAN_EXEMPT = (RULE_SHEET_SECTION, RULE_PDF_OUTLINE, RULE_XML_SECTION)
@ -829,7 +829,7 @@ def find_candidates(
first line, which is the right question for a heading a heuristic GUESSED
and the wrong one for a section a publisher DECLARED: a chapter followed
immediately by its first subsection is a container, not a false positive.
Measured on a 701-page process code: 683 of 2 762 marks are containers, and
Measured on a 701-page reference standard: 683 of 2 762 marks are containers, and
applying the check scores 2 079 of 2 761 boundaries instead of 2 762.
`sheet_section_rows` is D3's gate and it is OFF at False, where the scan is
@ -1559,7 +1559,7 @@ def _segment_path(candidate: Candidate, taken: set[str], prefix: str = "") -> st
def _link_shells(entries: list[dict[str, Any]], levels: Sequence[int], text: str) -> None:
"""Give each HEADING-ONLY entry the nearest ancestor holding text as `parent_id`.
A process code states its lettered points once, on the section that owns
A reference standard states its lettered points once, on the section that owns
them, and every section nested below inherits them. Built faithfully, the
nested section is a concept whose body is its heading line and nothing
else -- measured on one 2 761-concept standard, **710** of them -- and the

20
src/llm_ingestion_okf/quality.py
View file

@ -3,7 +3,7 @@
**This is not `okf check`, and the separation is the point.** `okf check` reads
a consumption skill and one payload against `docs/consumption-contract.md`: it
answers whether a payload carries what a claim must rest on. Measured
2026-09-10 by `vegnormal-okf` on three arms over one corpus, it returned 0
2026-09-10 by a downstream consumer repository on three arms over one corpus, it returned 0
findings and exit 0 on all three while their hit@k ranged from 6 of 6 to 0 of 6
-- a green contract check says nothing about whether the cut found anything
worth reading. This module asks that second question, and it is a SEPARATE
@ -55,7 +55,7 @@ CLI_ID = "okf quality"
#: The row a concept lands in when it declares no `source_file`. Not a file
#: type and never treated as one: measured 2026-09-12, three of the four
#: evidence corpora (`n100-2023`, `n200-2024`, `n500-2024`) carry the key on 0
#: evidence corpora (built by a producer other than this library) carry the key on 0
#: of 446, 0 of 1 133 and 0 of 270 concepts, because their producer is not this
#: library's Door B. A per-file-type gate has nothing to say about them, and
#: says that.
@ -166,7 +166,7 @@ def normalise_title(value: str) -> str:
Not a guess and not this module's invention: measured over the shipped
2 761-row fasit before any of this was written, the rule reproduces every
row's own `norm` from its own `title`, **2 761 of 2 761**. It is also the
normalisation `vegnormal-okf`'s measuring script applies, so a number
normalisation a downstream consumer's measuring script applies, so a number
produced here and a number produced there are the same number.
"""
return _WHITESPACE.sub("", value).lower()
@ -233,8 +233,8 @@ class BoundaryThreshold:
limit_recovered: int
limit_declared: int
#: Products behind the measurement. **One**, and it is printed on the row
#: rather than only recorded here: the fasit is R761's own NISO-STS
#: structure, so a bar read off it is pinned to one corpus and says nothing
#: rather than only recorded here: the fasit is one reference standard's own
#: NISO-STS structure, so a bar read off it is pinned to one corpus and says nothing
#: about a document nobody has a declared structure for.
corpora: int
source: str
@ -247,9 +247,9 @@ class BoundaryThreshold:
return f"{self.limit_recovered}/{self.limit_declared}"
#: Measured 2026-09-13 on `~/repos/vegnormal-okf/build/ferdig/r761-2025-generisk`,
#: the declared-structure (`.xml`) arm of R761 Prosesskoden:2025, against that
#: publisher's own 2 761 titled `<sec>` elements. Set at the value measured
#: Measured 2026-09-13 on one pinned reference build: the declared-structure
#: (`.xml`) arm of a 701-page reference standard, against that publisher's own
#: 2 761 titled `<sec>` elements. Set at the value measured
#: there rather than at a rounder number nearby, exactly like the two bars
#: above -- a REGRESSION bar against a pinned artifact, and a tight one:
#: `docs/2026-09-12-g37-terskler.md` SS 7 records that an older build of the
@ -261,8 +261,8 @@ BOUNDARY_THRESHOLD = BoundaryThreshold(
limit_declared=2761,
corpora=1,
source=(
"r761-2025-generisk against sk2-fasit-2761.json (R761 Prosesskoden:2025, "
"2 761 declared STS sections) -- ONE product, N = 1 corpus"
"one pinned reference build against its own declared structure "
"(2 761 declared STS sections) -- ONE product, N = 1 corpus"
),
)

24
src/llm_ingestion_okf/structure.py
View file

@ -37,15 +37,15 @@ from pathlib import Path
from .extract import strip_converter_attribute
from .profiles import STRUCTURED_BLOCK_KEYS, block_mapping_value, unquote_scalar
# A document number is either an alpha-prefixed identifier (`N500`, `V720`,
# `R610.4`) or a dotted numeric section (`4.2.1`). A BARE integer is
# A document number is either an alpha-prefixed identifier (`Q500`, `W720`,
# `P610.4`) or a dotted numeric section (`4.2.1`). A BARE integer is
# deliberately not a number: `12-things.md` and `2026-notes.md` are ordinary
# names, and admitting them would stamp a document number on most of a second
# brain that never had one. The trailing guard keeps `n500x` from reducing to
# `N500` — a partial match of a longer word is not an identifier. The guard is
# `\w` and NOT `[\w.]`: a number at the end of a sentence ("see N200.") is
# brain that never had one. The trailing guard keeps `q500x` from reducing to
# `Q500` — a partial match of a longer word is not an identifier. The guard is
# `\w` and NOT `[\w.]`: a number at the end of a sentence ("see Q200.") is
# followed by a full stop, and forbidding one there silently dropped every
# reference that happened to close a sentence. `R610.4` is unaffected because
# reference that happened to close a sentence. `P610.4` is unaffected because
# the dotted tail is greedy and consumes it first.
_NUMBER = r"(?:[A-Za-z]{1,3}\d{1,5}(?:\.\d{1,4})*|\d{1,4}(?:\.\d{1,4})+)(?!\w)"
_NUMBER_AT_START = re.compile(rf"^({_NUMBER})")
@ -59,7 +59,7 @@ _LINK = re.compile(r"\[[^\]]*\]\(([^)\s]+)\)")
# What makes a number token in running prose a POINTER rather than a quantity.
#
# The rule is positive — a closed set of cues — and not a blacklist, because a
# blacklist cannot work: measured 2026-08-29, `V221` and `V240` are genuine
# blacklist cannot work: measured 2026-08-29, `W221` and `W240` are genuine
# document numbers in one real corpus and `v0.3.0` is a software version in
# another, and the two are the same token shape. Nothing structural separates
# them; only the words in front of them do. Scanning every number instead
@ -173,7 +173,7 @@ def _split_frontmatter(text: str) -> tuple[dict[str, str], int]:
def _normalize_number(token: str) -> str:
# `n500` and `N500` are the same identifier written twice. Uppercasing the
# `q500` and `Q500` are the same identifier written twice. Uppercasing the
# alpha prefix is what lets a reference find its target without every
# consumer having to case-fold for itself.
return token.upper()
@ -222,8 +222,8 @@ def _scan_references(body: str, offset: int, own_number: str | None) -> tuple[st
into the index under the producer's name.
Link targets are collected first and their spans masked with spaces before
the number scan runs, so a link to `n500.md` yields the link target once
rather than the target plus a phantom `N500` read out of the URL. Masking
the number scan runs, so a link to `q500.md` yields the link target once
rather than the target plus a phantom `Q500` read out of the URL. Masking
with spaces rather than deleting keeps every later offset aligned, which is
what makes "first appearance" a property of the original text — and it is
also what keeps a cue from being read across a link it does not precede.
@ -641,8 +641,8 @@ def facet_values(name: str, bundle: BundleStructure, keys: Sequence[str]) -> dic
Relations are rendered as their SUBJECTS, each suffixed with
:data:`UNRESOLVED_MARKER` when the bundle holds nothing answering to it, so
an index reader sees the difference between "points at N200" and "points at
an N200 that is not here". `derived` gathers the document's own inferred
an index reader sees the difference between "points at Q200" and "points at
a Q200 that is not here". `derived` gathers the document's own inferred
fields plus any relation this library proposed rather than read.
"""
document = bundle.documents[name]

18
tests/fixtures/README.md vendored
View file

@ -17,7 +17,7 @@ and reproducible from that one file.
|---|---|
| `two-line-krav.pdf` | One heading plus one requirement row with label and value on the **same line**. That pairing is the property `pdfplumber` was chosen for. |
| `no-text-layer.pdf` | A structurally valid page with no text operators — the shape a scanned or image-only PDF presents. Must fail fast (`extractor_empty_pdf`), never persist as an empty concept. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape R761 carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: 2 763 nodes in, 2 762 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape a 701-page reference standard carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: 2 763 nodes in, 2 762 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `three-page-krav.pdf` | Three pages, one line of text each, and **the middle page carries no text operators**. The extractor drops empty pages, so the last page's text belongs to page 3 — which is what separates a page NUMBER from a count of the pages that produced text. Two pages could not tell those apart. |
## The XML fixtures
@ -31,9 +31,9 @@ round-trip-symmetric defect.
| Fixture | What it is for |
|---|---|
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in R761. The lettered points are what the 4 954 label-only `<sec>` in that document look like: promoted to headings they would bury its own 2 761. |
| `sts-empty-label.xml` | A `<sec>` carrying a `<label>` and **nothing else**, between a lettered point that has a body and the next titled section. The label is held as a prefix for a body line that never arrives, so it was overwritten and lost: measured on R761 that is exactly one `x)`, two characters of 1 283 395, ratio 0.999998. An exact invariant does not get to be 0.999998. |
| `sts-identity.xml` | A document that **states who it is**: exactly one `<std-ident>` with a `<doc-number>` and a `<year>`, one `<title-wrap>` whose `<full>` carries a **comma** (as R761's does, which is why that title cannot be written into a `sources` flow mapping verbatim), and a `<std-ref type="dated">`. Its body carries the `sec-type="spec"` shape the `description` rule reads: a titled `<sec>` whose first spec point has one `<p>`, a second spec point that must never become the description, a titled child with no spec point of its own, and a spec point with **two** `<p>` of which only the first counts. `sts-mini.xml` is the half identity (a `<title-wrap>`, no `<doc-number>`) and `sts-empty-label.xml` the absent one. |
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in that reference standard. The lettered points are what the 4 954 label-only `<sec>` in that document look like: promoted to headings they would bury its own 2 761. |
| `sts-empty-label.xml` | A `<sec>` carrying a `<label>` and **nothing else**, between a lettered point that has a body and the next titled section. The label is held as a prefix for a body line that never arrives, so it was overwritten and lost: measured on the reference standard that is exactly one `x)`, two characters of 1 283 395, ratio 0.999998. An exact invariant does not get to be 0.999998. |
| `sts-identity.xml` | A document that **states who it is**: exactly one `<std-ident>` with a `<doc-number>` and a `<year>`, one `<title-wrap>` whose `<full>` carries a **comma** (as the reference standard's does, which is why that title cannot be written into a `sources` flow mapping verbatim), and a `<std-ref type="dated">`. Its body carries the `sec-type="spec"` shape the `description` rule reads: a titled `<sec>` whose first spec point has one `<p>`, a second spec point that must never become the description, a titled child with no spec point of its own, and a spec point with **two** `<p>` of which only the first counts. `sts-mini.xml` is the half identity (a `<title-wrap>`, no `<doc-number>`) and `sts-empty-label.xml` the absent one. |
| `generic-feed.xml` | Known-negative: XML that is **not** STS. It must produce text and ONE plan — never zero, never a crash, and never element names promoted to headings. |
| `xml-doctype-bomb.xml` | Known-negative, security: a `<!DOCTYPE` with a small nested-entity expansion. It must be refused by `code`, and the test asserts the expansion appears in **no** output, including the error text. Small on purpose — the point is that it is never parsed, not that it detonates. |
| `xml-malformed.xml` | Known-negative: an unterminated tag must raise a typed `ExtractionError`, not leak `ParseError` and not yield zero concepts in silence. |
@ -208,13 +208,13 @@ python3 tests/fixtures/accounting/make_accounting_fixtures.py
| Fixture | What it carries that nothing else did |
|---|---|
| `topptekst-og-kommentar.docx` | A header, a footer, a comment, an endnote and a **text box** -- and a footnote, a table and a heading, three types the only other docx has at 0. The header says "Utkast - gjelder ikke etter 2026-01-01" and the comment says the requirement does NOT apply in tunnels: two statements that reverse the document's meaning and that the build carries none of. |
| `topptekst-og-kommentar.docx` | A header, a footer, a comment, an endnote and a **text box** -- and a footnote, a table and a heading, three types the only other docx has at 0. The header says "Utkast - gjelder ikke etter 2026-01-01" and the comment says the requirement does NOT apply in basements: two statements that reverse the document's meaning and that the build carries none of. |
| `notater-og-skjult.pptx` | A **speaker note** and a **hidden slide** (`show="0"`), plus a table and paragraphs. A hidden slide counted as an ordinary one is indistinguishable from one that is shown. |
| `skjult-ark-og-formel.xlsx` | A **hidden sheet**, a **formula** (`<f>B2*2</f>`) and a **picture**. The picture is what makes the operator's `.xlsx image` exception exercisable at all: the old fixture had none. |
| `liste-og-bilde.odt` | A **header and footer** (they live in `styles.xml`, so a reader of `content.xml` cannot see them), an **annotation**, a list and a picture. |
| `bilde.rtf` | A `\pict` picture: the rtf witness's image count was 0 in its only fixture. |
| `figur.html` | A picture and a table under `.html`; `side.htm` gained one too, so `.htm` and `.html` each exercise `image`. |
| `sts-rikt.xml` | A **`mixed-citation`**, an **`mml:math`**, a **`fig` with a caption**, a table with a label, cells, a list item and a footnote -- six STS roles the R761 delivery does not contain at all, which is why the gate's only real corpus could not see the hole in the role map. |
| `sts-rikt.xml` | A **`mixed-citation`**, an **`mml:math`**, a **`fig` with a caption**, a table with a label, cells, a list item and a footnote -- six STS roles a plain section-and-paragraph document does not carry, so each one of them reaches the role map. |
### The hand counts
@ -227,9 +227,7 @@ fixture's own bytes -- the XML parts of a zip, the control words of the rtf,
the objects of the PDF -- never by running the witness and writing down what
it said.
`witness/prosess-84-sts.twin.json` is the STS document written by hand in the
publisher's JSON node form (`standardContent`, nodes with `e`/`t`/`x`), so the
two STS witnesses can be compared on a fixture as well as on R761. Eight of
Eight of
the thirteen documents are byte copies of fixtures documented above
(`image-inbox/`, `k2-office/`, `prisark.xlsx`); the other five
(`notat.md`, `logg.txt`, `mengder.csv`, `parametre.json`, `side.htm`) are
@ -238,7 +236,7 @@ heading.
## What these fixtures do not cover
Structured table recovery. Measured on real Vegnormalene, only 45 of 196
Structured table recovery. Measured on a real regulatory reference corpus, only 45 of 196
detected table objects are clean enough to hand to `render_table` unchanged;
two independent parsers return the same wrong shape, because the breakage is in
the documents' ruling geometry rather than in either library. PDFs enter this

4
tests/fixtures/accounting/corpus/figur.html vendored
View file

@ -3,9 +3,9 @@
<head><title>Figur 84-1</title></head>
<body>
<h1>Figur 84-1</h1>
<p>Prinsippet for toleranseklasser.</p>
<p>Prinsippet for hevetidsklasser.</p>
<img src="graphics/figur-84-1.png" alt="Prinsippskisse">
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 mm</td></tr></table>
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 min</td></tr></table>
<ul><li>Klasse A</li><li>Klasse B</li></ul>
</body>
</html>

BIN
tests/fixtures/accounting/corpus/prosess-84-notat.docx → tests/fixtures/accounting/corpus/kapittel-84-notat.docx vendored
View file

Binary file not shown.

BIN
tests/fixtures/accounting/corpus/prosess-84-presentasjon.pptx → tests/fixtures/accounting/corpus/kapittel-84-presentasjon.pptx vendored
View file

Binary file not shown.

8
tests/fixtures/accounting/corpus/prosess-84-sts.xml → tests/fixtures/accounting/corpus/kapittel-84-sts.xml vendored
View file

@ -1,14 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>R761</doc-number></std-ident></front>
<front><std-ident><doc-number>P761</doc-number></std-ident></front>
<body>
<sec>
<label>84</label>
<title>Konstruksjoner av betong</title>
<title>Brød og boller</title>
<sec>
<label>84.1</label>
<title>Toleranser</title>
<p>Toleranseklasse er gitt i tabell 84-2.</p>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 84-2.</p>
<graphic xlink:href="graphics/tabell-84-2.png"/>
<p>Figur 84-1 viser prinsippet.</p>
<graphic xlink:href="figur-84-1.png"/>

BIN
tests/fixtures/accounting/corpus/prosess-84-tabell.pdf → tests/fixtures/accounting/corpus/kapittel-84-tabell.pdf vendored
View file

Binary file not shown.

8
tests/fixtures/accounting/corpus/prosess-84-web.html → tests/fixtures/accounting/corpus/kapittel-84-web.html vendored
View file

@ -1,9 +1,9 @@
<!doctype html>
<html><head><title>Prosess 84</title></head>
<html><head><title>Kapittel 84</title></head>
<body>
<h1>84 Konstruksjoner av betong</h1>
<p>Toleranseklasse for de enkelte konstruksjonsdeler er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Toleranseklasser">
<h1>84 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-84-1.png" alt="Figur 84-1 Prinsippskisse">

24
tests/fixtures/accounting/corpus/krav-rikt-tekstformat.rtf vendored
View file

@ -1,11 +1,11 @@
{\rtf1\ansi\ansicpg1252\deff0{\fonttbl{\f0\froman Times New Roman;}}\pard\sa180\b\fs32 Kravspesifikasjon for tunnelbelysning\b0\fs24\par
\pard\sa180 Dokumentet samler kravene til belysning i vegtunneler over 500 meter.\par
{\rtf1\ansi\ansicpg1252\deff0{\fonttbl{\f0\froman Times New Roman;}}\pard\sa180\b\fs32 Kravspesifikasjon for kj\u248 ?kkenbelysning\b0\fs24\par
\pard\sa180 Dokumentet samler kravene til belysning i storkj\u248 ?kken over 500 kvadratmeter.\par
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Dokumentnummer:\cell \pard\intbl SVV-2026-0417\cell \row
\pard\intbl Dokumentnummer:\cell \pard\intbl EKS-2026-0417\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Tittel:\cell \pard\intbl Tunnelbelysning i hovedl\u248 ?pet\cell \row
\pard\intbl Tittel:\cell \pard\intbl Kj\u248 ?kkenbelysning i hovedsalen\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Ansvarlig avdeling:\cell \pard\intbl Utbyggingsdivisjonen\cell \row
\pard\intbl Ansvarlig avdeling:\cell \pard\intbl Kj\u248 ?kkendivisjonen\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Fagomr\u229 ?de:\cell \pard\intbl Elektro og belysning\cell \row
\trowd\trgaph108\cellx3000\cellx6000
@ -13,21 +13,21 @@
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Gyldig fra:\cell \pard\intbl 01.03.2026\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Erstatter:\cell \pard\intbl SVV-2024-0188\cell \row
\pard\intbl Erstatter:\cell \pard\intbl EKS-2024-0188\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Tunnellengde:\cell \pard\intbl 1240 meter\cell \row
\pard\intbl Salens lengde:\cell \pard\intbl 42 meter\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Dimensjonerende fart:\cell \pard\intbl 80 km/t\cell \row
\pard\intbl Dimensjonerende gjester:\cell \pard\intbl 80 kuverter\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl \u197 ?rsd\u248 ?gntrafikk:\cell \pard\intbl 12400 kj\u248 ?ret\u248 ?y\cell \row
\pard\intbl \u197 ?rsproduksjon:\cell \pard\intbl 12400 porsjoner\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Terskelluminans:\cell \pard\intbl 145 candela\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Overgangssone:\cell \pard\intbl Tre trinn nedtrapping\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Innerstrekning:\cell \pard\intbl 3,0 candela\cell \row
\pard\intbl Innerste sone:\cell \pard\intbl 3,0 candela\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Utkj\u248 ?ringssone:\cell \pard\intbl Ingen forsterkning\cell \row
\pard\intbl Utgangssone:\cell \pard\intbl Ingen forsterkning\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Fargetemperatur:\cell \pard\intbl 4000 kelvin\cell \row
\trowd\trgaph108\cellx3000\cellx6000
@ -40,7 +40,7 @@
\pard\intbl M\u229 ?lemetode:\cell \pard\intbl M\u229 ?ling med luminanskamera\cell \row
\trowd\trgaph108\cellx3000\cellx6000
\pard\intbl Avviksh\u229 ?ndtering:\cell \pard\intbl S\u248 ?knad om fravik\cell \row
\pard\sa180 Luminansmatrise per trafikklasse\par
\pard\sa180 Luminansmatrise per romklasse\par
\trowd\trgaph108\cellx3000\cellx6000\cellx9000\cellx12000
\pard\intbl Sone\cell \pard\intbl Klasse A\cell \pard\intbl Klasse B\cell \pard\intbl Klasse C\cell \row
\trowd\trgaph108\cellx3000\cellx6000\cellx9000\cellx12000

BIN
tests/fixtures/accounting/corpus/krav-tekstdokument.odt vendored
View file

Binary file not shown.

BIN
tests/fixtures/accounting/corpus/liste-og-bilde.odt vendored
View file

Binary file not shown.

2
tests/fixtures/accounting/corpus/logg.txt vendored
View file

@ -1,4 +1,4 @@
Driftslogg for tunnelen.
Driftslogg for bakerovnen.
Vask utført i mars.
Lys kontrollert i april.

10
tests/fixtures/accounting/corpus/notat.md vendored
View file

@ -1,15 +1,15 @@
# Drift av tunneler
# Stell av bakerovner
Tunnelen driftes etter håndboka.
Ovnen stelles etter håndboka.
## Vask
Vask skjer to ganger i året.
| Sone | Intervall |
| Hylle | Intervall |
| --- | --- |
| Inngang | 6 mnd |
| Hovedløp | 12 mnd |
| Øverst | 6 mnd |
| Nederst | 12 mnd |
![Figur 1 Vaskeplan](graphics/tabell-84-2.png)

BIN
tests/fixtures/accounting/corpus/notater-og-skjult.pptx vendored
View file

Binary file not shown.

2
tests/fixtures/accounting/corpus/parametre.json vendored
View file

@ -1 +1 @@
{"tunnel": "Eksempeltunnelen", "lengde_m": 1240, "soner": ["inngang", "hovedlop"], "belysning": {"terskel_cd": 145, "fart_kmt": 80}}
{"ovn": "Eksempelovnen", "hoyde_cm": 124, "hyller": ["overst", "nederst"], "varme": {"grader_c": 145, "minutter": 80}}

12
tests/fixtures/accounting/corpus/side.htm vendored
View file

@ -1,10 +1,10 @@
<!doctype html>
<html><head><title>Side</title></head>
<body>
<h1>Belysning</h1>
<p>Terskelluminansen er 145 candela.</p>
<img src="graphics/tabell-84-2.png" alt="Sonekart">
<h2>Soner</h2>
<ul><li>Inngang</li><li>Hovedløp</li></ul>
<table><tr><th>Sone</th><th>Lengde</th></tr><tr><td>Inngang</td><td>120 m</td></tr></table>
<h1>Ovnen</h1>
<p>Steketemperaturen er 145 grader.</p>
<img src="graphics/tabell-84-2.png" alt="Hyllekart">
<h2>Hyller</h2>
<ul><li>Øverst</li><li>Nederst</li></ul>
<table><tr><th>Hylle</th><th>Temperatur</th></tr><tr><td>Øverst</td><td>180 C</td></tr></table>
</body></html>

16
tests/fixtures/accounting/corpus/sts-rikt.xml vendored
View file

@ -1,16 +1,16 @@
<standard>
<front><std-ident><doc-number>R762</doc-number><year>2025</year></std-ident></front>
<front><std-ident><doc-number>P762</doc-number><year>2025</year></std-ident></front>
<body>
<sec><label>85</label><title>Vegdekker</title>
<p>Dekket skal ha jevnhet etter <mixed-citation>NS-EN 13036-1:2010</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>IRI</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs vegbanen.</p></caption>
<sec><label>85</label><title>Bakeplater</title>
<p>Platen skal ha jevn varme etter <mixed-citation>EKS 1234-1:2030</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>T</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs bakeplaten.</p></caption>
<graphic xlink:href="figur-84-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<table-wrap><label>Tabell 85-1</label>
<table><tr><th>Klasse</th><th>IRI</th></tr><tr><td>1</td><td>1,5</td></tr></table>
<table><tr><th>Klasse</th><th>T</th></tr><tr><td>1</td><td>1,5</td></tr></table>
</table-wrap>
<list><list-item><p>Maales hvert 20. meter.</p></list-item></list>
<fn><p>Gjelder ikke gang- og sykkelveger.</p></fn>
<list><list-item><p>Maales hver 20. centimeter.</p></list-item></list>
<fn><p>Gjelder ikke steinovner.</p></fn>
</sec>
</body>
</standard>

BIN
tests/fixtures/accounting/corpus/topptekst-og-kommentar.docx vendored
View file

Binary file not shown.

796
tests/fixtures/accounting/inventory.json vendored
View file

File diff suppressed because it is too large Load diff

52
tests/fixtures/accounting/make_accounting_fixtures.py vendored
View file

@ -51,14 +51,14 @@ def build_zip(parts: dict[str, str | bytes]) -> bytes:
_DOCX_BODY = (
'<w:p><w:pPr><w:pStyle w:val="Heading1"/></w:pPr>'
"<w:r><w:t>Krav til gangbruer</w:t></w:r></w:p>"
"<w:p><w:r><w:t>Gangbruer skal ha rekkverk paa begge sider.</w:t></w:r></w:p>"
"<w:r><w:t>Krav til bakebrett</w:t></w:r></w:p>"
"<w:p><w:r><w:t>Bakebrett skal ha kant paa begge sider.</w:t></w:r></w:p>"
"<w:tbl><w:tr>"
"<w:tc><w:p><w:r><w:t>Bredde</w:t></w:r></w:p></w:tc>"
"<w:tc><w:p><w:r><w:t>3,0 m</w:t></w:r></w:p></w:tc>"
"<w:tc><w:p><w:r><w:t>0,4 m</w:t></w:r></w:p></w:tc>"
"</w:tr></w:tbl>"
"<w:p><w:r><w:pict><w:txbxContent>"
"<w:p><w:r><w:t>Merk: kravet gjelder ikke midlertidige bruer.</w:t></w:r></w:p>"
"<w:p><w:r><w:t>Merk: kravet gjelder ikke engangsbrett.</w:t></w:r></w:p>"
"</w:txbxContent></w:pict></w:r></w:p>"
)
@ -90,15 +90,15 @@ _DOCX_PARTS: dict[str, str | bytes] = {
+ f"<w:hdr {_W}><w:p><w:r><w:t>Utkast - gjelder ikke etter 2026-01-01</w:t></w:r></w:p>"
+ "</w:hdr>",
"word/footer1.xml": _XML
+ f"<w:ftr {_W}><w:p><w:r><w:t>Statens vegvesen, side 1</w:t></w:r></w:p></w:ftr>",
+ f"<w:ftr {_W}><w:p><w:r><w:t>Eksempelforlaget, side 1</w:t></w:r></w:p></w:ftr>",
"word/comments.xml": _XML
+ f"<w:comments {_W}>"
+ '<w:comment w:id="1"><w:p><w:r><w:t>Unntak: gjelder IKKE gangbruer i tunnel.'
+ '<w:comment w:id="1"><w:p><w:r><w:t>Unntak: gjelder IKKE bakebrett i kjeller.'
+ "</w:t></w:r></w:p></w:comment></w:comments>",
"word/footnotes.xml": _XML
+ f"<w:footnotes {_W}>"
+ '<w:footnote w:id="0"><w:p><w:r><w:t>separator</w:t></w:r></w:p></w:footnote>'
+ '<w:footnote w:id="2"><w:p><w:r><w:t>Se haandbok N400 kapittel 5.</w:t></w:r></w:p>'
+ '<w:footnote w:id="2"><w:p><w:r><w:t>Se kokebok Q400 kapittel 5.</w:t></w:r></w:p>'
+ "</w:footnote></w:footnotes>",
"word/endnotes.xml": _XML
+ f"<w:endnotes {_W}>"
@ -164,11 +164,11 @@ _PPTX_PARTS: dict[str, str | bytes] = {
+ f"<p:presentation {_P}><p:sldIdLst>"
+ '<p:sldId id="256" r:id="rId1"/><p:sldId id="257" r:id="rId2"/>'
+ "</p:sldIdLst></p:presentation>",
"ppt/slides/slide1.xml": _slide("Prosess 84 Konstruksjoner", "Toleranser er gitt i tabell."),
"ppt/slides/slide1.xml": _slide("Kapittel 84 Boller", "Hevetider er gitt i tabell."),
"ppt/slides/slide2.xml": _slide("Utgaatt lysbilde", "Ikke vis dette.", hidden=True),
"ppt/notesSlides/notesSlide1.xml": _XML
+ f"<p:notes {_P} {_A}><p:cSld><p:spTree><p:sp><p:txBody>"
+ "<a:p><a:r><a:t>Husk aa nevne at toleranseklassen er skjerpet.</a:t></a:r></a:p>"
+ "<a:p><a:r><a:t>Husk aa nevne at hevetidsklassen er skjerpet.</a:t></a:r></a:p>"
+ "</p:txBody></p:sp></p:spTree></p:cSld></p:notes>",
}
@ -267,17 +267,17 @@ _ODT_PARTS: dict[str, str | bytes] = {
"content.xml": _XML
+ f'<office:document-content {_ODT_NS} office:version="1.3">'
+ "<office:body><office:text>"
+ '<text:h text:outline-level="1">Drift av gangbruer</text:h>'
+ "<text:p>Gangbruer inspiseres hvert aar.</text:p>"
+ "<text:list><text:list-item><text:p>Rekkverk</text:p></text:list-item>"
+ "<text:list-item><text:p>Dekke</text:p></text:list-item></text:list>"
+ '<text:h text:outline-level="1">Stell av bakebrett</text:h>'
+ "<text:p>Bakebrett vaskes hver uke.</text:p>"
+ "<text:list><text:list-item><text:p>Kant</text:p></text:list-item>"
+ "<text:list-item><text:p>Flate</text:p></text:list-item></text:list>"
+ "<text:p>Se figuren under."
+ '<draw:frame><draw:image xlink:href="graphics/figur-84-1.png"/></draw:frame></text:p>'
+ "<office:annotation><text:p>Sjekk denne mot N400 foer utsendelse.</text:p>"
+ "<office:annotation><text:p>Sjekk denne mot Q400 foer utsendelse.</text:p>"
+ "</office:annotation>"
+ "<table:table><table:table-row>"
+ "<table:table-cell><text:p>Type</text:p></table:table-cell>"
+ "<table:table-cell><text:p>Gangbru</text:p></table:table-cell>"
+ "<table:table-cell><text:p>Bakebrett</text:p></table:table-cell>"
+ "</table:table-row></table:table>"
+ "</office:text></office:body></office:document-content>",
"styles.xml": _XML
@ -285,7 +285,7 @@ _ODT_PARTS: dict[str, str | bytes] = {
+ "<office:styles/><office:master-styles>"
+ '<style:master-page style:name="Standard">'
+ "<style:header><text:p>Intern arbeidsversjon</text:p></style:header>"
+ "<style:footer><text:p>Vegdirektoratet</text:p></style:footer>"
+ "<style:footer><text:p>Eksempelforlaget</text:p></style:footer>"
+ "</style:master-page></office:master-styles></office:document-styles>",
}
@ -307,9 +307,9 @@ _HTML = """<!DOCTYPE html>
<head><title>Figur 84-1</title></head>
<body>
<h1>Figur 84-1</h1>
<p>Prinsippet for toleranseklasser.</p>
<p>Prinsippet for hevetidsklasser.</p>
<img src="graphics/figur-84-1.png" alt="Prinsippskisse">
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 mm</td></tr></table>
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 min</td></tr></table>
<ul><li>Klasse A</li><li>Klasse B</li></ul>
</body>
</html>
@ -319,18 +319,18 @@ _HTML = """<!DOCTYPE html>
# --- sts: a citation, a formula, a figure with a caption, a table, a footnote -
_STS = """<standard>
<front><std-ident><doc-number>R762</doc-number><year>2025</year></std-ident></front>
<front><std-ident><doc-number>P762</doc-number><year>2025</year></std-ident></front>
<body>
<sec><label>85</label><title>Vegdekker</title>
<p>Dekket skal ha jevnhet etter <mixed-citation>NS-EN 13036-1:2010</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>IRI</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs vegbanen.</p></caption>
<sec><label>85</label><title>Bakeplater</title>
<p>Platen skal ha jevn varme etter <mixed-citation>EKS 1234-1:2030</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>T</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs bakeplaten.</p></caption>
<graphic xlink:href="figur-84-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<table-wrap><label>Tabell 85-1</label>
<table><tr><th>Klasse</th><th>IRI</th></tr><tr><td>1</td><td>1,5</td></tr></table>
<table><tr><th>Klasse</th><th>T</th></tr><tr><td>1</td><td>1,5</td></tr></table>
</table-wrap>
<list><list-item><p>Maales hvert 20. meter.</p></list-item></list>
<fn><p>Gjelder ikke gang- og sykkelveger.</p></fn>
<list><list-item><p>Maales hver 20. centimeter.</p></list-item></list>
<fn><p>Gjelder ikke steinovner.</p></fn>
</sec>
</body>
</standard>

216
tests/fixtures/accounting/witness/prosess-84-sts.twin.json vendored
View file

@ -1,216 +0,0 @@
{
"productInfo": {
"meta": {
"productTitle": "R761 fixture twin"
}
},
"standardContent": {
"id": null,
"tag": "#document",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "standard",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "front",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "std-ident",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "doc-number",
"a": {},
"c": [
{
"e": 2,
"t": "R761",
"x": null
}
]
}
}
]
}
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "body",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "sec",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "label",
"a": {},
"c": [
{
"e": 2,
"t": "84",
"x": null
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "title",
"a": {},
"c": [
{
"e": 2,
"t": "Konstruksjoner av betong",
"x": null
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "sec",
"a": {},
"c": [
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "label",
"a": {},
"c": [
{
"e": 2,
"t": "84.1",
"x": null
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "title",
"a": {},
"c": [
{
"e": 2,
"t": "Toleranser",
"x": null
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "p",
"a": {},
"c": [
{
"e": 2,
"t": "Toleranseklasse er gitt i tabell 84-2.",
"x": null
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "graphic",
"a": {
"xlink:href": "graphic_0001.png"
},
"c": []
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "p",
"a": {},
"c": [
{
"e": 2,
"t": "Figur 84-1 viser prinsippet.",
"x": null
}
]
}
},
{
"e": 1,
"t": null,
"x": {
"id": null,
"tag": "graphic",
"a": {
"xlink:href": "graphic_0002.png"
},
"c": []
}
}
]
}
}
]
}
}
]
}
}
]
}
}
]
}
}

23
tests/fixtures/accounting/witness/sts-label-in-title.twin.json vendored
View file

@ -1,23 +0,0 @@
{
"productInfo": {"meta": {"productTitle": "Label inside title"}},
"standardContent": {
"id": null, "tag": "#document", "a": {},
"c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "standard", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "body", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "sec", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "title", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "label", "a": {}, "c": [
{"e": 2, "t": "84.1", "x": null}
]}},
{"e": 2, "t": "Toleranser", "x": null}
]}},
{"e": 1, "t": null, "x": {"id": null, "tag": "p", "a": {}, "c": [
{"e": 2, "t": "Toleranseklasse er gitt i tabell 84-2.", "x": null}
]}}
]}}
]}}
]}}
]
}
}

37
tests/fixtures/accounting/witness/sts-mml-and-figcaption.twin.json vendored
View file

@ -1,37 +0,0 @@
{
"productInfo": {"meta": {"productTitle": "Formula and figure caption"}},
"standardContent": {
"id": null, "tag": "#document", "a": {},
"c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "standard", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "body", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "sec", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "title", "a": {}, "c": [
{"e": 2, "t": "Vegdekker", "x": null}
]}},
{"e": 1, "t": null, "x": {"id": null, "tag": "p", "a": {}, "c": [
{"e": 2, "t": "Kravet regnes som ", "x": null},
{"e": 1, "t": null, "x": {"id": null, "tag": "inline-formula", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "mml:math", "a": {}, "c": [
{"e": 2, "t": "IRI < 2", "x": null}
]}}
]}}
]}},
{"e": 1, "t": null, "x": {"id": null, "tag": "fig", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "graphic", "a": {"href": "figur-85-1.png"}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "figcaption", "a": {}, "c": [
{"e": 1, "t": null, "x": {"id": null, "tag": "graphic-caption-label", "a": {}, "c": [
{"e": 2, "t": "Figur 85-1", "x": null}
]}},
{"e": 1, "t": null, "x": {"id": null, "tag": "graphic-caption-title", "a": {}, "c": [
{"e": 2, "t": "Maalepunkter langs vegbanen.", "x": null}
]}}
]}}
]}}
]}}
]}}
]}}
]}}
]
}
}

4
tests/fixtures/font-heading-krav.pdf vendored
View file

@ -12,9 +12,9 @@ endobj
<< /Length 221 >>
stream
BT /F2 20 Tf 50 700 Td (Generelle tekniske krav) Tj ET
BT /F1 10 Tf 50 670 Td (Utkilingen skal ha helning 1:15.) Tj ET
BT /F1 10 Tf 50 670 Td (Deigkulene skal ha hevetid 1:15.) Tj ET
BT /F2 14 Tf 50 640 Td (Merking) Tj ET
BT /F1 10 Tf 50 610 Td (Kravet gjelder alle veiklasser.) Tj ET
BT /F1 10 Tf 50 610 Td (Kravet gjelder alle bakeformer.) Tj ET
endstream
endobj
5 0 obj

8
tests/fixtures/generic-feed.xml vendored
View file

@ -3,12 +3,12 @@
<title>Driftsmeldinger</title>
<description>Meldinger om drift og vedlikehold.</description>
<item>
<title>Stengt tunnel</title>
<description>Tunnelen er stengt for vedlikehold natt til fredag.</description>
<title>Stengt kantine</title>
<description>Kantinen er stengt for vedlikehold natt til fredag.</description>
</item>
<item>
<title>Redusert framkommelighet</title>
<description>Ett felt er stengt i anleggsperioden.</description>
<title>Redusert åpningstid</title>
<description>Ett kjøkken er stengt i ombyggingsperioden.</description>
</item>
</channel>
</rss>

BIN
tests/fixtures/image-inbox/prosess-84-notat.docx → tests/fixtures/image-inbox/kapittel-84-notat.docx vendored
View file

Binary file not shown.

BIN
tests/fixtures/image-inbox/prosess-84-presentasjon.pptx → tests/fixtures/image-inbox/kapittel-84-presentasjon.pptx vendored
View file

Binary file not shown.

8
tests/fixtures/image-inbox/prosess-84-sts.xml → tests/fixtures/image-inbox/kapittel-84-sts.xml vendored
View file

@ -1,14 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>R761</doc-number></std-ident></front>
<front><std-ident><doc-number>P761</doc-number></std-ident></front>
<body>
<sec>
<label>84</label>
<title>Konstruksjoner av betong</title>
<title>Brød og boller</title>
<sec>
<label>84.1</label>
<title>Toleranser</title>
<p>Toleranseklasse er gitt i tabell 84-2.</p>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 84-2.</p>
<graphic xlink:href="graphics/tabell-84-2.png"/>
<p>Figur 84-1 viser prinsippet.</p>
<graphic xlink:href="figur-84-1.png"/>

BIN
tests/fixtures/image-inbox/prosess-84-tabell.pdf → tests/fixtures/image-inbox/kapittel-84-tabell.pdf vendored
View file

Binary file not shown.

8
tests/fixtures/image-inbox/prosess-84-web.html → tests/fixtures/image-inbox/kapittel-84-web.html vendored
View file

@ -1,9 +1,9 @@
<!doctype html>
<html><head><title>Prosess 84</title></head>
<html><head><title>Kapittel 84</title></head>
<body>
<h1>84 Konstruksjoner av betong</h1>
<p>Toleranseklasse for de enkelte konstruksjonsdeler er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Toleranseklasser">
<h1>84 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-84-1.png" alt="Figur 84-1 Prinsippskisse">

12
tests/fixtures/k2-office-fasit.json vendored
View file

@ -55,7 +55,7 @@
"krav-rikt-tekstformat.rtf": {
"authored_titles": 1,
"titles": [
"Kravspesifikasjon for tunnelbelysning"
"Kravspesifikasjon for kjøkkenbelysning"
],
"bold_runs_that_are_not_titles": 0,
"declares_heading_style": false,
@ -64,8 +64,8 @@
"krav-rikt-tekstformat-fet-alene.rtf": {
"authored_titles": 2,
"titles": [
"Kravspesifikasjon for tunnelbelysning",
"Luminanskrav per trafikklasse"
"Kravspesifikasjon for kjøkkenbelysning",
"Luminanskrav per romklasse"
],
"bold_runs_that_are_not_titles": 0,
"declares_heading_style": false,
@ -74,7 +74,7 @@
"krav-rikt-tekstformat-fet-i-avsnitt.rtf": {
"authored_titles": 1,
"titles": [
"Kravspesifikasjon for tunnelbelysning"
"Kravspesifikasjon for kjøkkenbelysning"
],
"bold_runs_that_are_not_titles": 1,
"declares_heading_style": false,
@ -83,8 +83,8 @@
"krav-rikt-tekstformat-stil.rtf": {
"authored_titles": 2,
"titles": [
"Kravspesifikasjon for tunnelbelysning",
"Luminanskrav per trafikklasse"
"Kravspesifikasjon for kjøkkenbelysning",
"Luminanskrav per romklasse"
],
"bold_runs_that_are_not_titles": 0,
"declares_heading_style": true,

BIN
tests/fixtures/k2-office/krav-presentasjon.pptx vendored
View file

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show more