llm-ingestion-okf/docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md

K3 round 19: a document's own identity, per-run frontmatter, and a description from the source

Date: 2026-09-11 · Base: 56b00e7 · Commits: be169ee (red), ee8d5b5, 72617ac (red), 912b850, 0dbc331 (red), de7849e, 31ffc2c (red), 77e7cab · Measured from: a frozen git archive export of 77e7cab, never the working tree and never the okf on PATH.

A NISO-STS document built with okf build landed every concept under a directory named for its delivery file, and every sources entry named that file twice. A consumer's own build of the same kind of document carried an edition key, a publisher's address and a description, and okf build could carry none of them. This round closes those four points in general code: no name of a document, a publisher or a corpus appears in src/.

0. Where the measurement differs from the order, first

• HEAD was 56b00e7, not the e15de71 the order recorded. The R4 release landed after the order was written. No commit since e15de71 touched inbox.py, propose.py, cli.py, extract.py, profiles.py or tests/test_xml_sts.py (empty git log, rc 0), and uv.lock was clean, so nothing had to be checked out.
• The known-positive falls, and the directory name is the whole cause. S1–S6 hold at 6/6 · 6/6 · 6/6 at both k, as the order required. The known-positive (KP) row went from rank 1 at both k on the base to not delivered at the default k and rank 13 at k = 50. Decomposed in § 7: the same bundle with only the directory renamed back to the file name puts KP at rank 1 again, and the same bundle with every description removed does not move it.
• The order's own --frontmatter 'sources=...' value is not YAML that PyYAML reads. With it, 2 761 of 2 761 concepts' frontmatter fail yaml.safe_load (a : inside a flow-mapping plain scalar, in the URL and in R761:2025). The consumer bundle the order modelled it on has the same property: 446 of 446 of its sources lines fail the same way. okf check accepts both, with 0 findings. The flag writes the operator's value verbatim, as ordered; this is reported, not repaired.
• 1 807 descriptions are written, not 2 026. 2 026 titled sections carry a direct-child spec point. 2 of those points have no <p>, and 217 carry ": ", which a YAML reader refuses in description: a: b. The first build wrote them and made 217 frontmatters unreadable to PyYAML. That was repaired before delivery (31ffc2c red, 77e7cab); § 5.
• The 43-document reference corpus was not rebuilt. It holds 0 .xml of 86 files (known-positive in the same query: 66 .pdf), and every change here is reached only through .xml or through a flag the build did not receive. The pinned bundle's test runs as before (§ 8).

1. The spec, and what it does not say

Checked against the canonical text first (docs/okf-nokkelinventar.md § 0.1): curl of upstream okf/SPEC.md exit 0, 37 748 bytes, diff against the local copy 0 lines, 1 006 lines, "Version 0.2".

• § 4.1, l. 205: "Extensions: Producers MAY include any additional keys. Consumers SHOULD preserve unknown keys when round-tripping and MUST NOT reject documents with unrecognized fields."
• § 11, l. 760: "Unknown additional frontmatter keys." among what a consumer MUST NOT reject a bundle for.
• § 4.1 on description: "A single sentence summarizing the concept." No length limit exists in § 4.1, § 8 or § 11 (grep -n description over the spec is the denominator: 17 lines, none a limit). The one-paragraph limit in § 5 is this package's own.

Conformance is the floor and never the proof: every number below that says "okf check 0 findings" is a statement about the contract checker, and § 0 already has one bundle it accepts that a YAML parser does not.

2. (a) The document's identity names its directory and titles its address

extract.declared_identity(name, data) reads what a NISO-STS document states about itself: exactly one <std-ident> (<doc-number>, <year>) and exactly one <title-wrap> (<full>, else <main>). It is None for every other file type, for XML that is not STS, for an unparseable file and for a document stating neither. A value stated more than once is not read: an adopted standard carries one <std-ident> per issuing body, and taking the first is a guess.

Measured on R761: 1 <std-ident>, 1 <doc-number> (R761 Prosesskoden), 1 <title-wrap>, <year> 2025. The delivery file's stem occurs 0 times in the document. <doc-type> reads Innledning, the name of a chapter, and nothing uses it.

The directory. okf build replaces the file's stem with the <doc-number> reduced to the id grammar and keeps the folders above it: r761-prosesskoden/, where the base wrote the UUID.

A declared name two documents claim is used by neither. If a second document in the run declares the same name, or its file name reduces to it, both keep their file name, and stderr says which documents and which name. The collision gate Door B already has would refuse both with "rename one", and a name read from inside a document is not one a rename can change. Tested with two documents sharing a <doc-number> and with a document whose file name holds the declared name.

The sources title. The candidates are <doc-number> + <year>, then the <title-wrap> title, then the file name. The first that passes the gate and can be written into the flow mapping verbatim wins. The order left the choice between the first two to measurement:

sources[0].title	hit@k S1–S6, both k	KP (default / 50)	bytes spent over 8 questions (default / 50)	excerpts delivered at k = 50
R761 Prosesskoden 2025 (shipped)	1,1,1,1,1,1	– / 13	278 693 / 853 803	348
<full> with its commas removed	1,1,1,1,1,1	– / 13	282 289 / 854 200	346

They rank identically. consume passes sources through to the excerpt and ranks on neither. The full title costs 3 596 bytes more at the default k and two excerpts at k = 50. The order's rule for a tie is the shorter one. The deciding fact is structural anyway: R761's <full> carries commas, which end a flow mapping, and the guard refuses the quoted scalar that could carry them. It cannot be written verbatim, so the comma-free row exists only in this measurement. A title is never cleaned up, because a title with its commas removed is not the document's title.

resource stays the inbox-relative file. The address says where the bytes are; the title says what they call themselves.

3. (b) --frontmatter KEY=VALUE

One form, repeatable, and a --frontmatter-file was not built. The consumer bundle measured carries eight keys of its own, and eight flags are still readable. A file would be a second parser for the same mapping.

• Split on the FIRST =, and only there. A publisher's address carries ?languageCode=nb.
• Written verbatim on ONE line. This package's readers are line-oriented, and a block-form sources is invisible to parse_frontmatter (measured in vegnormal-okf 2026-09-08 and cited by the order). The flow form is the only one that survives our own readers. The test reads a stated sources back through both parse_frontmatter and consume.read_sources.
• Precedence: flag > what the document declares > file name. A run may ADD any key and REPLACE only sources and description, the two keys with a layer below the flag. Tested on both layers. A stated sources gives exactly one sources line, on the STS document and on a markdown one.
• Refused before a proposal is written (run_frontmatter_invalid): every key the door writes itself (inbox._door_keys). That covers the hash, the offsets, the locators, type, ingested_at, the bundle id, the ownership stamp, the derived facets and derived itself. It also covers Door A's ingest_manifest, which would make that door claim a Door B file. Also refused: a key outside [A-Za-z_][A-Za-z0-9_-]*, and a value that is empty, spans lines or has surrounding whitespace, since parse_frontmatter strips it. A key given twice is refused rather than resolved.
• Nothing moves without it. A test holds the flagged tree to the plain one minus exactly the stated line. okf project does not take the flag, because it owns no flag that moves a bundle's bytes.

4. (c) and the YAML repair

The NISO-STS reader records, per titled <sec>, the first <p> of the first direct-child <sec sec-type="spec">, whole. The plan entry carries it beside title, because a plan is the record a rebuild replays. The door writes it after the gate has seen it: it is document text persisted outside the body the gate screens, so it is kept only on the non-blocking floor and only as the sanitized text. A section with no such point gets no key. Nothing is derived from the title.

Measured on R761: 2 026 of 2 761 titled sections carry a direct-child spec point. 264 of those points hold more than one <p> and 2 hold none. The first <p> runs 17 / 109 / 273 / 521 / 942 characters at min / median / p90 / p99 / max. A character cap was not chosen, because a cut inside a paragraph writes a sentence the source never wrote.

The repair. The first build from de7849e wrote 2 024 descriptions, and PyYAML's safe_load refused 218 of 2 761 frontmatters: 217 on description, all carrying ": ", and 1 on a title the base already had. inbox._yaml_plain is now checked where the value is written. It refuses a leading YAML indicator, ": ", " #", a trailing :, a tab or a line break. Over the 2 024 measured values the rule and PyYAML agree on every one: 217 refused, 0 refused that PyYAML reads, 0 kept that it does not. Omitted rather than quoted, because a quoted value comes back from our line readers with its quotes. After: 1 807 descriptions, and 1 of 2 761 frontmatters refused by PyYAML, which is the base's own title.

That title class is older than this round and is not touched here: 41 of 436 concepts on the pinned 43-document bundle fail safe_load on title.

5. (d) --ingested-at alone: confirmed, nothing built

Built with --ingested-at 2026-09-08T12:00:00Z and no --proposed-at: 2 761 of 2 761 concepts carry that stamp, on the segmented route where the door reads the plan's proposed_at, and log.md is dated ## 2026-09-08. The base carried the same, so the 1970-01-01T00:00:00Z in the consumer's bundle came from a build that passed no stamp. The flag works as specified. tests/test_cli_build.py::test_ingested_at_alone_stamps_every_concept_the_same already held it.

6. V1 FERDIG — R761, base 56b00e7 against 77e7cab

Built with --bundle-id r761-prosesskoden-2025 --okf-version 0.2 --ingested-at 2026-09-08T12:00:00Z. The flagged build adds --frontmatter utgave=R761:2025 and a sources value addressing the publisher's viewer.

row	base	after	denominator
document directory	14ce59dc-… (the file's UUID)	r761-prosesskoden	1
concepts	2 761	2 761	—
conservation	110 = 110	110 = 110	N = 110 (109 images extractor_unknown, expected)
shells (body with no non-heading line)	710	710	2 761
sources[0].title, no flag	the file name	R761 Prosesskoden 2025	2 761
sources, flagged	—	the stated value, one line	2 761
utgave, flagged	—	2 761	2 761
description	0	1 807	2 026 with a spec point
ingested_at = the stamp	2 761	2 761	2 761
frontmatter refused by PyYAML, no flag	1 (title)	1 (title)	2 761
hit@1 · hit@8 · hit@50, S1–S6, default k	6/6 · 6/6 · 6/6	6/6 · 6/6 · 6/6	6
hit@1 · hit@8 · hit@50, S1–S6, k = 50	6/6 · 6/6 · 6/6	6/6 · 6/6 · 6/6	6
KP rank, default k / k = 50	1 / 1	not delivered / 13	1
okf check, KP and S5 × both k	—	4 of 4, 16 rules, 0 findings, exit 0	4

hit@k is vegnormal-okf's experiments/hitk_sk2.py, run read-only against the scratch bundle with its five flags. Its payloads land at fixed /tmp/hitk-sk2-<id>.json paths and were moved out between runs. The same cells hold on the flagged and the unflagged build. On de7849e, okf check was also run over all 8 default-k payloads: 8 of 8, 0 findings.

7. The known-positive, decomposed

The KP question contains the document's own number and name. consume's signal 1 is title + concept_id.replace("/", " "), so once every concept id starts with r761-prosesskoden/, every concept matches those two question tokens, and the gold concept's distinguishing match on its own title disappears into a 2 761-way tie. With the UUID directory, the id matched nothing.

variant (all from de7849e, unflagged)	KP default k	KP k = 50	S1–S6
as built	not delivered	13	all rank 1
V1: every description line removed	not delivered	13	all rank 1
V2: only the directory renamed back to the UUID (root index link rewritten to match)	1	1	all rank 1
as built, okf consume --rarity-weight	4	4	all rank 1
base 56b00e7, --rarity-weight	6	6	all rank 1

--rarity-weight weighs a token by log(N/df), and a token in every concept weighs 0. It delivers KP at the default k on this bundle with S1–S6 unmoved. It stays off: it is a ranking decision this round was fenced out of, and its wider measurement (docs/2026-09-08-sjeldenhetsvekt.md) was mixed. Which way to go is the operator's. The options are to keep the directory and repair the id-segment signal for a segment every concept shares, or to accept the row.

8. Nothing else moved

• Five-document folder. 5 = 5, 52 md. diff -r against the base build differs in one line of one file: the Converter line in log.md, which records the absolute path of the venv that ran it, and the two exports sit in different directories. With that line ignored (diff -r -I), rc 0. The folder holds 0 .xml, so it is a real known-negative.
• The four goldens. tests/test_segmented_golden.py is green in the working tree. In the export its fourth test is the one known red: it runs git status, and an export has no repository.
• The pinned 43-document bundle. tests/test_default_bundle_pin.py, 7 passed in the export with GOLD_SET copied in.
• Suite. 1 602 → 1 658 passed, 1 skipped (56 new). ruff check src tests tools, ruff format --check . and mypy --strict src are clean, each with its rc captured directly, after git add and after commit.
• Nothing was written in vegnormal-okf. Its git status is empty, and the two hitk_sk2 bytecode files in its ignored __pycache__ date from 2026-09-10.

Honesty limits

• N = 1 STS document. "General" is a claim about the code: no document, publisher or corpus is named in src/, and every rule is keyed to a schema element. It is not a measurement over several documents. The fixtures are hand-written from the measured document's shapes.
• The title choice rests on hit@k and bytes, not on what reads well, and hit@k could not separate the two candidates at all. The tie-break and the flow-terminator fact decided it.
• The one-paragraph description limit is ours. The spec asks for "a single sentence" and sets no length. 942 characters is not one sentence, and it is what the source wrote.
• The YAML rule is measured on 2 024 values from one document against one parser (PyYAML 6.0.3, YAML 1.1). A YAML 1.2 reader is not measured.
• KP's fall is measured on one question. Whether a question that names its own single-document bundle is common is not measured.
• okf check accepting a bundle is not a quality gate. § 0 has a bundle it accepts on 2 761 of 2 761 concepts that PyYAML refuses on the same count.
• The 43-document corpus was reasoned out, not rebuilt: 0 .xml of 86, and the five-document folder is the measured known-negative in its place.