llm-ingestion-okf/docs/2026-09-11-k3-runde22-yaml-lesbar-frontmatter.md

K3 round 22 — frontmatter a YAML reader reads back the same

Order K3-22, 2026-09-11. Red 06e61a5, fix ed0418f, base 0308169 (v0.8.3). PyYAML 6.0.3, guard 1.3.0, Python 3.14.0 (working tree and the frozen exports, run through the repository's own interpreter with the export first on PYTHONPATH; __file__ verified under the scratch export for both).

Deviations, first

1. The order's form for the flow mapping cannot ship. It asked for sources: [{ resource: "https://…?x=nb", title: … }] -- a double-quoted leaf inside the flow mapping. PyYAML reads that; the pinned guard does not. Guard 1.3.0 okf.parse_frontmatter refuses ANY quote inside a flow mapping ("a quoted scalar inside a flow mapping is not a supported form"), double or single, measured below. Writing it would have moved 4 605 PyYAML failures to 4 605 guard failures, and broken the promise tests/test_provenance.py already holds ("what Door B writes must survive the guard's own frontmatter grammar"). So a flow leaf is VALIDATED, never quoted: a value no flow form both readers accept is refused with the door's existing code. Quoting is used where the guard admits it -- a block scalar.
2. The failure is bigger in this repository's own output than the order's table shows, and in the other place. The order's table has no bundle okf built with its defaults. The pinned K2 default bundle fails safe_load on 41 of 455 frontmatter blocks, and 1 more parses while PyYAML reads a different value than ours (a title carrying " #" -- PyYAML stops at the comment, no error). All 42 are block title scalars (FEIL B); 0 sources values okf wrote failed anywhere measured. The 4 605 ? failures (FEIL A) come from a consumer's own sources writer, which does not go through this library.
3. The consumer repository is at f783e52, not 6fad6a2. The hit@k instrument was taken from 6fad6a2, as ordered, into private scratch with the payload path changed (2 lines differ from the original). Its tree was clean before and after (git status --porcelain: 0 lines both times).
4. The default-k hit@k run passes four flags, not five (--bundle --questions --okf --out, no --k), reproducing how the previous round ran it; the k = 50 run passes all five.

1. The standard

SPEC (_okf-canonical ad30107), verbatim:

• § 4, opening: "Every concept is a UTF-8 markdown file with two parts:
  1. A YAML frontmatter block, delimited by --- on its own line at the start of the file and a closing --- on its own line." (l. 155-158)
• § 11, Conformance, point 1: "Every non-reserved .md file in the tree contains a parseable YAML frontmatter block." (l. 740-741)

What the spec does not say, and it is a weakness of the spec: it names no YAML version and no subset -- 0 hits for a version number, "YAML version" or "subset", with the query validated against the known-positive YAML (5 hits). "Parseable" is therefore whatever reader the consumer has, and PyYAML is the common one. That is why the reader, not this corpus, decides the rule.

2. The measurement, reproduced

yaml.safe_load over the block between the two fences; the denominator is files WITH a frontmatter block (index files without one are not counted -- e.g. 5 514 .md against 2 757 with frontmatter in the first row).

tree	with frontmatter	OK	FAIL	order's figure
consumer R761 (N-route)	2 757	1	2 756	same
consumer N100	447	1	446	same
consumer N200	1 134	1	1 133	same
consumer N500	271	1	270	same
consumer R761, built by okf build	2 763	2 762	1	same
this repository's tests/fixtures	12	12	0	same
K2 default bundle (pinned)	455	414	41 (+1 misread)	not in the order
R761 HTML build (v0.8.3)	3 208	3 207	1	not in the order
R761 XML build (v0.8.3)	2 763	2 762	1	not in the order
five-document project (v0.8.3)	29	29	0	not in the order
examples/ · skills/	14 · 3	14 · 3	0 · 0	not in the order

Every figure the order gave reproduces exactly.

3. The two causes

FEIL A -- ? inside a flow mapping. The consumer writes sources: [{ resource: https://<viewer>/api/…/860019?languageCode=nb, title: … }] unquoted. PyYAML's scanner ends a plain scalar in FLOW context at ?:

while parsing a flow mapping ... expected ',' or '}', but got '?'

The colon in https: is not the cause: https://h.no/a/1 in the same place parses.

FEIL B -- a block scalar written verbatim. The profile emitter wrote every value as key: value. A title ending in : gives

mapping values are not allowed here

and the K2 bundle adds a leading - ("sequence entries are not allowed here"), a leading * or ** ("expected alphabetic or numeric character" / an undefined alias) and ": " inside a title.

4. The constructs, measured

PyYAML 6.0.3 (BaseLoader), and guard 1.3.0 okf.parse_frontmatter:

construct	PyYAML	guard
sources: [{ resource: https://h.no/a/1?x=nb, title: T }]	FAIL, got '?'	admits
sources: [{ resource: https://h.no/a/1, title: T }]	OK	admits
sources: [{ resource: a=b&c.pdf, title: T }]	OK	admits
sources: [{ resource: fil.pdf, title: N100:2023 }]	OK	admits
sources: [{ resource: fil.pdf, title: N100: 2023 }]	FAIL, got ':'	--
sources: [{ resource: fil.pdf, title: Kap #3 }]	FAIL, <stream end>	--
sources: [{ resource: fil.pdf, title: -x }]	OK	admits
sources: [{ resource: fil.pdf, title: *x }]	FAIL, alias	--
sources: [{ resource: "https://h.no/a/1?x=nb", title: T }]	OK	REFUSES
sources: [{ resource: 'https://h.no/a/1?x=nb', title: T }]	OK	REFUSES
block sources: / - resource: https://h.no/a/1?x=nb / title: T	OK	admits
title: Eksempel kontur:	FAIL, mapping values	refuses
title: "Eksempel kontur:"	OK	admits (keeps the quotes)
title: N100: 2023 · title: - punkt · title: *stjerne	FAIL	--
title: Kap #3	OK, as Kap	admits
utgave: R761 Prosesskoden:2025 · title: Kap#3	OK	admits

5. The form, and why it is the narrowest

Block scalars: double-quote exactly the values K3-19's rule refuses. The rule (no leading indicator, no ": ", no " #", no trailing ":", no tab or line break) is now profiles.yaml_block_plain, and over every top-level value in eleven measured trees it agrees with PyYAML read verbatim on every one: 0 refused that PyYAML returns verbatim, 0 kept that it does not (K2 5 524 values, the HTML build 38 752, the XML build 35 177, the five consumer bundles and the rest). Quoting everything would move bytes in every concept of every bundle and buy no valid file; this moves only the values that were failing.

Double, never single -- counted. Values wrapped in a surrounding " pair: 0 in twelve trees (the eleven plus skills/). Wrapped in ': 11 193, all in the consumer bundles (10 937 + 67 + 86 + 103), written that way on purpose. A reader that unquotes " changes the meaning of no value that exists; one that also unquoted ' would change 11 193.

Flow leaves: no quoted form exists, so validation. Plain fails PyYAML on ?, ,[]{}, ": ", " #", a trailing : and a leading indicator; quoted fails the guard. profiles.yaml_flow_plain is the block rule plus ,[]{}?'", with one exception the block rule does not make: a leading - followed by a non-space, which both readers take (del/-utkast.pdf). Refusing that would refuse a document both read; the block rule keeps refusing it because it also decides which spec points are written at all, and in a block a refusal costs only a pair of quotes. The refusal uses each door's existing code: inbox_source_file_unaddressable (also when the FILE NAME, which becomes the entry's title, is not plain), inbox_source_title_unaddressable, source_reference_unquotable (Door A), run_frontmatter_invalid (--frontmatter with a flow value).

Over a constructed list of 39+ values (in tests/test_yaml_frontmatter.py) the safe direction is required and the over-refusals are pinned: block keeps 0 PyYAML would misread and over-refuses 3 (-punkt, ?spm, :kolon, which get quoted); flow keeps 0 that PyYAML or the guard would refuse and over-refuses 0.

Readers read both forms. materialize.parse_frontmatter, the index reader's and the structure reader's copies, and both read_sources branches unquote a "-wrapped value, decoding \" and \\ and nothing else. Other backslash escapes (\n, \t, \x.., \u....) are kept as written: the emitter never writes them. '-wrapped values are untouched -- except in the structure reader, which already unquoted both forms before K3-22 on its own rule (version: '2021' is a string) and keeps doing so. The flow-mapping split is quote-aware, so { title: "a, b" } is one pair; a " opens a quoted scalar only where a value can start, so a plain value with a " in its middle splits as before.

The generated SKILL.md header goes through the block rule too: description carries the root index's bundle_id raw.

6. What moved bytes, and what did not

Each tree built twice, from the base export and the fix export, and compared with diff -rq:

build	concepts / md	files that differ	the line
five-document project	26 / 52	0	--
R761 XML (--bundle-id r761-prosesskoden-2025 --okf-version 0.2 --ingested-at 2026-09-08T12:00:00Z)	2 761 / 5 501	1	a title ending in :, now double-quoted
R761 HTML, 828 files	3 206 / 6 015	1	the same section's title
K2 default, rebuilt (43 documents)	453 / 864	42	42 title lines, now double-quoted: the 41 that failed safe_load and the 1 that parsed to a truncated title

The K2 base rebuild is byte-identical to the pinned K2 artefact on every file but log.md, which the corpus harness had not yet written when the two were compared -- so the 42 lines are K3-22's and nothing else's.

• Goldens and fixtures: 0 bytes. The golden tests are byte-exact and pass; tests/fixtures is untouched and still 12 of 12.
• Readers on existing bundles: 0 differences. Base readers and fix readers return identical parse_frontmatter and read_sources results on 25 273 of 25 273 files across nine trees (the five consumer bundles, the pinned K2 bundle, the HTML and XML builds, tests/fixtures). N100/N200/N500 are read exactly as before.
• One behaviour moved, and it is a refusal. okf build --frontmatter 'sources=[{ resource: <URL with ?>, … }]' now exits 2 and writes nothing. Measured on R761 with K3-19's flags: the base export writes 2 761 of 2 761 concepts PyYAML refuses (2 760 on ?, 1 on the title), the fix export refuses the run. Two K3-19 tests built with such an address; they now build with one without ?, and a new test holds the refusal.
• The K2 pin holds concept count and hit ranks, not bytes, so it did not need moving: 7 of 7 passed in the frozen fix export, with the gold set copied in (without it the pin is 7 skipped, which is not a measurement).

7. Acceptance

check	result
PyYAML valid, fix builds	five-document 28/28 · R761 XML 2 763/2 763 · HTML 3 208/3 208 · K2 454/454 (base 413/454)
dict-equal to parse_frontmatter + read_sources, every file	28/28 · 2 763/2 763 · 3 208/3 208 · K2 454/454 (base 412/454)
generated SKILL.md headers, safe_load	4 of 4 (five-document, R761 XML, HTML, K2)
hit@k R761, base = fix	S1-S6 hit@1/8/50 6/6 · 6/6 · 6/6 at default k and at k = 50; KP rank 1 at both; KN not delivered at both
okf check on every payload	32 of 32 exit 0, "conformant: 16 rules … 0 findings" (len(RULES) = 16)
grep -rc "import yaml" src/	0 on each of 22 files
[project.dependencies]	unchanged, ["llm-ingestion-guard>=1.2,<2.0"]; uv.lock +2 lines, the dev dependency
suite, working tree	1 667 passed / 1 skipped before; 1 753 / 1 after (85 + 1 new)
suite, frozen fix export from a scratch cwd	1 750 / 1 / 3 failed: the two known (test_segmented_golden four-goldens, test_guard_adapter cwd-relative path) and test_cli_build::…installed_copy, which fails identically on the base export under this runner -- the runner's PYTHONPATH reaches the subprocess the test installs into

The hit@k base row reproduces the previous round's before anything else was read.

8. Found, and outside this order

The guard's parser refuses nearly every segmented concept okf writes, for a reason that is not YAML. Guard 1.3.0 admits a flow sequence of MAPPINGS only ("a flow sequence admits flow mappings only"), and okf writes scalar flow sequences at top level: source_offset: [1, 24], source_lines, source_pages, derived: [references], references. Measured with okf.parse_frontmatter over whole frontmatters: five-document 26 of 28 refused, R761 XML 2 761 of 2 763, HTML 3 206 of 3 208, pinned K2 412 of 455 -- identical on base and fix, so it predates K3-22. PyYAML reads all of them. What this means for Door C importing an okf bundle is not measured here (import_bundle was not run).

Honesty limits

• PyYAML is one reader. The rules are validated against it and the guard; a YAML 1.2 reader accepts more (? inside a flow scalar), so the rules are stricter than 1.2, never looser than PyYAML on the values measured.
• The flow-collection exemption trusts its producers. The emitter writes a value shaped [...] or {...} as it stands. Over the eleven trees only structural keys carry that shape and no title does; a producer that built a malformed collection would still be written. The run-stated route (--frontmatter) is validated; the internal producers are measured, not checked at emission.
• Two escapes. \" and \\ are written and decoded; any other escape is neither.
• The 217 descriptions stay omitted. A derived spec point carrying ": " is still not written (K3-19), although it could now be written quoted. Writing it would add values, which this order excluded.
• The constructed list is constructed. The "0 / 0" over real values is the measurement; the constructed list pins the directions on shapes the corpora may not carry.
• Conformance is the floor, never the proof. § 11 point 1 now holds on every build measured; that says the frontmatter parses, not that a bundle is good.

Corrected

README.md said of the flow form: "both are valid YAML, and a real YAML consumer recovers the same structure from either". Measured false: an unquoted URL with a query string inside a flow mapping is not valid YAML for PyYAML. The paragraph now states the limit, and CLAUDE.md carried the same sentence and is corrected with it.

Reproduce

# PyYAML validity, denominator = files with a frontmatter block
python3 - <<'EOF'
import sys, yaml, pathlib
root = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else ".")
fm = ok = 0
for p in root.rglob("*.md"):
    lines = p.read_text(encoding="utf-8").splitlines()
    if not lines or lines[0].strip() != "---":
        continue
    fm += 1
    try:
        yaml.safe_load("\n".join(lines[1:lines.index("---", 1)])); ok += 1
    except yaml.YAMLError:
        pass
print(f"with frontmatter={fm} ok={ok} fail={fm - ok}")
EOF