fix(frontmatter): a nested key must not substitute for a top-level one

The line-oriented frontmatter grammar exists in three copies, each with the
duplication documented at its site: `materialize` reads a path, `structure`
needs a character offset, `profiles` returns body lines. All three keyed on
`key.strip()`, which discards the indentation that is the only thing telling
a nested key from a top-level one. An indented `title:` under a `sources:`
block therefore landed in the same flat namespace as the document's own
`title:` and, arriving later, won.

The failure is substitution, not omission. A dropped value is visible to
whoever reads the concept; a substituted one is not -- the document carries a
title that looks entirely right and belongs to something else. Because
`number` derives from `title` and `parent` derives from `number`, one
substitution walks the hierarchy. Measured, not inferred: a document titled
`N100.2` with a nested source titled `N200.7` came back as N200.7 with parent
N200 instead of N100.2 with parent N100.

Measured incidence across the two corpora, denominators stated:
`_okf-canonical` @ ad30107, 54 documents with parsable frontmatter, 49 carry
a nested key colliding with a top-level name (90.7%); `_okf-upstream` @
9a15b13, 66 documents, 58 collide (87.9%). The colliding key is `title`, and
often `resource` with it -- in `acme_retail/tables/orders.md` the concept's
own BigQuery resource pointer was replaced by a nested one. This is a fix
that clears observed damage, not a hardening without a witness.

The fix refuses indented lines; it does not read them. Block form stays
unreadable -- `sources` and `verified` still come back empty -- so D4's
flow-form emission rule is untouched and the structured reader is still D1b.
Two characterization tests that pinned the old behaviour now pin the new: the
block-list family still DROPS its value, and only the key-space pollution is
gone. That family is not otherwise addressed here.

Test first, red before the code was touched, with known-positive controls for
all three parsers so that a parser returning nothing could not pass.

Order: 20260830T000740Z-4733930312-from-.claude

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-08-31 23:31:53 +02:00
commit 2337a328d9
6 changed files with 239 additions and 20 deletions

View file

@ -124,6 +124,15 @@ def parse_frontmatter(path: Path) -> dict[str, str]:
for line in lines[1:]:
if line.strip() == "---":
break
# An INDENTED key belongs to the block above it, not to the document.
# Without this, `key.strip()` would flatten it into the same namespace
# as the top-level keys and, arriving later, SUBSTITUTE for one of them
# -- a `sources:` entry's own `title:` silently becoming the document's,
# carrying `number` and `parent` with it. Skipping is deliberately not
# parsing: the nested value is not read, only refused. The structured
# reader is D1b.
if line[:1] in (" ", "\t"):
continue
key, sep, value = line.partition(":")
if sep:
frontmatter[key.strip()] = value.strip()

View file

@ -369,6 +369,15 @@ def _split_frontmatter(text: str) -> tuple[dict[str, str], list[str]]:
for offset, line in enumerate(lines[1:], start=1):
if line.strip() == "---":
return head, lines[offset + 1 :]
# An INDENTED key belongs to the block above it, not to the document.
# Without this, `key.strip()` would flatten it into the same namespace
# as the top-level keys and, arriving later, SUBSTITUTE for one of them
# -- a `sources:` entry's own `title:` silently becoming the document's,
# carrying `number` and `parent` with it. Skipping is deliberately not
# parsing: the nested value is not read, only refused. The structured
# reader is D1b.
if line[:1] in (" ", "\t"):
continue
key, sep, value = line.partition(":")
if sep:
head[key.strip()] = value.strip()

View file

@ -138,6 +138,15 @@ def _split_frontmatter(text: str) -> tuple[dict[str, str], int]:
offset += len(line)
if line.strip() == "---":
return declared, offset
# An INDENTED key belongs to the block above it, not to the document.
# Without this, `key.strip()` would flatten it into the same namespace
# as the top-level keys and, arriving later, SUBSTITUTE for one of them
# -- a `sources:` entry's own `title:` silently becoming the document's,
# carrying `number` and `parent` with it. Skipping is deliberately not
# parsing: the nested value is not read, only refused. The structured
# reader is D1b.
if line[:1] in (" ", "\t"):
continue
key, sep, value = line.partition(":")
if sep:
declared[key.strip()] = _unquote(value.strip())