feat(materialize): sources takes a list and renders N flow mappings

PM decision B6 asked for a list-taking _render_sources so a concept can record
more than one source, and prescribed the block list as the emitted form. The
list is delivered; the block form is not.

Three measurements, not an argument. Our own parse_frontmatter skips indented
lines, so a block list round-trips to an empty value with every entry silently
gone -- and _is_ingest_owned reads through that same parser. The consumer B6
was written for accepts the multi-entry flow sequence and classifies a block
sequence as unreadable provenance, so block would hand it exactly the state it
cannot read. And B6's own acceptance test asks for a round trip through this
parser, which no block form can pass.

A single source renders byte-identically, so all six goldens are unmoved. The
unquotable-value gate now runs on every entry, not just the first. New code
sources_empty refuses an empty list.

1023 -> 1034 tests, including the negative control that pins the block form's
silent data loss.
This commit is contained in:
Kjell Tore Guttormsen 2026-09-02 16:12:19 +02:00
commit 16eeeb007e
4 changed files with 240 additions and 32 deletions

View file

@ -13,7 +13,7 @@ import hashlib
import logging
import re
import unicodedata
from collections.abc import Mapping
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
@ -190,37 +190,64 @@ def _source_locator(source: Source) -> str:
return source.base_url
def _render_sources(source: Source) -> str:
"""§5 `sources` as an inline flow sequence of one flow mapping.
def _render_sources(sources: Sequence[Source]) -> str:
"""§5 `sources` as an inline flow sequence of N flow mappings.
Two keys, not upstream's five: a manifest source has no `author`, no
`last_modified`, and no bundle-internal `resource` in upstream's sense, and
inventing them would be writing fields with no reader.
Two keys per entry, not upstream's five: a manifest source has no `author`,
no `last_modified`, and no bundle-internal `resource` in upstream's sense,
and inventing them would be writing fields with no reader.
The flow form rather than upstream's block list, measured and chosen: a
block list read back through this library's line-oriented parser turns each
item line into a KEY nobody wrote and `_is_ingest_owned` reads through
that same parser. The flow form also satisfies commons' §5 "all values MUST
be single-line", and §11 requires parseable YAML rather than block YAML.
**A LIST, and the flow form PM decision B6 delivered, its mechanism not.**
B6 asked for a list-taking renderer and prescribed the BLOCK list as the
emitted form. The list is here; the block form is not, and the reason is
measured rather than argued:
Refusing an unquotable locator is the point of the check rather than a
nicety: `[{ id: x, resource: data, backup }]` is not a parse ERROR, it is a
mapping with a `backup` key nobody wrote. A silently wrong provenance record
is worse than a refused run, and repairing the value by quoting it would
change bytes the operator supplied. Validation, not repair the same
posture as the filename-length gate.
- `parse_frontmatter` is line-oriented and skips indented lines, so a block
list round-trips to an EMPTY value with every entry gone, silently. We
would be writing provenance we cannot read back, and `_is_ingest_owned`
reads through that same parser.
- The consumer B6 was written for accepts `[{ k: v }, { k: v }]` plural
and classifies a block sequence as unreadable provenance. Block would hand
it exactly the state it reports as unreadable.
- B6's own acceptance test asks for a round trip through this parser. No
block form can pass it.
The flow form also satisfies commons' §5 "all values MUST be single-line",
and §11 requires parseable YAML rather than block YAML. Reading block needs
the structured reader (D1b); until then the constraint binds what we write.
A single source renders byte-identically to the one-entry form that shipped
before this took a list, which is what keeps all six goldens unmoved.
Refusing an unquotable value is the point of the check rather than a nicety:
`[{ id: x, resource: data, backup }]` is not a parse ERROR, it is a mapping
with a `backup` key nobody wrote. A silently wrong provenance record is
worse than a refused run, and repairing the value by quoting it would change
bytes the operator supplied. Validation, not repair the same posture as
the filename-length gate. Every entry is checked, not only the first: a gate
that reads the head of a list is a gate the second entry walks past.
"""
locator = _source_locator(source)
for label, value in (("id", source.id), ("resource", locator)):
if _FLOW_UNSAFE_RE.search(value):
raise MaterializationError(
f"the source {label} {value!r} contains a character that would "
"restructure the `sources` flow mapping (one of `,[]{}` or a "
"colon followed by whitespace) — refusing to emit a provenance "
"record that parses cleanly into something no one wrote",
code="source_reference_unquotable",
)
return f"[{{ id: {source.id}, resource: {locator} }}]"
if not sources:
raise MaterializationError(
"a `sources` list must name at least one source — `sources: []` "
"reads as a measured absence when it is the absence of a "
"measurement",
code="sources_empty",
)
entries = []
for source in sources:
locator = _source_locator(source)
for label, value in (("id", source.id), ("resource", locator)):
if _FLOW_UNSAFE_RE.search(value):
raise MaterializationError(
f"the source {label} {value!r} contains a character that would "
"restructure the `sources` flow mapping (one of `,[]{}` or a "
"colon followed by whitespace) — refusing to emit a provenance "
"record that parses cleanly into something no one wrote",
code="source_reference_unquotable",
)
entries.append(f"{{ id: {source.id}, resource: {locator} }}")
return f"[{', '.join(entries)}]"
def _render_concept_file(
@ -248,7 +275,7 @@ def _render_concept_file(
# profiles would append `sources` to every v0.1 bundle — additivity is a
# property of what is constructed here, not of the emitter.
if "sources" in profile.frontmatter.order:
frontmatter["sources"] = _render_sources(manifest.source)
frontmatter["sources"] = _render_sources([manifest.source])
return f"---\n{profile.frontmatter.emit(frontmatter)}\n---\n\n{body}"