fix(profiles,materialize,structure,consume): a block sources sequence is decoded, not skipped
One grammar, four call sites. `read_block_mappings` moves out of `consume.read_sources` -- where it was written and measured -- into `profiles`, the module both the flat readers and `consume` already import, and the three copies of the line-oriented frontmatter grammar now decode a block sequence for the keys `STRUCTURED_BLOCK_KEYS` names. Two copies of a block grammar would be two answers to one question. The value TYPE was the real choice and it was measured, not argued. `parse_frontmatter` is public API (`okf.parse_frontmatter`) returning `dict[str, str]`, and a list of mappings is not a `str`. Widening the return type to `str | list[dict[str, str]]` costs 15 `mypy --strict` errors across four of the five modules that touch the reader, plus a signature every caller outside this repository would have to follow. Rendering the entries back into the flow form those same readers already round-trip costs 0. The rendering is a READING projection and says so: it is not a claim that the value is writable -- `yaml_flow_plain` still refuses a `?` and the guard still refuses a quote inside a flow mapping, which is why the producer writes block in the first place. `STRUCTURED_BLOCK_KEYS` is one key wide. `sources` is the key `read_sources` already knows how to read; a fixture in this tree carries a block `verified:` that still reads as an empty value, and a test pins that state so the next widening is a decision rather than a side effect. Nothing nested reaches the document's namespace: the entries land inside their own value, and the K3-20 substitution guarantee is asserted per reader copy. Three tests that pinned the old behaviour are rewritten to what is now true, none weakened on its other half: the block round trip in `test_multi_source_provenance` (the evidence behind `_render_sources`' reason 1), the v0.2 characterization (whose key-space assertion is the half that must never weaken), and K3-22's shipped-file known-positive, where the one difference is counted and pinned at 1. Suite 1807 passed / 1 skipped, rc 0, 94 s -- 1782/1 before plus 25 new. ruff clean, `mypy --strict` clean over 21 files, `uv.lock` untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
eb327bfd0c
commit
28f9a4b540
8 changed files with 211 additions and 65 deletions
|
|
@ -26,6 +26,7 @@ from llm_ingestion_guard import okf as guard_okf
|
|||
|
||||
from llm_ingestion_okf.consume import _frontmatter_lines, _parse_flow_mappings, read_sources
|
||||
from llm_ingestion_okf.materialize import parse_frontmatter
|
||||
from llm_ingestion_okf.profiles import STRUCTURED_BLOCK_KEYS
|
||||
from llm_ingestion_okf.profiles import _split_frontmatter as _profiles_split
|
||||
from llm_ingestion_okf.structure import _split_frontmatter as _structure_split
|
||||
|
||||
|
|
@ -278,7 +279,7 @@ def test_every_fixture_frontmatter_keeps_every_value_a_reference_reader_finds()
|
|||
cannot turn this green over an empty set."""
|
||||
paths = _fixture_frontmatters()
|
||||
assert len(paths) == 12
|
||||
carrying_block_sources = 0
|
||||
carrying_sources_list = 0
|
||||
for path in paths:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
flat = parse_frontmatter(path)
|
||||
|
|
@ -290,13 +291,15 @@ def test_every_fixture_frontmatter_keeps_every_value_a_reference_reader_finds()
|
|||
for key, value in reference.items():
|
||||
if value in (None, "", [], {}):
|
||||
continue
|
||||
if isinstance(value, (list, dict)) and key != "sources":
|
||||
if isinstance(value, (list, dict)) and key not in STRUCTURED_BLOCK_KEYS:
|
||||
continue
|
||||
assert flat.get(key, "") != "", f"{path.name}: {key} lost"
|
||||
if isinstance(reference.get("sources"), list):
|
||||
carrying_block_sources += 1
|
||||
carrying_sources_list += 1
|
||||
assert _parse_flow_mappings(flat["sources"]) == reference["sources"]
|
||||
assert carrying_block_sources == 1
|
||||
# Two of the twelve: one block form and one flow form, both read by
|
||||
# PyYAML as a list and both required to decode to the same entries here.
|
||||
assert carrying_sources_list == 2
|
||||
|
||||
|
||||
def test_a_block_key_outside_the_named_set_is_still_empty() -> None:
|
||||
|
|
@ -308,6 +311,7 @@ def test_a_block_key_outside_the_named_set_is_still_empty() -> None:
|
|||
measurement covers. A fixture in this tree carries a block `verified:` for
|
||||
exactly this reason, and it still reads as an empty value. Whoever widens
|
||||
the set will see this test, which is the point of pinning it."""
|
||||
assert STRUCTURED_BLOCK_KEYS == frozenset({"sources"})
|
||||
path = FIXTURES / "consume-bundle" / "dyp" / "nivaa" / "blokkform-verifisert.md"
|
||||
reference = yaml.safe_load(path.read_text(encoding="utf-8").split("---\n")[1])
|
||||
assert isinstance(reference["verified"], list)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue