fix(profiles,materialize,structure,consume): a block sources sequence is decoded, not skipped
One grammar, four call sites. `read_block_mappings` moves out of `consume.read_sources` -- where it was written and measured -- into `profiles`, the module both the flat readers and `consume` already import, and the three copies of the line-oriented frontmatter grammar now decode a block sequence for the keys `STRUCTURED_BLOCK_KEYS` names. Two copies of a block grammar would be two answers to one question. The value TYPE was the real choice and it was measured, not argued. `parse_frontmatter` is public API (`okf.parse_frontmatter`) returning `dict[str, str]`, and a list of mappings is not a `str`. Widening the return type to `str | list[dict[str, str]]` costs 15 `mypy --strict` errors across four of the five modules that touch the reader, plus a signature every caller outside this repository would have to follow. Rendering the entries back into the flow form those same readers already round-trip costs 0. The rendering is a READING projection and says so: it is not a claim that the value is writable -- `yaml_flow_plain` still refuses a `?` and the guard still refuses a quote inside a flow mapping, which is why the producer writes block in the first place. `STRUCTURED_BLOCK_KEYS` is one key wide. `sources` is the key `read_sources` already knows how to read; a fixture in this tree carries a block `verified:` that still reads as an empty value, and a test pins that state so the next widening is a decision rather than a side effect. Nothing nested reaches the document's namespace: the entries land inside their own value, and the K3-20 substitution guarantee is asserted per reader copy. Three tests that pinned the old behaviour are rewritten to what is now true, none weakened on its other half: the block round trip in `test_multi_source_provenance` (the evidence behind `_render_sources`' reason 1), the v0.2 characterization (whose key-space assertion is the half that must never weaken), and K3-22's shipped-file known-positive, where the one difference is counted and pinned at 1. Suite 1807 passed / 1 skipped, rc 0, 94 s -- 1782/1 before plus 25 new. ruff clean, `mypy --strict` clean over 21 files, `uv.lock` untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
eb327bfd0c
commit
28f9a4b540
8 changed files with 211 additions and 65 deletions
|
|
@ -30,6 +30,7 @@ from pathlib import Path
|
|||
|
||||
import pytest
|
||||
|
||||
from llm_ingestion_okf.consume import _frontmatter_lines, _parse_flow_mappings, read_sources
|
||||
from llm_ingestion_okf.errors import MaterializationError
|
||||
from llm_ingestion_okf.manifest import FileSource, HttpSource, Source, SqlSource
|
||||
from llm_ingestion_okf.materialize import _render_sources, parse_frontmatter
|
||||
|
|
@ -100,11 +101,19 @@ def test_two_sources_round_trip_through_our_own_parser(tmp_path: Path) -> None:
|
|||
assert sorted(frontmatter) == ["generated", "sources", "title", "type"]
|
||||
|
||||
|
||||
def test_the_block_form_round_trips_to_nothing(tmp_path: Path) -> None:
|
||||
"""The NEGATIVE CONTROL, and the measured reason the block form is not
|
||||
emitted. Two entries go in; an empty string comes back, and no error is
|
||||
raised anywhere. This test must stay green: it is the evidence, not a
|
||||
regression guard."""
|
||||
def test_the_block_form_round_trips_through_the_flat_reader(tmp_path: Path) -> None:
|
||||
"""Two entries go in and BOTH come back (K3-24).
|
||||
|
||||
This test carried the opposite assertion until 2026-09-12, and it was the
|
||||
evidence behind the first of the three reasons `_render_sources` gives for
|
||||
not emitting the block form: a block list round-tripped to an EMPTY value
|
||||
with every entry gone, silently. That reason is now false, and the test
|
||||
says what is true instead of standing as a justification nothing measures.
|
||||
|
||||
It is still a control and not a regression guard: what it pins is that
|
||||
the flat reader and `read_sources` return the same entries from the same
|
||||
bytes. The emission rule did not move with it -- reasons 2 and 3 are
|
||||
separate measurements and live in the docstring they belong to."""
|
||||
path = tmp_path / "concept.md"
|
||||
path.write_text(
|
||||
"---\n"
|
||||
|
|
@ -120,9 +129,16 @@ def test_the_block_form_round_trips_to_nothing(tmp_path: Path) -> None:
|
|||
)
|
||||
|
||||
frontmatter = parse_frontmatter(path)
|
||||
entries, present = read_sources(_frontmatter_lines(path))
|
||||
|
||||
assert frontmatter["sources"] == ""
|
||||
assert "golden-db" not in "".join(frontmatter.values())
|
||||
assert present
|
||||
assert _parse_flow_mappings(frontmatter["sources"]) == [
|
||||
{"id": "golden-catalogue", "resource": "fixture"},
|
||||
{"id": "golden-db", "resource": "OKF_GOLDEN_SQL_DB"},
|
||||
]
|
||||
assert _parse_flow_mappings(frontmatter["sources"]) == [dict(entry) for entry in entries]
|
||||
# The quieter half, unchanged: an entry's keys stay inside the value.
|
||||
assert set(frontmatter) == {"type", "sources"}
|
||||
|
||||
|
||||
# --- the refusal applies to every entry, not only the first -----------------
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue