open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(materialize): emit frontmatter and index values verbatim

Browse source 
a7e1bc8 unified title rendering at both sites and routed the open question
to commons, the spec owner: does §5's whitespace collapse apply to `title`,
or only to `source_query`? The ruling is source_query only. §5 mandates the
collapse for that one field — where a multi-line SQL SELECT must render on
one line — while every other value is validated single-line at manifest load
and emitted as-is: validation, not repair.

_single_line is renamed _collapse_whitespace and now runs for source_query
alone. Frontmatter values, the index link label, and the labels_by_target
map all pass through verbatim. A title carrying an internal whitespace run
(or any other operator-supplied bytes) now survives byte-for-byte at both
the frontmatter and the index-label site, instead of being silently altered.

No golden bytes change: no title in the shipped fixtures or either consumer
carries a collapsible whitespace run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
This commit is contained in:
Kjell Tore Guttormsen 2026-07-23 06:41:27 +02:00
commit c011534e9f
2 changed files with 46 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

32
tests/test_materialize.py
View file

@ -159,6 +159,38 @@ def test_title_renders_identically_in_frontmatter_and_index(tmp_path: Path) -> N
assert frontmatter_title == index_label
def test_title_emitted_verbatim_not_collapsed(tmp_path: Path) -> None:
# §5 mandates whitespace-run collapse ONLY for `source_query`
# (ingest-spec.md:140-141). `title` is validated single-line at manifest
# load and emitted verbatim — validation, not repair — so an internal
# whitespace run survives at BOTH the frontmatter and the index-label site.
src = tmp_path / "src"
manifest_path = write_manifest(
src,
file_manifest_data(
[
{
"id": "orders",
"title": "Energiforbruk 2024",
"query": "orders.csv",
"okf_type": "dataset",
"max_rows": 100,
}
]
),
)
(src / "data").mkdir()
(src / "data" / "orders.csv").write_text("a,b\n1,x\n", encoding="utf-8", newline="")
bundle = tmp_path / "bundle"
materialize_bundle(manifest_path, bundle, ingested_at=INGESTED_AT)
concept = (bundle / "ingest-orders.md").read_text(encoding="utf-8")
index = (bundle / "index.md").read_text(encoding="utf-8")
assert "title: Energiforbruk 2024\n" in concept
assert "- [Energiforbruk 2024](ingest-orders.md)" in index
def test_result_lists_written_concept_files(file_setup: tuple[Path, Path]) -> None:
manifest_path, bundle = file_setup
result = materialize_bundle(manifest_path, bundle, INGESTED_AT)