portfolio-optimiser-claude/tests/test_ingest_spec_loadbearing.py
Kjell Tore Guttormsen de11f2db35 test(ingest-spec): §11's two tables get the §12 row-anchor form
_STRUCTURE_MARKERS binds one phrase per section — blind to a row DROPPED
from a section that keeps its other prose, exactly the gap _CONTRACT_FIELDS
closes for §12. §11 carries two machine-readable tables of its own (golden
extraction format, 4 rows; load-bearing conformance seams, 9 rows), keyed
on a plain-text first column, never row-anchored until now.

Repeats the §12 form (table slice, undocumented-row check, row-removed/
row-renamed red-proofs, fail-closed heading rename) for both tables. The
selection is the hard part for the 9 seam rows: each is a CLAIM that a test
in this repo goes RED when the seam detaches, not a contract field. 7 of 9
anchor to a named test; 2 are deliberately left unanchored because no test
here makes the claim true — "Stamp integrity (curated writers)" is the
library's authoring surface, which exposes no curated-writer primitive at
all, and "Title link-safety" has no bracket-title test (only "two\nlines"
is covered, a different rejection).

Denominator measured on disk: each of the 11 anchored rows individually
deleted from the real shared/ingest-spec.md (git-restored + sha256-verified
after every mutation), full suite run — every row reds exactly its own two
new tests, nothing pre-existing. 884 -> 923 tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018TPNsXNGzqJ96N8y3ehg1h
2026-08-20 23:18:23 +02:00

635 lines
30 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

"""Spec-integrity seam for the ingest spec (ingest-spec §11).
The D7 analog of MAF's I1 framework-guard: this repo consumes ``shared/ingest-spec.md``
UNCHANGED from commons, and this test keeps the contract honest — it goes RED when the
spec goes missing, names a concrete agent toolkit (the framework-neutrality rule), or
stops documenting a contract field. It is the load-bearing guard the ingest layer relies
on to keep being implementable "from this spec alone".
Form mirrored from the sibling ``test_method_spec_loadbearing.py``: every predicate takes
the spec TEXT as an argument, so the detach-proofs are tests in the suite rather than a
one-off spot-check that dies with the session. Red-proofs run against a mutated COPY of
the spec in ``tmp_path`` — never against ``shared/`` itself.
**Second half — structure markers (added after a measurement).** Everything above anchors
§12 plus two whole-document properties. Measured AT THE TIME (806-test suite, before the
§11 row anchors below existed): gutting ANY of §1§11 to its heading alone left this guard
green for 11 of 11 sections — and measured wider, for 10 of the 11 nothing in the whole
suite went red either, so a commons pull could empty a normative section and no test would
notice. (§7 was the single overlap: the O2 ratchet reads its `generated` row.) That claim is
now STALE for §11 specifically — gutting §11 today also reds all 11 row anchors below, since
their tables live in the same body the marker phrase does. ``_STRUCTURE_MARKERS`` closes that
hole by
binding ONE verbatim normative phrase to the section that OWNS it, the form already proven in
``test_method_spec_rule_phrases_loadbearing.py``. Two properties make the binding real rather
than decorative: the phrase must appear in ITS OWN section body (presence anywhere is what a
whole-document check already cannot see), and it must be UNIQUE in the document (a phrase
occurring twice could be satisfied from a section other than its owner).
The SELECTION is ours, not mirrored: each phrase names a seam THIS repo actually carries and
cites the owning module, because a phrase anchoring a seam we do NOT implement would be
green-but-dead of the opposite kind — protecting spec text nothing here depends on. §7's
``generated`` row is deliberately NOT the chosen phrase: the O2 ratchet
(``test_ingest_stamp_conformance_loadbearing.py``) already anchors that row, so §7 is bound
here on a second, independent sentence.
Matching is whitespace- and emphasis-normalized so a reflow or a bolding change does not
false-red; the normalization has its own controls below.
**Third half — §11 row anchors (STATE pkt. 4, ORDER 20260818T155933Z).** ``_STRUCTURE_MARKERS``
binds ONE phrase per section, so it cannot see a row DROPPED from a section that keeps its
other prose intact — exactly the §12 gap ``_CONTRACT_FIELDS`` closes for the machine-readable
contract fields. §11 carries two of its own machine-readable tables, keyed on a plain-text
(not backtick) FIRST column, never covered by a row anchor: the golden extraction format
(4 rows) and the load-bearing conformance seams (9 rows). This section repeats the §12 form
— ``*_table`` slice, ``_undocumented_*`` row check, row-removed/row-renamed red-proofs,
fail-closed heading rename — for both.
The SELECTION is, again, ours — and here it is the hard part, because §11's 9 seam rows are
not machine-readable contract fields, they are CLAIMS that a test in this repo goes RED when
the row's seam detaches. A row anchored without such a test would be green-but-dead of the
opposite kind: protecting spec text nothing here depends on. Two of the nine are NOT anchored
because no test in this repo makes that claim true (see ``_LOAD_BEARING_SEAMS`` below for the
per-row citation, and the comment beside it for the two exclusions and why).
DENOMINATOR, measured on disk (not just detach-proved in-process): each of the 11 anchored
rows was individually deleted from the real ``shared/ingest-spec.md`` (git-restored after
every mutation, `git status` clean + sha256 unchanged before and after), and the full 923-test
suite run against the mutation. Every one of the 11 reds EXACTLY its own two new tests
(``test_spec_documents_*`` + ``test_guard_red_when_*_row_removed``) — nothing pre-existing
reds for any of them, so these anchors are the only thing standing there, not a second copy
of coverage that already existed.
KNOWN LIMITATION (inherited, not fixed here): the row match is exact-whitespace
(``f"| {row} |"``), like ``_CONTRACT_FIELDS`` above and unlike ``_STRUCTURE_MARKERS``'s
normalized match. A commons reflow that pads these two tables for column alignment would
false-red all 11 row anchors at once. Left as-is for consistency with the §12 form this
section mirrors; not a live problem today (measured against the current spec).
"""
from __future__ import annotations
import re
from pathlib import Path
import pytest
SPEC = Path(__file__).resolve().parents[1] / "shared" / "ingest-spec.md"
# Concrete agent toolkits / vendor stacks the framework-neutral spec MUST NOT name.
_FORBIDDEN_TOOLKITS = (
"claude",
"anthropic",
"openai",
"gpt",
"gemini",
"llama",
"langchain",
"autogen",
"crewai",
"semantic kernel",
"microsoft agent framework",
"agent sdk",
"bedrock",
"vertex",
"foundry",
"maf",
)
# The §12 cross-check table is the ANCHOR, and the spec appoints it itself: "Every field
# of the machine-readable contracts, mapped to its normative section (completeness is
# enforced by the spec-integrity test)" — this test is that enforcer. Asserting over the
# whole spec text instead would be green-but-dead: prose saturates the field names —
# `generated` alone appears 22 times OUTSIDE §12 and exactly once inside it (measured
# after the V1 pull) — so no amendment dropping a row could ever turn it red.
_CROSS_CHECK_HEADING = "## 12. Cross-check table"
# Every field of the machine-readable contracts the D7 implementation depends on — the
# spec's §12 cross-check table must keep documenting each (spec-integrity).
_CONTRACT_FIELDS = (
"manifest_version",
"source",
"bundle_summary",
"extractions",
"source_system",
"source_query",
"ingested_at",
"ingest_manifest",
"generated",
"okf_type",
"max_rows",
"root",
"connection_ref", # the sql source reference the D7 sql connector (I5) depends on
)
def _cross_check_table(text: str) -> str:
"""The §12 section body — heading to end-of-spec or the next section, whichever first."""
start = text.index(_CROSS_CHECK_HEADING) # RED (ValueError) if §12 is renamed or dropped
end = text.find("\n## ", start + len(_CROSS_CHECK_HEADING))
return text[start:] if end == -1 else text[start:end]
def _named_toolkits(text: str) -> list[str]:
low = text.lower()
return [tok for tok in _FORBIDDEN_TOOLKITS if tok in low]
def _undocumented_fields(text: str) -> list[str]:
# The row's FIRST column is the documented-field claim; a field named only in another
# row's "Contract" prose does not count.
table = _cross_check_table(text)
return [field for field in _CONTRACT_FIELDS if f"| `{field}` |" not in table]
def _slice_defects(table: str) -> list[str]:
"""Defects in the anchor itself — a slice that widened into the surrounding spec."""
defects = []
if not table.startswith(_CROSS_CHECK_HEADING):
defects.append("does not start at the §12 heading")
if "Honesty rule" in table:
defects.append("leaked §1 prose — the anchor is not a slice")
if "\n## " in table:
defects.append("leaked a following section")
return defects
def _row_removed(text: str, field: str) -> str:
table = _cross_check_table(text)
kept = [ln for ln in table.splitlines(keepends=True) if not ln.startswith(f"| `{field}` |")]
return text.replace(table, "".join(kept))
def _row_renamed(text: str, field: str) -> str:
table = _cross_check_table(text)
return text.replace(table, table.replace(f"| `{field}` |", f"| `{field}_renamed` |"))
# --- The guard itself (against the real spec) ---------------------------------------
def _spec_is_present(path: Path) -> bool:
"""The presence predicate itself, so the red-proof can exercise THE SAME one."""
return path.is_file()
def test_spec_is_present() -> None:
# RED if the spec goes missing (the layer stops being implementable from spec alone).
assert _spec_is_present(SPEC), "ingest-spec.md missing — subtree pull the commons contract"
def test_spec_names_no_agent_toolkit() -> None:
present = _named_toolkits(SPEC.read_text(encoding="utf-8"))
assert not present, f"framework-neutral spec names a concrete toolkit: {present}"
def test_cross_check_slice_is_a_slice_and_not_the_whole_spec() -> None:
# Guards the anchor itself: a slice that degenerated into the full text would make
# every row assertion below green-but-dead again, silently. RED if it widens.
defects = _slice_defects(_cross_check_table(SPEC.read_text(encoding="utf-8")))
assert not defects, f"the §12 anchor degenerated: {defects}"
@pytest.mark.parametrize("field", _CONTRACT_FIELDS)
def test_spec_documents_contract_field(field: str) -> None:
# RED when an amendment drops or renames a §12 row.
undocumented = _undocumented_fields(SPEC.read_text(encoding="utf-8"))
assert field not in undocumented, (
f"contract field {field!r} is no longer a row in the §12 cross-check table"
)
# --- Red-proofs: the guard MUST fail on a detached spec (mutated copy, never shared/) --
def test_guard_red_when_spec_missing(tmp_path: Path) -> None:
# Was VACUOUS — see the twin in ``test_method_spec_loadbearing.py``: it asserted
# a file is absent from a fresh ``tmp_path``, true by construction, and never
# touched the guard it is named for. Now it exercises THE SAME predicate the
# guard calls, both directions, positive control first.
assert _spec_is_present(SPEC)
assert not _spec_is_present(tmp_path / "ingest-spec.md")
@pytest.mark.parametrize("toolkit", _FORBIDDEN_TOOLKITS)
def test_guard_red_when_toolkit_injected(tmp_path: Path, toolkit: str) -> None:
mutated = SPEC.read_text(encoding="utf-8") + f"\n\nBuilt on {toolkit}.\n"
copy = tmp_path / "ingest-spec.md"
copy.write_text(mutated, encoding="utf-8")
assert toolkit in _named_toolkits(copy.read_text(encoding="utf-8"))
@pytest.mark.parametrize("field", _CONTRACT_FIELDS)
def test_guard_red_when_row_removed_from_cross_check_table(tmp_path: Path, field: str) -> None:
# M1: the row is gone from §12 — even though prose elsewhere may still name the field.
copy = tmp_path / "ingest-spec.md"
copy.write_text(_row_removed(SPEC.read_text(encoding="utf-8"), field), encoding="utf-8")
assert field in _undocumented_fields(copy.read_text(encoding="utf-8"))
@pytest.mark.parametrize("field", _CONTRACT_FIELDS)
def test_guard_red_when_row_renamed_in_cross_check_table(tmp_path: Path, field: str) -> None:
# M2: a detach-proof is not a value-proof — the row still exists, under another name.
copy = tmp_path / "ingest-spec.md"
copy.write_text(_row_renamed(SPEC.read_text(encoding="utf-8"), field), encoding="utf-8")
assert field in _undocumented_fields(copy.read_text(encoding="utf-8"))
def test_guard_red_when_section12_heading_renamed(tmp_path: Path) -> None:
# M3: fail-closed — no §12 heading means no anchor, and the guard must raise, not
# silently fall back to a wider (green-but-dead) slice.
mutated = SPEC.read_text(encoding="utf-8").replace(
_CROSS_CHECK_HEADING, "## 12. Field reference"
)
copy = tmp_path / "ingest-spec.md"
copy.write_text(mutated, encoding="utf-8")
with pytest.raises(ValueError):
_cross_check_table(copy.read_text(encoding="utf-8"))
def test_slice_guard_red_when_anchor_degenerates_to_whole_spec() -> None:
# M4: the anchor can degenerate. If _cross_check_table ever returned the full text,
# the row assertions would go green-but-dead again — this proves the slice guard is
# what catches that, and that it is not itself green by accident.
defects = _slice_defects(SPEC.read_text(encoding="utf-8"))
assert defects, "the slice guard accepts the whole spec as the §12 table — it is dead"
# --- Structure markers: one normative phrase bound to the section that OWNS it -------
# section marker -> (verbatim normative phrase, the seam it anchors / owning module)
_STRUCTURE_MARKERS: dict[str, tuple[str, str]] = {
"## 1.": (
"data reaches the model ONLY via OKF bundles",
"bundle-only data path — context by navigation, never query-time retrieval (okf.py)",
),
"## 2.": (
"A bundle without `index.md` is an error",
"a missing root index is fatal, not skipped (okf.py navigate_bundle)",
),
"## 3.": (
"The promotion gate is the ONLY path into the verdict layer",
"verdict reservation — promote() is the sole verdict writer (promotion.py, ingest.py)",
),
"## 4.": (
"Credentials never live in the manifest",
"`connection_ref` is a NAME resolved from the environment at run time (ingest.py)",
),
"## 5.": (
"there is NO wall-clock default",
"`ingested_at` is a required argument of materialize() (ingest.py)",
),
"## 6.": (
"ALL other links — curated and promoted — are preserved verbatim",
"re-ingest layer safety — a promoted verdict's index link survives (promotion.py)",
),
"## 7.": (
"OKF consumers preserve unknown frontmatter fields",
"frontmatter parsing keeps unknown keys, so §7 rides through (okf.py)",
),
"## 8.": (
"the manifest cannot grant itself network access",
"the offline invariant at the consumer seam — no network opt-in is passed (ingest.py)",
),
"## 9.": (
"ingest NEVER runs automatically",
"no scheduler, no run-path call site — the manifest path is explicit (ingest.py)",
),
"## 10.": (
"given the same source content, manifest, and `ingested_at`, the result is byte-identical",
"the byte-exact golden extractions (examples/ingest-golden-*, test_ingest_golden.py)",
),
"## 11.": (
"Test suites MUST run without credentials and without network access",
"the suite-wide offline invariant (test_sdk_isolation.py, no key and no network)",
),
}
def _normalize(text: str) -> str:
"""Whitespace- and emphasis-insensitive: a reflow or a bolding change must not red."""
return re.sub(r"\s+", " ", text.replace("*", "")).strip()
def _section_body(text: str, marker: str) -> str:
"""Heading to the next section. RED (ValueError) if the section is renamed or dropped."""
start = text.index(marker)
end = text.find("\n## ", start + len(marker))
return text[start:] if end == -1 else text[start:end]
def _is_anchored(text: str, marker: str) -> bool:
phrase, _ = _STRUCTURE_MARKERS[marker]
return _normalize(phrase) in _normalize(_section_body(text, marker))
def _occurrences(text: str, marker: str) -> int:
phrase, _ = _STRUCTURE_MARKERS[marker]
return _normalize(text).count(_normalize(phrase))
def _body_emptied(text: str, marker: str) -> str:
"""The section, gutted to its heading alone — the pull this guard exists to catch."""
body = _section_body(text, marker)
heading = body.splitlines()[0]
return text.replace(body, heading + "\n\n")
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_section_keeps_its_structure_marker(marker: str) -> None:
# RED when the owning section loses the normative content this repo builds on — which
# the §12-anchored half above cannot see, because §12 survives an emptied §1-§11.
phrase, seam = _STRUCTURE_MARKERS[marker]
assert _is_anchored(SPEC.read_text(encoding="utf-8"), marker), (
f"{marker} no longer carries its structure marker ({seam}): {phrase!r}"
)
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_structure_marker_is_unique_in_the_spec(marker: str) -> None:
# A phrase occurring twice could be satisfied from a section other than its owner,
# reopening the hole quietly. Binding is only meaningful while the phrase is unique.
count = _occurrences(SPEC.read_text(encoding="utf-8"), marker)
assert count == 1, f"{marker}'s structure marker occurs {count} times — binding is ambiguous"
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_guard_red_when_section_body_emptied(marker: str) -> None:
# The value-proof: GREEN before the mutation, RED after it, same mutation both ways.
text = SPEC.read_text(encoding="utf-8")
assert _is_anchored(text, marker) # positive control first
assert not _is_anchored(_body_emptied(text, marker), marker)
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_the_older_half_is_blind_to_the_emptied_section(marker: str) -> None:
# The DENOMINATOR, machine-checked: 11 of 11 gutted sections leave the §12-anchored
# half fully green. Without this the value-proof above would only show that the new
# marker CAN red, not that it is the only thing standing there.
gutted = _body_emptied(SPEC.read_text(encoding="utf-8"), marker)
assert not _named_toolkits(gutted), "the toolkit rule reds here — this section is not blind"
assert not _undocumented_fields(gutted), "a §12 row went missing — not a clean measurement"
assert not _slice_defects(_cross_check_table(gutted)), "the §12 slice moved — not blind"
assert not _is_anchored(gutted, marker), "and the new marker is what does red"
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_guard_red_when_marker_moves_to_another_section(marker: str) -> None:
# Presence is not anchoring. The phrase survives in the document — appended after §12 —
# and the owning section must still report unanchored.
text = SPEC.read_text(encoding="utf-8")
phrase, _ = _STRUCTURE_MARKERS[marker]
moved = _body_emptied(text, marker).rstrip("\n") + f"\n\n{phrase}\n"
assert _normalize(phrase) in _normalize(moved) # control: it IS still in the document
assert not _is_anchored(moved, marker)
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_guard_red_when_section_heading_renamed(marker: str) -> None:
# Fail-closed, as the §12 slice already is: no heading means no anchor, and the slice
# must raise rather than silently fall back to a wider (green-but-dead) match.
renamed = SPEC.read_text(encoding="utf-8").replace(marker, marker.replace("## ", "## §"))
with pytest.raises(ValueError):
_section_body(renamed, marker)
@pytest.mark.parametrize("marker", list(_STRUCTURE_MARKERS))
def test_normalization_tolerates_reflow_and_emphasis(marker: str) -> None:
# A line-rewrap or a bolding change is not a contract change; it must not red.
text = SPEC.read_text(encoding="utf-8")
phrase, _ = _STRUCTURE_MARKERS[marker]
reflowed = text.replace(phrase, phrase.replace(" ", "\n ", 1))
assert _is_anchored(reflowed, marker), "a reflow false-reds the guard"
bolded = text.replace(phrase, f"**{phrase}**")
assert _is_anchored(bolded, marker), "an emphasis change false-reds the guard"
def test_normalization_is_not_so_loose_that_anything_matches() -> None:
# The tolerance above is only safe while a CHANGED WORD still reds. Without this,
# `_normalize` could degenerate into something that matches everything.
assert _normalize("a b\n c") == "a b c"
assert _normalize("**bold** text") == "bold text"
assert _normalize("MUST fail fast") != _normalize("MUST fail slowly")
# --- §11 row anchors: two tables, plain-text FIRST column, the §12 form repeated ------
_GOLDEN_FORMAT_HEADING = "**Golden extraction format:**"
_LOAD_BEARING_HEADING = "**Load-bearing conformance tests**"
# Golden extraction format (§11), entry -> the test that reads/writes it by this exact name.
_GOLDEN_FORMAT_ENTRIES: dict[str, str] = {
"manifest.json": "test_ingest_golden.py (GOLDEN / 'manifest.json', every test)",
"fixture/": "test_ingest_golden.py::TestMutationControl (case / 'fixture' / 'costs.csv')",
"ingested-at.txt": "test_ingest_golden.py::_ingested_at (the fixed §5 timestamp)",
"expected-bundle/": "test_ingest_golden.py::test_materializes_golden_byte_for_byte",
}
# Load-bearing conformance seams (§11), seam -> the test that goes RED when it detaches.
# Anchored: seven of nine — each cites a test in THIS repo that makes the row's claim true.
_LOAD_BEARING_SEAMS: dict[str, str] = {
"Provenance stamping": (
"test_ingest_loadbearing.py::TestProvenanceStamping"
"::test_generated_file_carries_the_provenance_layer"
),
"Navigability": (
"test_ingest_loadbearing.py::TestNavigability::test_generated_bundle_navigates_via_unchanged_okf"
),
"Verdict reservation": (
"test_ingest_loadbearing.py::TestVerdictReservation::test_verdict_okf_type_is_rejected"
),
"Re-ingest layer safety": (
"test_ingest_loadbearing.py::TestReingestLayerSafety"
"::test_promoted_verdict_and_curated_survive_reingest"
),
"Golden regression": "test_ingest_golden.py::test_materializes_golden_byte_for_byte",
# We carry this seam as a REFUSAL, not as an http connector (§1 makes http optional):
# the adapter never passes the network opt-in, so an http source is refused fail-fast.
"Network gate": (
"test_ingest_adoption.py::TestOfflineInvariant::test_http_source_is_refused_at_the_network_gate"
),
# Self-referential and deliberately so: this row's three clauses ARE the three guards
# already in this file.
"Spec integrity": (
"test_spec_is_present / test_spec_names_no_agent_toolkit / "
"test_spec_documents_contract_field (this file)"
),
}
# NOT anchored — no test in this repo makes the row's claim true; anchoring either would be
# green-but-dead, protecting spec text nothing here depends on:
#
# - "Stamp integrity (curated writers)": the reject-on-complete-stamp check is the §3
# verbatim-authoring-path's job. It lives in the llm-ingestion-okf LIBRARY, not in this
# repo's src/ — verified: the library's public surface (Extraction, FileSource, HttpSource,
# IngestError, IngestResult, Manifest, ManifestError, MaterializationError, NetworkGateError,
# RenderError, SourceError, SqlSource, connectors, errors, load_manifest, manifest,
# materialize, materialize_bundle, render) exposes no curated-writer / authoring primitive
# at all, and no test here forges the ownership stamp and asserts rejection. `inbox.py`'s
# "authoring primitive" is a different one (method-spec §5 verdict JSON, not OKF concept
# frontmatter) and does not touch this seam.
# - "Title link-safety": no test here supplies a title containing `[` or `]` and asserts
# fail-fast rejection. `test_ingest.py::TestManifestValidation::test_malformed_manifest_is_rejected`
# mutates title to `"two\nlines"` — a different rejection, not this one.
def _golden_format_table(text: str) -> str:
"""The golden-format table body — heading to the start of the next table."""
start = text.index(_GOLDEN_FORMAT_HEADING) # RED (ValueError) if the heading is renamed
end = text.index(_LOAD_BEARING_HEADING, start)
return text[start:end]
def _load_bearing_table(text: str) -> str:
"""The load-bearing table body — heading to end-of-spec or the next section."""
start = text.index(_LOAD_BEARING_HEADING) # RED (ValueError) if the heading is renamed
end = text.find("\n## ", start + len(_LOAD_BEARING_HEADING))
return text[start:] if end == -1 else text[start:end]
def _golden_slice_defects(table: str) -> list[str]:
defects = []
if not table.startswith(_GOLDEN_FORMAT_HEADING):
defects.append("does not start at the golden-format heading")
if _LOAD_BEARING_HEADING in table:
defects.append("leaked the load-bearing table")
if "\n## " in table:
defects.append("leaked past §11 into a following section")
return defects
def _load_bearing_slice_defects(table: str) -> list[str]:
defects = []
if not table.startswith(_LOAD_BEARING_HEADING):
defects.append("does not start at the load-bearing heading")
if _CROSS_CHECK_HEADING in table:
defects.append("leaked §12")
if "\n## " in table:
defects.append("leaked a following section")
return defects
def _undocumented_golden_entries(text: str) -> list[str]:
table = _golden_format_table(text)
return [entry for entry in _GOLDEN_FORMAT_ENTRIES if f"| `{entry}` |" not in table]
def _undocumented_loadbearing_seams(text: str) -> list[str]:
table = _load_bearing_table(text)
return [seam for seam in _LOAD_BEARING_SEAMS if f"| {seam} |" not in table]
def _golden_row_removed(text: str, entry: str) -> str:
table = _golden_format_table(text)
marker = f"| `{entry}` |"
kept = [ln for ln in table.splitlines(keepends=True) if not ln.startswith(marker)]
return text.replace(table, "".join(kept))
def _golden_row_renamed(text: str, entry: str) -> str:
table = _golden_format_table(text)
marker = f"| `{entry}` |"
return text.replace(table, table.replace(marker, f"| `{entry}_renamed` |"))
def _loadbearing_row_removed(text: str, seam: str) -> str:
table = _load_bearing_table(text)
marker = f"| {seam} |"
kept = [ln for ln in table.splitlines(keepends=True) if not ln.startswith(marker)]
return text.replace(table, "".join(kept))
def _loadbearing_row_renamed(text: str, seam: str) -> str:
table = _load_bearing_table(text)
marker = f"| {seam} |"
return text.replace(table, table.replace(marker, f"| {seam} (renamed) |"))
def test_golden_format_slice_is_a_slice_and_not_the_whole_spec() -> None:
defects = _golden_slice_defects(_golden_format_table(SPEC.read_text(encoding="utf-8")))
assert not defects, f"the golden-format anchor degenerated: {defects}"
def test_load_bearing_slice_is_a_slice_and_not_the_whole_spec() -> None:
defects = _load_bearing_slice_defects(_load_bearing_table(SPEC.read_text(encoding="utf-8")))
assert not defects, f"the load-bearing anchor degenerated: {defects}"
@pytest.mark.parametrize("entry", _GOLDEN_FORMAT_ENTRIES)
def test_spec_documents_golden_format_entry(entry: str) -> None:
# RED when an amendment drops or renames a golden-format row.
undocumented = _undocumented_golden_entries(SPEC.read_text(encoding="utf-8"))
assert entry not in undocumented, f"golden-format entry {entry!r} is no longer a row in §11"
@pytest.mark.parametrize("seam", _LOAD_BEARING_SEAMS)
def test_spec_documents_loadbearing_seam(seam: str) -> None:
# RED when an amendment drops or renames an anchored load-bearing row.
undocumented = _undocumented_loadbearing_seams(SPEC.read_text(encoding="utf-8"))
assert seam not in undocumented, f"load-bearing seam {seam!r} is no longer a row in §11"
@pytest.mark.parametrize("entry", _GOLDEN_FORMAT_ENTRIES)
def test_guard_red_when_golden_row_removed(tmp_path: Path, entry: str) -> None:
text = SPEC.read_text(encoding="utf-8")
assert entry not in _undocumented_golden_entries(text) # positive control first
copy = tmp_path / "ingest-spec.md"
copy.write_text(_golden_row_removed(text, entry), encoding="utf-8")
assert entry in _undocumented_golden_entries(copy.read_text(encoding="utf-8"))
@pytest.mark.parametrize("entry", _GOLDEN_FORMAT_ENTRIES)
def test_guard_red_when_golden_row_renamed(tmp_path: Path, entry: str) -> None:
# A detach-proof is not a value-proof — the row still exists, under another name.
copy = tmp_path / "ingest-spec.md"
copy.write_text(_golden_row_renamed(SPEC.read_text(encoding="utf-8"), entry), encoding="utf-8")
assert entry in _undocumented_golden_entries(copy.read_text(encoding="utf-8"))
@pytest.mark.parametrize("seam", _LOAD_BEARING_SEAMS)
def test_guard_red_when_loadbearing_row_removed(tmp_path: Path, seam: str) -> None:
text = SPEC.read_text(encoding="utf-8")
assert seam not in _undocumented_loadbearing_seams(text) # positive control first
copy = tmp_path / "ingest-spec.md"
copy.write_text(_loadbearing_row_removed(text, seam), encoding="utf-8")
assert seam in _undocumented_loadbearing_seams(copy.read_text(encoding="utf-8"))
@pytest.mark.parametrize("seam", _LOAD_BEARING_SEAMS)
def test_guard_red_when_loadbearing_row_renamed(tmp_path: Path, seam: str) -> None:
copy = tmp_path / "ingest-spec.md"
copy.write_text(
_loadbearing_row_renamed(SPEC.read_text(encoding="utf-8"), seam), encoding="utf-8"
)
assert seam in _undocumented_loadbearing_seams(copy.read_text(encoding="utf-8"))
def test_guard_red_when_golden_format_heading_renamed(tmp_path: Path) -> None:
# Fail-closed: no heading means no anchor, and the slice must raise, not silently fall
# back to a wider (green-but-dead) match.
mutated = SPEC.read_text(encoding="utf-8").replace(
_GOLDEN_FORMAT_HEADING, "**Golden fixture format:**"
)
copy = tmp_path / "ingest-spec.md"
copy.write_text(mutated, encoding="utf-8")
with pytest.raises(ValueError):
_golden_format_table(copy.read_text(encoding="utf-8"))
def test_guard_red_when_load_bearing_heading_renamed(tmp_path: Path) -> None:
mutated = SPEC.read_text(encoding="utf-8").replace(
_LOAD_BEARING_HEADING, "**Load-bearing regression tests**"
)
copy = tmp_path / "ingest-spec.md"
copy.write_text(mutated, encoding="utf-8")
with pytest.raises(ValueError):
_load_bearing_table(copy.read_text(encoding="utf-8"))
def test_golden_slice_guard_red_when_anchor_degenerates_to_whole_spec() -> None:
defects = _golden_slice_defects(SPEC.read_text(encoding="utf-8"))
assert defects, "the golden-format slice guard accepts the whole spec — it is dead"
def test_load_bearing_slice_guard_red_when_anchor_degenerates_to_whole_spec() -> None:
defects = _load_bearing_slice_defects(SPEC.read_text(encoding="utf-8"))
assert defects, "the load-bearing slice guard accepts the whole spec — it is dead"