llm-ingestion-okf/tests/test_k2_office_fixtures.py
Kjell Tore Guttormsen 9d1f4b14ed test(fixtures): replace sector-specific example material with generic, fictitious examples — green
Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.

- Generated fixtures are regenerated from their generators. Their structural
  counts are identical before and after: elements, images, rows, cells,
  headings, bookmarks and the witness inventory's per-document totals. The
  image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
  each are replaced by a generic, repeatable --corpus PATH with no default.
  Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
  removed, along with its three twins and five tests. The mutation harness
  loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
  are removed, and 40 are neutralized. Dead links are removed, and no new
  dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.

Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.

Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:52:02 +02:00

118 lines
4.6 KiB
Python

"""The K2 denominator for `pptx`/`odt`/`rtf`: a synthetic fixture set, N = 3.
`docs/2026-09-04-k2-pptx-odt-rtf.md` measured the corpus denominator for these
three office rows and found it ZERO -- `K2/trinn1` holds 43 files and not one
of them is a `pptx`, an `odt` or an `rtf`. So those three rows in
`extract._EVIDENCE` read `unmeasured`: they work by construction and had never
met a document anyone wrote.
This module is the smallest thing that changes that without inventing a corpus:
three hand-laid documents carrying the SAME content in three containers, and a
hand-counted fasit committed beside them. It pins the one property a report
cannot assert for itself -- that all three go through Door B, offline, and land
as concepts -- so the numbers in
`docs/2026-09-07-k2-pptx-odt-rtf-fixtures.md` stay re-measurable rather than
becoming a memory of a run.
WHAT IT DOES NOT DO. Three synthetic documents in one house style are not a
corpus. A green run here says the row is exercised, never that the format is
covered. Round 9 (2026-09-09) measured all three end to end for the first time
and moved them `unmeasured` -> `constructed`, a third evidence class defined in
`extract._EVIDENCE`: the row has met a document, but not one anyone wrote for
their own purposes. This suite asserts that class rather than the old one.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
import pytest
from llm_ingestion_okf.extract import _EVIDENCE
from llm_ingestion_okf.inbox import GateDecision, process_inbox
pytest.importorskip("pypandoc", reason="office conversion needs the [extract] extra")
INGESTED_AT = "2026-09-07T00:00:00Z"
WARN = "warn"
FIXTURES = Path(__file__).parent / "fixtures"
DROP = FIXTURES / "k2-office"
FASIT = FIXTURES / "k2-office-fasit.json"
#: The three containers, one per row this fixture set exists to reach.
EXPECTED_FILES = (
"krav-presentasjon.pptx",
"krav-rikt-tekstformat.rtf",
"krav-tekstdokument.odt",
)
@dataclass
class StubGuard:
"""The pinned guard's surface, offline. A TEST DOUBLE -- never in `src/`."""
def __call__(self, text: str) -> GateDecision:
return GateDecision(sanitized_text=text, disposition=WARN, reasons=())
def test_the_drop_directory_holds_exactly_the_three_containers() -> None:
"""N = 3, and N is read off the directory rather than asserted from memory.
The generator and the fasit live one level UP on purpose: Door B walks this
directory recursively, so anything else parked here would enter the run and
the denominator would stop being three.
"""
assert sorted(path.name for path in DROP.iterdir()) == list(EXPECTED_FILES)
def test_all_three_go_through_door_b(tmp_path: Path) -> None:
"""3/3 merged, offline, through the same call the operator makes."""
result = process_inbox(
DROP,
tmp_path / "bundle",
INGESTED_AT,
okf_type="note",
gate=StubGuard(), # type: ignore[arg-type]
)
assert [item.source_file for item in result.persisted] == list(EXPECTED_FILES)
assert (result.quarantined, result.rejected, result.failed) == ((), (), ())
def test_every_persisted_concept_carries_the_documents_own_strings(tmp_path: Path) -> None:
"""A concept that arrived empty would still count as persisted.
So the count is not the whole assertion: each written concept has to carry
the title, a label from the pairing table and a cell from the grid -- one
string from each of the three structures the fixture is built out of.
"""
bundle = tmp_path / "bundle"
result = process_inbox(
DROP,
bundle,
INGESTED_AT,
okf_type="note",
gate=StubGuard(), # type: ignore[arg-type]
)
for item in result.persisted:
text = (bundle / item.path).read_text(encoding="utf-8")
assert "Kravspesifikasjon for kjøkkenbelysning" in text
assert "Årsproduksjon:" in text
assert "Klasse C" in text
def test_the_fasit_is_hand_counted_and_committed_beside_the_fixtures() -> None:
"""The denominator the report divides by, frozen before the measurement ran."""
fasit = json.loads(FASIT.read_text(encoding="utf-8"))
assert sorted(fasit["documents"]) == list(EXPECTED_FILES)
for name in EXPECTED_FILES:
entry = fasit["documents"][name]
assert entry["cells"] == fasit["shared"]["cells"]
assert entry["pairs"] == fasit["shared"]["pairs"]
def test_three_synthetic_documents_do_not_make_a_row_measured() -> None:
"""One house style, N = 3, written by us -- that is exercise, not coverage."""
assert [_EVIDENCE[key] for key in (".pptx", ".odt", ".rtf")] == ["constructed"] * 3