open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index

Browse source 
Two producer-side findings from the consumer's S7c acid test (ordre 20260908T063454Z-3648220855-from-.claude), both measured on K2 before and after, both with the corpus rebuilt from scratch.

FUNN 3 -- THE FORM. The converter's default markdown writer emits simple tables, which pad every cell out to the width of the widest cell in its column. Measured on the tender's price sheet: one 594-character prose cell produced a 67 244-character whitespace carpet with runs of up to 887 characters between a label and its amount, 19 integral amounts carrying a converter `.0`, and a header row naming one column. The bytes reached a live model in 2 of 11 prompts and 0 of 11 answers. The spreadsheet row now writes pipe tables with `--columns=1` (load-bearing: the pipe writer pads to a width computed from it, so at the default 72 a narrow table gains runs of up to 45). Same sheet after: 11 048 characters, longest run 2, one row per line, 0 artificial `.0`. Spreadsheet-only, and the scoping is pinned by three digests -- the same change moves the odt fixture 1366 -> 1105, so it can fail.

The `.0` rewrite is bounded twice: to a cell whose whole content is such a number, anchored between unescaped pipes, and skipped when the literal is in the workbook's shared string table -- the converter renders the number 92 and the TEXT "92.0" identically, so the output alone cannot tell them apart. Read with zipfile and xml.etree; no new dependency.

FUNN 2 -- THE LOG LINK. `link_log_in_root_index` (95eb271) is removed. Consumption contract SS 9.2 forbids a consumer from enumerating the bundle directory unless the profile says the index is derived, so the index tree is the entire map a consumer may use and everything it links is a document: their navigator returned 630 where our pre-pass counts 629, and a corpus run's own log was citable as content. The log is still written to the bundle root (SPEC section 9); `tools/okf_consume.py` keeps its exclusion for the bundles already built with the link.

K2 rebuilt twice. BEFORE reproduces the consumer's ref exactly (`sha256-tree:f14872a0...c8a92a`, 629 concepts) and their three consume figures to the token (57 289 / 62 149 / 58 401). AFTER: 629 concepts, `merged + coded rejections = 43 = N`, new ref `sha256-tree:c26eed6a...e3261f`, 627 of 629 concepts byte-identical, 1104 of 1108 files identical to the delivered bundle.

ONE REGRESSION, MEASURED AND NOT FIXED: on the mandate-shaped question with the vocabulary bridge the priced concept moves from candidate rank 10 to 19, so `--k 12` withholds it `below_k`; `--cost-vocabulary --k 20` delivers it at 65 912 o200k. The cause is measured rather than argued -- restoring only the concept's title on the new short body ranks it 10 again. The chain ends at the orphan check (`propose.py:461`), which drops the sheet heading once a table block opens two lines below it. That is the already-reported orphan gate, and changing it is a default-ON segmentation rule affecting every document type. The specific question is unaffected: rank 1 before and after. The priced excerpt's budget share falls from 56.5 % to 9.7 %.

11 new tests (RED first), 8 mutations, 8 red, with an unmutated control green each time. One mutation survived twice before the fixture could make it fire, and both survivals are written down. 1279 -> 1287 tests. mypy --strict clean on 28 files. ruff clean. Both proposer goldens byte-unchanged. One frozen literal moved with the fix and is reported rather than hidden.

Report: docs/2026-09-08-prisform-og-loggen-k2.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 10:06:58 +02:00
commit 56ae274246
13 changed files with 1018 additions and 128 deletions
Download patch file Download diff file Expand all files Collapse all files

103
tests/fixtures/make_fixtures.py vendored
View file

@ -162,6 +162,108 @@ _XLSX_PARTS = {
}
# A price sheet, and the negative control for it, in ONE workbook.
#
# Sheet 1 mirrors the shape measured on the K2 price sheet: row 1 carries a
# single title cell, so the table's HEADER ROW names one column while the rows
# below it carry three. That is what makes a reader see one column and a
# whitespace carpet where the source has a label and an amount. Column B also
# holds one long prose cell, which is what makes the simple-table writer pad
# every other row in that column out to its width.
#
# Sheet 2 is the negative control in the same file: one column in the SOURCE,
# so there are no columns to recover and nothing for a fix to invent.
#
# THREE NUMERIC CELLS AND ONE THAT ONLY LOOKS NUMERIC. `5647500` and `250000`
# are stored as numbers and are integral; `12.5` is stored as a number and is
# not; `92.0` is a SHARED STRING. The converter renders the first two as
# `5647500.0` and `250000.0` and the last one as `92.0` -- identical output for
# a number and for text, which is the whole reason the shared string table is
# consulted before any of them is rewritten.
_PRISARK_STRINGS = (
"Prisskjema",
"Post",
"Beskrivelse",
"Sum",
"01",
"Rigging og drift av byggeplass, medregnet alt som ikke er priset "
"spesifikt nedenfor og alt som er innkalkulert i de angitte prisene",
"02",
"Andel",
"03",
"92.0",
"Notat",
"Ingen kolonner her",
"Sum ikke oppgitt",
# A cell whose own text contains a pipe and a number. The converter escapes
# the pipe inside a pipe table, and the escape is what the rewrite's
# delimiter test has to survive: a `5.0` INSIDE a cell is not a cell.
"Kode 4 | 5.0",
"04",
)
_PRISARK_SHEET1 = (
'<row r="1"><c r="A1" t="s"><v>0</v></c></row>'
'<row r="2"><c r="A2" t="s"><v>1</v></c><c r="B2" t="s"><v>2</v></c>'
'<c r="C2" t="s"><v>3</v></c></row>'
'<row r="3"><c r="A3" t="s"><v>4</v></c><c r="B3" t="s"><v>5</v></c>'
'<c r="C3"><v>5647500</v></c></row>'
'<row r="4"><c r="A4" t="s"><v>6</v></c><c r="B4" t="s"><v>7</v></c>'
'<c r="C4"><v>12.5</v></c></row>'
'<row r="5"><c r="A5" t="s"><v>8</v></c><c r="B5" t="s"><v>9</v></c>'
'<c r="C5"><v>250000</v></c></row>'
'<row r="6"><c r="A6" t="s"><v>14</v></c><c r="B6" t="s"><v>13</v></c></row>'
)
_PRISARK_SHEET2 = (
'<row r="1"><c r="A1" t="s"><v>10</v></c></row>'
'<row r="2"><c r="A2" t="s"><v>11</v></c></row>'
'<row r="3"><c r="A3" t="s"><v>12</v></c></row>'
)
def _sheet(dimension: str, rows: str) -> str:
return (
_XML
+ '<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">'
+ f'<dimension ref="{dimension}"/><sheetData>'
+ rows
+ "</sheetData></worksheet>"
)
_PRISARK_PARTS = {
"[Content_Types].xml": _XML
+ '<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">'
+ '<Default Extension="xml" ContentType="application/xml"/>'
+ '<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>'
+ '<Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml"/>'
+ '<Override PartName="/xl/worksheets/sheet1.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/>'
+ '<Override PartName="/xl/worksheets/sheet2.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/>'
+ '<Override PartName="/xl/sharedStrings.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sharedStrings+xml"/>'
+ "</Types>",
"_rels/.rels": _XLSX_PARTS["_rels/.rels"],
"xl/_rels/workbook.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet" Target="worksheets/sheet1.xml"/>'
+ '<Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet" Target="worksheets/sheet2.xml"/>'
+ '<Relationship Id="rId3" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/sharedStrings" Target="sharedStrings.xml"/>'
+ "</Relationships>",
"xl/workbook.xml": _XML
+ '<workbook xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"'
+ ' xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships">'
+ '<sheets><sheet name="Prisark" sheetId="1" r:id="rId1"/>'
+ '<sheet name="Enkeltkolonne" sheetId="2" r:id="rId2"/></sheets></workbook>',
"xl/sharedStrings.xml": _XML
+ '<sst xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"'
+ f' count="{len(_PRISARK_STRINGS)}" uniqueCount="{len(_PRISARK_STRINGS)}">'
+ "".join(f"<si><t>{value}</t></si>" for value in _PRISARK_STRINGS)
+ "</sst>",
"xl/worksheets/sheet1.xml": _sheet("A1:C6", _PRISARK_SHEET1),
"xl/worksheets/sheet2.xml": _sheet("A1:A3", _PRISARK_SHEET2),
}
def build_ooxml(parts: dict[str, str]) -> bytes:
"""Zip the parts with a fixed timestamp and no compression variance.
@ -191,6 +293,7 @@ if __name__ == "__main__":
("two-line-krav.docx", _DOCX_PARTS),
("no-styles-krav.docx", _DOCX_NO_STYLES_PARTS),
("two-line-krav.xlsx", _XLSX_PARTS),
("prisark.xlsx", _PRISARK_PARTS),
):
(HERE / name).write_bytes(build_ooxml(parts))
print(f"wrote {name}")

BIN
tests/fixtures/prisark.xlsx vendored Normal file
View file

Binary file not shown.

43
tests/test_cli_build.py
View file

@ -426,3 +426,46 @@ def test_the_packaged_modules_do_not_reach_back_into_the_clone() -> None:
for name in ("propose.py", "corpus.py", "cli.py"):
source = (PROJECT_ROOT / "src" / "llm_ingestion_okf" / name).read_text(encoding="utf-8")
assert "sys.path" not in source
def test_the_root_index_does_not_link_the_run_log(tmp_path: Path) -> None:
"""Producer and consumer must count the same documents. Measured: they did not.
`link_log_in_root_index` (`95eb271`) appended a markdown link to `log.md`
from the root index so a reader entering there could reach the one file
carrying `N`. The consumption contract SS 9.2 forbids a consumer from
enumerating the bundle directory unless the named profile says the index is
derived -- which for this profile it does not -- so the index tree IS the
whole map a consumer is allowed to use. Anything the index links is a
document, by that contract.
The cost was measured on K2 by the first consumer to walk the bundle with a
live model: their navigator followed the link and returned 630 documents
where this repository's own pre-pass counts 629, and the corpus run's own
log was reachable and citable as content. Our pre-pass excluding `log.md`
(`5a0c879`, F2) fixed the count on OUR side only; the disagreement is
produced HERE.
The log still exists at the bundle root, which is where SS 9 puts it and is
all F2 ever required. Upstream's own bundles show the link was never
required either: measured at `9a15b13`, 0 of the 24 shipped `index.md`
files name the single `log.md` in the set.
"""
inbox = inbox_with_subdirectories(tmp_path)
bundle = tmp_path / "bundle"
assert build(inbox, bundle) == 0
assert (bundle / "log.md").is_file(), "the log itself stays in the bundle"
index = (bundle / "index.md").read_text(encoding="utf-8")
assert "log.md" not in index
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools"))
import okf_consume
concepts = {
name
for name in tree(bundle)
if name.endswith(".md") and not name.endswith("index.md") and name != "log.md"
}
assert concepts, "the fixture must produce concepts for this control to mean anything"
assert len(okf_consume.enumerate_concepts(bundle)) == len(concepts)

65
tests/test_corpus_run.py
View file

@ -402,30 +402,25 @@ def test_a_second_run_into_the_same_bundle_reproduces_it_byte_for_byte(tmp_path:
assert second == first
def test_the_root_index_links_the_bundles_own_log(tmp_path: Path) -> None:
"""A log nothing links is a file on disk, not a member of the bundle.
def test_the_root_index_does_not_link_the_bundles_own_log(tmp_path: Path) -> None:
"""The log is in the bundle and out of the index. Both halves matter.
Measured on the artifact: the K2 bundle carried a conformant root `log.md`
that no index named, so a consumer walking the bundle from `index.md` --
which is the walk section 8 exists to support -- never reached the one file
carrying `N`.
The harness DID link it (`95eb271`), so a consumer entering at `index.md`
could reach the one file carrying `N`. That was a LOCAL choice and said so:
upstream's own reference bundles do not link their logs -- measured at
`9a15b13`, 0 of the 24 shipped `index.md` files name the single `log.md` in
the bundle set -- so upstream showed the link was permitted, never required.
This is a LOCAL choice, not a conformance requirement, and the distinction
is worth keeping straight. Section 9 lets `log.md` sit at any level and
section 8 has an index enumerate its directory's contents, but upstream's
own reference bundles do not link it: measured at `9a15b13`, 0 of the 24
shipped `index.md` files name the single `log.md` in the bundle set. So
upstream proves the link is not required, not that it is disallowed.
It was removed on 2026-09-08 because the cost was measured. Consumption
contract SS 9.2 forbids a consumer from enumerating the bundle directory
unless the named profile says the index is derived, so the index tree is the
ENTIRE map a consumer may use and everything it links is a document. A
consumer walking K2 that way returned 630 documents where our own pre-pass
counts 629, and the corpus run's own log was readable and citable as
content. `5a0c879` fixed the count on our side of a disagreement produced on
this one.
It is made HERE, in the harness, because the library cannot make it. The
log's content is the run's outcome, so it cannot be written before the
indexes are projected -- and an index that enumerated `log.md` off the
directory would gain the link only on the SECOND run, breaking the
rebuild-equals-incremental property the segmented bundle is built on. The
harness instead writes the link after the log, and only when it is not
already there -- a test, not an append, because the two reprojections
disagree about this line: the per-directory one drops it as a managed
entry, the flat one keeps it because its target is not an owned concept.
Section 9 still puts `log.md` at any level, and it is still written there.
"""
root = corpus(tmp_path, {"doc.md": SEGMENTABLE, "flat.md": SUBSTANTIVE})
plans = tmp_path / "plans"
@ -450,18 +445,12 @@ def test_the_root_index_links_the_bundles_own_log(tmp_path: Path) -> None:
]
assert okf_corpus_run.main(argv) == 0
index = (bundle / "index.md").read_text(encoding="utf-8")
assert "](log.md)" in index
assert index.count("](log.md)") == 1
assert (bundle / "log.md").is_file(), "the log is still written to the bundle"
for index_path in bundle.rglob("index.md"):
assert "log.md" not in index_path.read_text(encoding="utf-8"), index_path
# The link is to the log in THIS directory, so a nested index must not
# carry one: there is no `log.md` beside it to reach.
for nested in bundle.rglob("*/index.md"):
assert "](log.md)" not in nested.read_text(encoding="utf-8")
# Rebuild equals incremental, still. This is what discriminates the two
# ways the append could be wrong: a link the reprojection keeps would be
# doubled here, and one the harness forgot to re-write would vanish.
# Rebuild equals incremental, still -- the property the removed append was
# most likely to break, and the one that has to survive its removal too.
first = {
path.relative_to(bundle).as_posix(): path.read_bytes()
for path in sorted(bundle.rglob("*"))
@ -476,15 +465,14 @@ def test_the_root_index_links_the_bundles_own_log(tmp_path: Path) -> None:
assert second == first
def test_the_log_link_holds_on_the_unsegmented_path_too(tmp_path: Path) -> None:
def test_the_unsegmented_path_leaves_the_log_out_of_the_index_too(tmp_path: Path) -> None:
"""The two run modes reproject through different code, so both are pinned.
A run without `--plans-dir` uses `STRUCTURED_V1`, whose index is not
per-directory and is rewritten by the singular reprojection rather than the
per-directory one. The harness writes `log.md` in both modes, so a link
that only held on the segmented path would leave the plainer bundle with
exactly the orphan this closes -- and if that path kept the line instead of
dropping it, the second run would carry two.
per-directory one. The removed append behaved differently under the two --
the per-directory reprojection dropped the line, the flat one kept it -- so
a removal verified on one path only would prove nothing about the other.
"""
root = corpus(tmp_path, {"a.md": SUBSTANTIVE, "b.md": SUBSTANTIVE})
bundle = tmp_path / "bundle"
@ -500,8 +488,9 @@ def test_the_log_link_holds_on_the_unsegmented_path_too(tmp_path: Path) -> None:
]
assert okf_corpus_run.main(argv) == 0
assert (bundle / "log.md").is_file()
index = (bundle / "index.md").read_text(encoding="utf-8")
assert "](log.md)" in index
assert "log.md" not in index
assert okf_corpus_run.main(argv) == 0
assert (bundle / "index.md").read_text(encoding="utf-8") == index

174
tests/test_extract.py
View file

@ -9,7 +9,9 @@ call — it is pure, deterministic plumbing.
from __future__ import annotations
import hashlib
import importlib.util
import re
import sys
from pathlib import Path
@ -325,10 +327,12 @@ DOCX_TEXT = "# Krav til helning\n\n60 og 70 1:15"
# xlsx: the sheet name becomes a heading and the rows become a table. The
# label/value pairing survives on one row, which is the property that matters.
XLSX_TEXT = (
"## Krav {#sheet-1}\n\n Krav til helning \n"
" ------------------ ------\n 60 og 70 1:15"
)
#
# THIS LITERAL MOVED ONCE, deliberately, and the move is the fix reported in
# `docs/2026-09-08-prisform-og-loggen-k2.md`: the spreadsheet row now writes
# pipe tables, so the cells arrive delimited instead of padded. Every character
# of content is the same; only the table form changed.
XLSX_TEXT = "## Krav {#sheet-1}\n\n| Krav til helning | |\n|----|----|\n| 60 og 70 | 1:15 |"
# The negative control, committed rather than described: the SAME document
# without `word/styles.xml`. The body survives and the heading marker does not.
@ -457,3 +461,165 @@ def test_empty_csv_fails_fast() -> None:
with pytest.raises(ExtractionError) as excinfo:
extract_text("empty.csv", b"")
assert excinfo.value.code == "extractor_empty_csv"
# --- xlsx: the spreadsheet form a consumer has to read ---
#
# Measured on K2 and reported upstream by the first consumer to read the bundle
# with a live model (`docs/2026-09-08-prisform-og-loggen-k2.md`): the converter's
# DEFAULT markdown writer emits simple tables, which pad every cell out to the
# width of the widest cell in its column. One long prose cell therefore turns
# every other row in that column into a whitespace carpet -- 887 characters
# between a label and its amount on the real sheet -- while the header row names
# a single column because only the first source cell in row 1 is filled. The
# bytes reach the reader and the STRUCTURE does not.
#
# `prisark.xlsx` is that shape in miniature, hand-laid rather than recorded, and
# it carries its own negative control on a second sheet.
PRISARK = "prisark.xlsx"
def _table_rows(text: str) -> list[list[str]]:
"""Every pipe-table row in `text`, as its cells, in order."""
rows = []
for line in text.split("\n"):
stripped = line.strip()
if not (stripped.startswith("|") and stripped.endswith("|")):
continue
cells = [cell.strip() for cell in stripped[1:-1].split("|")]
if all(set(cell) <= set("-:") and cell for cell in cells):
continue # the header separator is punctuation, not a row
rows.append(cells)
return rows
@requires_pandoc
def test_a_spreadsheet_keeps_its_columns_one_row_per_line() -> None:
"""The label and the amount arrive as separate cells on one line.
This is the property the whole change exists for. Asserted as properties
rather than only as a frozen literal, because a literal pins bytes and says
nothing about which of them was the point.
"""
data = (FIXTURES / PRISARK).read_bytes()
with pytest.warns(ExtractionWarning):
text = extract_text(PRISARK, data)
rows = _table_rows(text)
assert [
"01",
"Rigging og drift av byggeplass, medregnet alt som ikke er "
"priset spesifikt nedenfor og alt som er innkalkulert i de angitte "
"prisene",
"5647500",
] in rows
assert ["02", "Andel", "12.5"] in rows
longest = max((len(run) for run in re.findall(r" {2,}", text)), default=0)
assert longest <= 8, f"a whitespace run of {longest} is a carpet, not a column"
@requires_pandoc
def test_an_integral_amount_loses_the_converters_decimal_and_a_real_one_keeps_it() -> None:
"""`5647500` is a number; `92.0` in the same sheet is TEXT.
The converter renders both as `<digits>.0`, so the output alone cannot tell
them apart. The shared string table can, and is what the rewrite consults --
which is why this test asserts both directions from ONE document.
"""
data = (FIXTURES / PRISARK).read_bytes()
with pytest.warns(ExtractionWarning):
text = extract_text(PRISARK, data)
assert "5647500.0" not in text
assert "250000.0" not in text
assert "12.5" in text, "a genuine decimal is a value, not a converter artefact"
assert ["03", "92.0", "250000"] in _table_rows(text), (
"a shared-string cell reading 92.0 is author text and survives verbatim"
)
assert "Kode 4 \\| 5.0" in text, (
"a `5.0` INSIDE a cell is not a cell: the delimiter test is what sees that"
)
@requires_pandoc
def test_a_single_column_sheet_gains_no_columns() -> None:
"""The negative control, in the same document as the case it controls.
Sheet 2 has ONE column in the source. There is nothing to recover, so the
fix must not invent a second cell anywhere on it. Its three values arrive
in order and alone.
It is NOT byte-identical before and after the change, and that is measured
rather than glossed: the writer emits a pipe table for every table it
writes, so a one-column table changes delimiter form too. What must not
change is the cell content and the column count.
"""
data = (FIXTURES / PRISARK).read_bytes()
with pytest.warns(ExtractionWarning):
text = extract_text(PRISARK, data)
single = text.split("## Enkeltkolonne")[1]
rows = _table_rows(single)
assert [cells for cells in rows if any(cells)] == [
["Notat"],
["Ingen kolonner her"],
["Sum ikke oppgitt"],
]
# The scoping control. The same writer change applied to the other four office
# rows was MEASURED to move them (the odt fixture 1366 -> 1105 characters), so
# this digest can fail; it is not a tautology. The change is deliberately
# spreadsheet-only: a spreadsheet IS a grid and has no prose fallback, while
# moving docx/pptx/odt/rtf would move a corpus denominator nothing has measured.
# A red here means the writer stopped being scoped -- read the diff and decide.
OFFICE_TEXT_DIGESTS = {
"k2-office/krav-tekstdokument.odt": (
"58c9776f0d7f2b2a3a9d2774e4ae243b265c31b5b6b96914ef4db419fa66e4e2"
),
"k2-office/krav-presentasjon.pptx": (
"752420a04d651a416938ff9f0b3c2de5849bd2ea1d52063dd88aaae65ab99b90"
),
"k2-office/krav-rikt-tekstformat.rtf": (
"79cbf756eb482bb603f82c171d11efe74b9ba62ab9ef679ecd6bb8c3b3740ffb"
),
}
@requires_pandoc
@pytest.mark.parametrize("relative", sorted(OFFICE_TEXT_DIGESTS))
def test_the_other_office_rows_are_untouched_by_the_spreadsheet_writer(relative: str) -> None:
path = FIXTURES / relative
with pytest.warns(ExtractionWarning):
text = extract_text(path.name, path.read_bytes())
assert hashlib.sha256(text.encode("utf-8")).hexdigest() == OFFICE_TEXT_DIGESTS[relative]
# The whole fixture, frozen against the same named converter version as the
# literals above. The property tests say WHAT matters; this one catches any
# other byte moving without anybody noticing.
PRISARK_TEXT = (
"## Prisark {#sheet-1}\n\n"
"| Prisskjema | | |\n"
"|----|----|----|\n"
"| Post | Beskrivelse | Sum |\n"
"| 01 | Rigging og drift av byggeplass, medregnet alt som ikke er priset "
"spesifikt nedenfor og alt som er innkalkulert i de angitte prisene | 5647500 |\n"
"| 02 | Andel | 12.5 |\n"
"| 03 | 92.0 | 250000 |\n"
"| 04 | Kode 4 \\| 5.0 | |\n\n"
"## Enkeltkolonne {#sheet-2}\n\n"
"| Notat |\n"
"|----|\n"
"| Ingen kolonner her |\n"
"| Sum ikke oppgitt |"
)
@requires_pandoc
def test_prisark_extracts_to_its_frozen_text() -> None:
data = (FIXTURES / PRISARK).read_bytes()
with pytest.warns(ExtractionWarning):
assert extract_text(PRISARK, data) == PRISARK_TEXT

23
tests/test_okf_consume.py
View file

@ -115,21 +115,31 @@ def _copy_bundle(source: Path, target: Path) -> None:
def test_the_index_walk_excludes_a_linked_log_from_concept_navigation(tmp_path: Path) -> None:
"""A linked `log.md` is bundle metadata, not a concept -- measured on K2 (S7 F2).
`link_log_in_root_index` (`corpus.py`) links a run's own log from the root
index so a reader entering at `index.md` can reach it (`95eb271`). That link
`link_log_in_root_index` (`corpus.py`, `95eb271`) linked a run's own log from
the root index so a reader entering at `index.md` could reach it. That link
makes the log reachable by the same walk this instrument uses to enumerate
concepts, and a walk that does not distinguish "linked" from "concept"
counts it as a 630th concept on a 629-concept bundle -- exactly what the S7
acid test measured, with the log then ranked and cut like real content.
THE PRODUCER NO LONGER WRITES THAT LINK (2026-09-08), so the fixture writes
it here instead. The exclusion stays and is not dead code: every bundle
built between `95eb271` and that removal carries the link, including the
ones consumers are reading today, and this instrument must count 629 on
those too.
"""
from llm_ingestion_okf.corpus import LOG_NAME, link_log_in_root_index
from llm_ingestion_okf.profiles import SEGMENTED_OKF_V0_2
from llm_ingestion_okf.corpus import LOG_NAME
copy = tmp_path / "bundle"
_copy_bundle(GOLDEN, copy)
(copy / LOG_NAME).write_text("# Corpus run history\n\nN = 3\n", encoding="utf-8")
link_log_in_root_index(copy, SEGMENTED_OKF_V0_2)
index = (copy / "index.md").read_text(encoding="utf-8")
index_path = copy / "index.md"
index_path.write_text(
index_path.read_text(encoding="utf-8") + f"- [Corpus run history]({LOG_NAME})\n",
encoding="utf-8",
newline="",
)
index = index_path.read_text(encoding="utf-8")
assert "](log.md)" in index, (
"the fixture must actually link the log for this control to mean anything"
)
@ -1046,6 +1056,7 @@ def test_no_corpus_document_name_reaches_any_file_this_work_tracks() -> None:
PROJECT_ROOT / "docs" / "2026-09-07-okf-konsumskill-maaling.md",
PROJECT_ROOT / "docs" / "2026-09-08-blindsone-below-k-k2.md",
PROJECT_ROOT / "docs" / "2026-09-08-blindsone-laas2-budsjett-k2.md",
PROJECT_ROOT / "docs" / "2026-09-08-prisform-og-loggen-k2.md",
PROJECT_ROOT / "README.md",
PROJECT_ROOT / "CLAUDE.md",
]