open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity
llm-ingestion-okf/tests/fixtures/accounting/make_accounting_fixtures.py
Kjell Tore Guttormsen ed8d9d709f
 test(accounting): row 6 sees a refusal, a second real corpus, and 34 of 34 mutants 
Row 6 was GREEN with R761 100 % rejected: every element of a refused document
is booked as a coded rejection, so u = 0 and d = 0 and the bundle is empty.
`refused_whole` asks that question on its own now -- the build order asked for
an honest red there, and PM re-measured the green on 2026-09-18 with
`okf build` exiting 1 unseen.

A skipped row no longer leaves the verdict unqualified (`GATE GREEN (row 6 not
run: ...)`), and the exit code is non-zero locally when a corpus source is on
the machine and its row did not run. m-2.

N200 Vegbygging:2024 joins R761 as a second real corpus. R761 holds 0 `fig`,
0 formulas and 0 references, so the only real corpus could not have found the
hole in the STS role map; N200 carries 194 citations, 49 figures and 135
footnotes. A `.json` whose root holds an STS node tree is counted as STS
rather than as keys and leaves -- the container is not the content.

M-4: the review's 26 mutants, ported to the code as it stands, plus 8 for the
new checks. 34 of 34 killed. `tools/okf_gate_mutants.py` runs on a copy of the
tree, and a pattern that does not match is an ERROR and exit 2 -- a mutant
that could not be applied was never measured. That fired once, on M13, after a
refactor moved the line it edits.

m-3: `APPROVED_EXCEPTIONS` was read by no row, so approving one changed
nothing. Each pair is now checked against the witness's own vocabulary and the
run says why it moves no denominator. The gate also prints its OWN limits
beside the verdict, m-5 among them.

The product's accounting tests state the new truth instead of the old one:
`okf build --accounting` over the fixture corpus exits 1 with SIX unaccounted
elements in its own vocabulary -- its first real finding, reachable only now
that fixtures carry the constructs. Four shared element names disagree with
the witness, each pinned with its cause; one of the four is a double count
this package makes (a text box's paragraph, once inside the box and again in
the paragraph carrying it).

Three fixture defects were found and fixed while building them, each of which
would have reported a loss the build never had: a shared string table not
related to the workbook, a `graphicData` with no `uri`, and an odt
`styles.xml` without `<office:styles/>`.

Report: docs/2026-09-18-regnskapsgaten-herdet.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 02:48:34 +02:00

357 lines
17 KiB
Python
Raw Blame History

"""The accounting corpus's SECOND document per format: the elements the
witness could not see until 2026-09-18.
An independent review of the gate found that the witness -- and therefore the
whole accounting, because rows 2 and 3 require the build's inventory to EQUAL
it -- counted no header, no comment, no speaker note, no hidden sheet, no
formula, no citation and no figure caption. What nothing counts, nothing can
lose visibly. It also found that 20 of 63 element types had a count of ZERO in
their only fixture, so six of seven witness mutants survived.
These documents are written BY HAND, part by part, for the reason the XML
fixtures state: a library that writes and then reads its own format proves
only that it agrees with itself. Every count they carry is written down in
`tests/fixtures/README.md` by a person reading these strings, not by running
the witness over them.
python3 tests/fixtures/accounting/make_accounting_fixtures.py
"""
from __future__ import annotations
import io
import zipfile
from pathlib import Path
HERE = Path(__file__).resolve().parent
CORPUS = HERE / "corpus"
_ZIP_DATE = (2020, 1, 1, 0, 0, 0)
_XML = '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'
_W = 'xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"'
_A = 'xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"'
_P = 'xmlns:p="http://schemas.openxmlformats.org/presentationml/2006/main"'
_S = 'xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"'
def build_zip(parts: dict[str, str | bytes]) -> bytes:
"""Zip the parts with a fixed timestamp, so a regeneration that changes
nothing leaves the bytes alone and `git diff --quiet` stays a real check."""
out = io.BytesIO()
with zipfile.ZipFile(out, "w", compression=zipfile.ZIP_DEFLATED) as archive:
for name, payload in parts.items():
info = zipfile.ZipInfo(name, date_time=_ZIP_DATE)
info.compress_type = zipfile.ZIP_DEFLATED
archive.writestr(info, payload)
return out.getvalue()
# --- docx: a header, a footer, a comment, an endnote and a text box ----------
_DOCX_BODY = (
'<w:p><w:pPr><w:pStyle w:val="Heading1"/></w:pPr>'
"<w:r><w:t>Krav til gangbruer</w:t></w:r></w:p>"
"<w:p><w:r><w:t>Gangbruer skal ha rekkverk paa begge sider.</w:t></w:r></w:p>"
"<w:tbl><w:tr>"
"<w:tc><w:p><w:r><w:t>Bredde</w:t></w:r></w:p></w:tc>"
"<w:tc><w:p><w:r><w:t>3,0 m</w:t></w:r></w:p></w:tc>"
"</w:tr></w:tbl>"
"<w:p><w:r><w:pict><w:txbxContent>"
"<w:p><w:r><w:t>Merk: kravet gjelder ikke midlertidige bruer.</w:t></w:r></w:p>"
"</w:txbxContent></w:pict></w:r></w:p>"
)
_DOCX_PARTS: dict[str, str | bytes] = {
"[Content_Types].xml": _XML
+ '<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">'
+ '<Default Extension="xml" ContentType="application/xml"/>'
+ '<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.'
+ 'relationships+xml"/>'
+ '<Override PartName="/word/document.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.wordprocessingml.document.main+xml"/>'
+ '<Override PartName="/word/styles.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.wordprocessingml.styles+xml"/>'
+ "</Types>",
"_rels/.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/officeDocument" Target="word/document.xml"/></Relationships>',
"word/_rels/document.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/styles" Target="styles.xml"/></Relationships>',
"word/styles.xml": _XML
+ f"<w:styles {_W}>"
+ '<w:style w:type="paragraph" w:styleId="Heading1"><w:name w:val="heading 1"/></w:style>'
+ "</w:styles>",
"word/document.xml": _XML + f"<w:document {_W}><w:body>{_DOCX_BODY}</w:body></w:document>",
"word/header1.xml": _XML
+ f"<w:hdr {_W}><w:p><w:r><w:t>Utkast - gjelder ikke etter 2026-01-01</w:t></w:r></w:p>"
+ "</w:hdr>",
"word/footer1.xml": _XML
+ f"<w:ftr {_W}><w:p><w:r><w:t>Statens vegvesen, side 1</w:t></w:r></w:p></w:ftr>",
"word/comments.xml": _XML
+ f"<w:comments {_W}>"
+ '<w:comment w:id="1"><w:p><w:r><w:t>Unntak: gjelder IKKE gangbruer i tunnel.'
+ "</w:t></w:r></w:p></w:comment></w:comments>",
"word/footnotes.xml": _XML
+ f"<w:footnotes {_W}>"
+ '<w:footnote w:id="0"><w:p><w:r><w:t>separator</w:t></w:r></w:p></w:footnote>'
+ '<w:footnote w:id="2"><w:p><w:r><w:t>Se haandbok N400 kapittel 5.</w:t></w:r></w:p>'
+ "</w:footnote></w:footnotes>",
"word/endnotes.xml": _XML
+ f"<w:endnotes {_W}>"
+ '<w:endnote w:id="0"><w:p><w:r><w:t>separator</w:t></w:r></w:p></w:endnote>'
+ '<w:endnote w:id="3"><w:p><w:r><w:t>Kravet ble skjerpet i 2024.</w:t></w:r></w:p>'
+ "</w:endnote></w:endnotes>",
}
# --- pptx: a speaker note and a hidden slide --------------------------------
def _slide(title: str, body: str, *, hidden: bool = False) -> str:
show = ' show="0"' if hidden else ""
return (
_XML
+ f"<p:sld {_P} {_A}{show}><p:cSld><p:spTree>"
+ '<p:sp><p:nvSpPr><p:nvPr><p:ph type="title"/></p:nvPr></p:nvSpPr>'
+ f"<p:txBody><a:p><a:r><a:t>{title}</a:t></a:r></a:p></p:txBody></p:sp>"
+ "<p:sp><p:nvSpPr><p:nvPr/></p:nvSpPr>"
+ f"<p:txBody><a:p><a:r><a:t>{body}</a:t></a:r></a:p></p:txBody></p:sp>"
+ "<p:graphicFrame><a:graphic><a:graphicData "
+ 'uri="http://schemas.openxmlformats.org/drawingml/2006/table">'
+ "<a:tbl><a:tr>"
+ "<a:tc><a:txBody><a:p><a:r><a:t>Post</a:t></a:r></a:p></a:txBody></a:tc>"
+ "<a:tc><a:txBody><a:p><a:r><a:t>84.1</a:t></a:r></a:p></a:txBody></a:tc>"
+ "</a:tr></a:tbl></a:graphicData></a:graphic></p:graphicFrame>"
+ "</p:spTree></p:cSld></p:sld>"
)
_PPTX_PARTS: dict[str, str | bytes] = {
"[Content_Types].xml": _XML
+ '<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">'
+ '<Default Extension="xml" ContentType="application/xml"/>'
+ '<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.'
+ 'relationships+xml"/>'
+ '<Override PartName="/ppt/presentation.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.presentationml.presentation.main+xml"/>'
+ "".join(
f'<Override PartName="/ppt/slides/slide{n}.xml" ContentType="application/vnd.'
f'openxmlformats-officedocument.presentationml.slide+xml"/>'
for n in (1, 2)
)
+ '<Override PartName="/ppt/notesSlides/notesSlide1.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.presentationml.notesSlide+xml"/>'
+ "</Types>",
"_rels/.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/officeDocument" Target="ppt/presentation.xml"/></Relationships>',
"ppt/_rels/presentation.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/slide" Target="slides/slide1.xml"/>'
+ '<Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/slide" Target="slides/slide2.xml"/></Relationships>',
"ppt/slides/_rels/slide1.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/notesSlide" Target="../notesSlides/notesSlide1.xml"/></Relationships>',
"ppt/presentation.xml": _XML
+ f"<p:presentation {_P}><p:sldIdLst>"
+ '<p:sldId id="256" r:id="rId1"/><p:sldId id="257" r:id="rId2"/>'
+ "</p:sldIdLst></p:presentation>",
"ppt/slides/slide1.xml": _slide("Prosess 84 Konstruksjoner", "Toleranser er gitt i tabell."),
"ppt/slides/slide2.xml": _slide("Utgaatt lysbilde", "Ikke vis dette.", hidden=True),
"ppt/notesSlides/notesSlide1.xml": _XML
+ f"<p:notes {_P} {_A}><p:cSld><p:spTree><p:sp><p:txBody>"
+ "<a:p><a:r><a:t>Husk aa nevne at toleranseklassen er skjerpet.</a:t></a:r></a:p>"
+ "</p:txBody></p:sp></p:spTree></p:cSld></p:notes>",
}
# --- xlsx: a hidden sheet and a formula --------------------------------------
_XLSX_STRINGS = ["Post", "Enhet", "Mengde", "Sum", "Internt", "Kladd"]
_XLSX_PARTS: dict[str, str | bytes] = {
"[Content_Types].xml": _XML
+ '<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">'
+ '<Default Extension="xml" ContentType="application/xml"/>'
+ '<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.'
+ 'relationships+xml"/>'
+ '<Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-'
+ 'officedocument.spreadsheetml.sheet.main+xml"/>'
# Declaring the parts is not decoration. Without the sharedStrings
# override every `t="s"` cell converts to an EMPTY cell -- the fixture
# would have reported four cells lost that the build never lost.
+ '<Override PartName="/xl/worksheets/sheet1.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.spreadsheetml.worksheet+xml"/>'
+ '<Override PartName="/xl/worksheets/sheet2.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.spreadsheetml.worksheet+xml"/>'
+ '<Override PartName="/xl/sharedStrings.xml" ContentType="application/vnd.'
+ 'openxmlformats-officedocument.spreadsheetml.sharedStrings+xml"/>'
+ "</Types>",
"_rels/.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/officeDocument" Target="xl/workbook.xml"/></Relationships>',
"xl/workbook.xml": _XML
+ f'<workbook {_S} xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships"><sheets>'
+ '<sheet name="Mengder" sheetId="1" r:id="rId1"/>'
+ '<sheet name="Internt" sheetId="2" state="hidden" r:id="rId2"/>'
+ "</sheets></workbook>",
"xl/_rels/workbook.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/worksheet" Target="worksheets/sheet1.xml"/>'
+ '<Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/worksheet" Target="worksheets/sheet2.xml"/>'
# The shared string table is reached through the WORKBOOK's relationship,
# not by its path: without this line every `t="s"` cell converts empty and
# the fixture reports four cells lost that the build never lost.
+ '<Relationship Id="rId3" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/sharedStrings" Target="sharedStrings.xml"/></Relationships>',
"xl/sharedStrings.xml": _XML
+ f'<sst {_S} count="{len(_XLSX_STRINGS)}" uniqueCount="{len(_XLSX_STRINGS)}">'
+ "".join(f"<si><t>{value}</t></si>" for value in _XLSX_STRINGS)
+ "</sst>",
"xl/worksheets/sheet1.xml": _XML
+ f'<worksheet {_S}><dimension ref="A1:C3"/><sheetData>'
+ '<row r="1"><c r="A1" t="s"><v>0</v></c><c r="B1" t="s"><v>1</v></c>'
+ '<c r="C1" t="s"><v>2</v></c></row>'
+ '<row r="2"><c r="A2" t="s"><v>3</v></c><c r="B2"><v>12</v></c>'
+ '<c r="C2"><f>B2*2</f><v>24</v></c></row>'
+ '</sheetData><drawing r:id="rId1" xmlns:r="http://schemas.openxmlformats.org/'
+ 'officeDocument/2006/relationships"/></worksheet>',
"xl/drawings/drawing1.xml": _XML
+ '<xdr:wsDr xmlns:xdr="http://schemas.openxmlformats.org/drawingml/2006/'
+ 'spreadsheetDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main">'
+ "<xdr:twoCellAnchor><xdr:pic><xdr:nvPicPr>"
+ '<xdr:cNvPr id="1" name="Diagram"/>'
+ "</xdr:nvPicPr></xdr:pic></xdr:twoCellAnchor></xdr:wsDr>",
"xl/worksheets/_rels/sheet1.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/'
+ 'relationships/drawing" Target="../drawings/drawing1.xml"/></Relationships>',
"xl/worksheets/sheet2.xml": _XML
+ f'<worksheet {_S}><dimension ref="A1:A1"/><sheetData>'
+ '<row r="1"><c r="A1" t="s"><v>5</v></c></row>'
+ "</sheetData></worksheet>",
}
# --- odt: a header, a list, an annotation and a picture ----------------------
_ODT_NS = (
'xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" '
'xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" '
'xmlns:table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" '
'xmlns:draw="urn:oasis:names:tc:opendocument:xmlns:drawing:1.0" '
'xmlns:style="urn:oasis:names:tc:opendocument:xmlns:style:1.0" '
'xmlns:xlink="http://www.w3.org/1999/xlink"'
)
_ODT_PARTS: dict[str, str | bytes] = {
"mimetype": "application/vnd.oasis.opendocument.text",
"META-INF/manifest.xml": _XML
+ '<manifest:manifest xmlns:manifest="urn:oasis:names:tc:opendocument:xmlns:manifest:1.0">'
+ '<manifest:file-entry manifest:full-path="/" manifest:media-type="application/vnd.oasis.'
+ 'opendocument.text"/>'
+ '<manifest:file-entry manifest:full-path="content.xml" manifest:media-type="text/xml"/>'
+ "</manifest:manifest>",
"content.xml": _XML
+ f'<office:document-content {_ODT_NS} office:version="1.3">'
+ "<office:body><office:text>"
+ '<text:h text:outline-level="1">Drift av gangbruer</text:h>'
+ "<text:p>Gangbruer inspiseres hvert aar.</text:p>"
+ "<text:list><text:list-item><text:p>Rekkverk</text:p></text:list-item>"
+ "<text:list-item><text:p>Dekke</text:p></text:list-item></text:list>"
+ "<text:p>Se figuren under."
+ '<draw:frame><draw:image xlink:href="graphics/figur-84-1.png"/></draw:frame></text:p>'
+ "<office:annotation><text:p>Sjekk denne mot N400 foer utsendelse.</text:p>"
+ "</office:annotation>"
+ "<table:table><table:table-row>"
+ "<table:table-cell><text:p>Type</text:p></table:table-cell>"
+ "<table:table-cell><text:p>Gangbru</text:p></table:table-cell>"
+ "</table:table-row></table:table>"
+ "</office:text></office:body></office:document-content>",
"styles.xml": _XML
+ f'<office:document-styles {_ODT_NS} office:version="1.3">'
+ "<office:styles/><office:master-styles>"
+ '<style:master-page style:name="Standard">'
+ "<style:header><text:p>Intern arbeidsversjon</text:p></style:header>"
+ "<style:footer><text:p>Vegdirektoratet</text:p></style:footer>"
+ "</style:master-page></office:master-styles></office:document-styles>",
}
# --- rtf: a picture ----------------------------------------------------------
_RTF = (
r"{\rtf1\ansi\deff0{\fonttbl{\f0 Times New Roman;}}"
r"\pard Figur 84-1 viser prinsippet.\par"
r"\pard{\pict\pngblip\picw16\pich16 89504e470d0a1a0a}\par"
"}"
)
# --- html: a picture and a caption -------------------------------------------
_HTML = """<!DOCTYPE html>
<html lang="no">
<head><title>Figur 84-1</title></head>
<body>
<h1>Figur 84-1</h1>
<p>Prinsippet for toleranseklasser.</p>
<img src="graphics/figur-84-1.png" alt="Prinsippskisse">
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 mm</td></tr></table>
<ul><li>Klasse A</li><li>Klasse B</li></ul>
</body>
</html>
"""
# --- sts: a citation, a formula, a figure with a caption, a table, a footnote -
_STS = """<standard>
<front><std-ident><doc-number>R762</doc-number><year>2025</year></std-ident></front>
<body>
<sec><label>85</label><title>Vegdekker</title>
<p>Dekket skal ha jevnhet etter <mixed-citation>NS-EN 13036-1:2010</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>IRI</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs vegbanen.</p></caption>
<graphic xlink:href="figur-84-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<table-wrap><label>Tabell 85-1</label>
<table><tr><th>Klasse</th><th>IRI</th></tr><tr><td>1</td><td>1,5</td></tr></table>
</table-wrap>
<list><list-item><p>Maales hvert 20. meter.</p></list-item></list>
<fn><p>Gjelder ikke gang- og sykkelveger.</p></fn>
</sec>
</body>
</standard>
"""
def main() -> int:
written = {
"topptekst-og-kommentar.docx": build_zip(_DOCX_PARTS),
"notater-og-skjult.pptx": build_zip(_PPTX_PARTS),
"skjult-ark-og-formel.xlsx": build_zip(_XLSX_PARTS),
"liste-og-bilde.odt": build_zip(_ODT_PARTS),
"bilde.rtf": _RTF.encode("latin-1"),
"figur.html": _HTML.encode("utf-8"),
"sts-rikt.xml": _STS.encode("utf-8"),
}
for name, payload in written.items():
(CORPUS / name).write_bytes(payload)
print(f"wrote {name}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Reference in a new issue View git blame Copy permalink