open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

test(extract): the file that IS the product had no reader, so state what one owes

Browse source 
RED, 11 failing. Reproduced first, before any code: `okf build` on the folder
a publisher's own viewer delivers gives **110 of 110 unreadable, 0 plans, exit
2**, with `no extractor is registered for file extension '.xml'`. The
conservation identity `merged + coded rejections == N` is never written at all
-- the run aborts earlier on `FAILED - no segmentation plans` -- so the hole
was not even visible as a count.

The one xml file IS the whole product: R761 Prosesskoden:2025, the document
round 12 met as a 701-page PDF, in NISO-STS form. Measured on the file with
`xml.etree.ElementTree`: 7 715 `<sec>`, 2 761 with a `<title>`, 4 954 with a
`<label>` and no title, 10 `<table-wrap>`, 12 528 `<p>`, root `<standard>`, 0
`<!DOCTYPE` and 0 `<!ENTITY`. Its `<sec>`-nesting depths over the titled
sections are 28/118/500/1141/868/97/9 -- row for row the fasit's own
distribution. The ceiling is therefore structural rather than computed.

FOUR HAND-WRITTEN FIXTURES, none through `make_fixtures.py` and none
serialised by `ElementTree`: a library that writes and reads its own format
proves only that it agrees with itself. A known-positive STS mini, generic
non-STS xml, a `<!DOCTYPE` with an entity expansion, and a malformed file.

Two assertions that already existed are extended rather than duplicated: the
converter fence, because a file routed to the converter is read by a second
parser that never sees this reader's DTD refusal, and the evidence table,
because a row without a class is the failure that test exists for.

pytest -q: 11 failed, 1554 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 03:30:48 +02:00
commit 324cd0b1e5
7 changed files with 306 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

54
tests/fixtures/sts-mini.xml vendored Normal file
View file

@ -0,0 +1,54 @@
<standard>
<front>
<std-doc-meta>
<title-wrap><full>Testnormal for fiksturbruk</full></title-wrap>
</std-doc-meta>
</front>
<body>
<sec id="s-forord" sec-type="foreword">
<title>Forord</title>
<p>Dette forordet baerer ingen nummerering.</p>
</sec>
<sec id="s-1">
<label>1</label>
<title>Bruksomraade</title>
<p>Normalen gjelder for testing av ekstraktoren.</p>
<sec id="s-1-1">
<label>1.1</label>
<title>Omfang</title>
<p>Omfanget dekker hele arbeidet.</p>
<sec id="s-1-1-1">
<label>1.1.1</label>
<title>Materialer</title>
<p>Materialene skal vaere godkjente.</p>
<sec id="s-1-1-1-a" sec-type="spec">
<label>a)</label>
<p>Betong skal ha fasthetsklasse B35.</p>
</sec>
<sec id="s-1-1-1-b" sec-type="spec">
<label>b)</label>
<p>Armering skal vaere av klasse B500NC.</p>
</sec>
</sec>
</sec>
</sec>
<sec id="s-2">
<label>2</label>
<title>Tabeller</title>
<p>Tabellen under er en enkelt tabellblokk.</p>
<table-wrap id="tab-2-1">
<label>Tabell 2-1</label>
<table>
<tbody>
<tr><td>Prosess</td><td>Enhet</td></tr>
<tr><td>Sprengning</td><td>kubikkmeter</td></tr>
</tbody>
</table>
</table-wrap>
<list list-type="alpha-lower">
<list-item><p>Foerste punkt i lista.</p></list-item>
<list-item><p>Andre punkt i lista.</p></list-item>
</list>
</sec>
</body>
</standard>