open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

test(extract): the file that IS the product had no reader, so state what one owes

Browse source 
RED, 11 failing. Reproduced first, before any code: `okf build` on the folder
a publisher's own viewer delivers gives **110 of 110 unreadable, 0 plans, exit
2**, with `no extractor is registered for file extension '.xml'`. The
conservation identity `merged + coded rejections == N` is never written at all
-- the run aborts earlier on `FAILED - no segmentation plans` -- so the hole
was not even visible as a count.

The one xml file IS the whole product: R761 Prosesskoden:2025, the document
round 12 met as a 701-page PDF, in NISO-STS form. Measured on the file with
`xml.etree.ElementTree`: 7 715 `<sec>`, 2 761 with a `<title>`, 4 954 with a
`<label>` and no title, 10 `<table-wrap>`, 12 528 `<p>`, root `<standard>`, 0
`<!DOCTYPE` and 0 `<!ENTITY`. Its `<sec>`-nesting depths over the titled
sections are 28/118/500/1141/868/97/9 -- row for row the fasit's own
distribution. The ceiling is therefore structural rather than computed.

FOUR HAND-WRITTEN FIXTURES, none through `make_fixtures.py` and none
serialised by `ElementTree`: a library that writes and reads its own format
proves only that it agrees with itself. A known-positive STS mini, generic
non-STS xml, a `<!DOCTYPE` with an entity expansion, and a malformed file.

Two assertions that already existed are extended rather than duplicated: the
converter fence, because a file routed to the converter is read by a second
parser that never sees this reader's DTD refusal, and the evidence table,
because a row without a class is the failure that test exists for.

pytest -q: 11 failed, 1554 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 03:30:48 +02:00
commit 324cd0b1e5
7 changed files with 306 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

9
tests/fixtures/xml-malformed.xml vendored Normal file
View file

@ -0,0 +1,9 @@
<standard>
<body>
<sec id="s-1">
<label>1</label>
<title>Uavsluttet
<p>Denne taggen lukkes aldri.</p>
</sec>
</body>
</standard>