test(extract): the file that IS the product had no reader, so state what one owes
RED, 11 failing. Reproduced first, before any code: `okf build` on the folder a publisher's own viewer delivers gives **110 of 110 unreadable, 0 plans, exit 2**, with `no extractor is registered for file extension '.xml'`. The conservation identity `merged + coded rejections == N` is never written at all -- the run aborts earlier on `FAILED - no segmentation plans` -- so the hole was not even visible as a count. The one xml file IS the whole product: R761 Prosesskoden:2025, the document round 12 met as a 701-page PDF, in NISO-STS form. Measured on the file with `xml.etree.ElementTree`: 7 715 `<sec>`, 2 761 with a `<title>`, 4 954 with a `<label>` and no title, 10 `<table-wrap>`, 12 528 `<p>`, root `<standard>`, 0 `<!DOCTYPE` and 0 `<!ENTITY`. Its `<sec>`-nesting depths over the titled sections are 28/118/500/1141/868/97/9 -- row for row the fasit's own distribution. The ceiling is therefore structural rather than computed. FOUR HAND-WRITTEN FIXTURES, none through `make_fixtures.py` and none serialised by `ElementTree`: a library that writes and reads its own format proves only that it agrees with itself. A known-positive STS mini, generic non-STS xml, a `<!DOCTYPE` with an entity expansion, and a malformed file. Two assertions that already existed are extended rather than duplicated: the converter fence, because a file routed to the converter is read by a second parser that never sees this reader's DTD refusal, and the evidence table, because a row without a class is the failure that test exists for. pytest -q: 11 failed, 1554 passed, 1 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
525d68ee90
commit
324cd0b1e5
7 changed files with 306 additions and 2 deletions
16
tests/fixtures/README.md
vendored
16
tests/fixtures/README.md
vendored
|
|
@ -20,6 +20,22 @@ and reproducible from that one file.
|
|||
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape R761 carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: 2 763 nodes in, 2 762 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
|
||||
| `three-page-krav.pdf` | Three pages, one line of text each, and **the middle page carries no text operators**. The extractor drops empty pages, so the last page's text belongs to page 3 — which is what separates a page NUMBER from a count of the pages that produced text. Two pages could not tell those apart. |
|
||||
|
||||
## The XML fixtures
|
||||
|
||||
Hand-written text, directly in this directory — **not** through
|
||||
`make_fixtures.py`, which builds binary containers, and never serialised by
|
||||
`ElementTree`. The reason is the same one the OOXML policy states: a library
|
||||
that writes and then reads its own format proves only that it agrees with
|
||||
itself, and an `ElementTree`-written fixture would stay green through any
|
||||
round-trip-symmetric defect.
|
||||
|
||||
| Fixture | What it is for |
|
||||
|---|---|
|
||||
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in R761. The lettered points are what the 4 954 label-only `<sec>` in that document look like: promoted to headings they would bury its own 2 761. |
|
||||
| `generic-feed.xml` | Known-negative: XML that is **not** STS. It must produce text and ONE plan — never zero, never a crash, and never element names promoted to headings. |
|
||||
| `xml-doctype-bomb.xml` | Known-negative, security: a `<!DOCTYPE` with a small nested-entity expansion. It must be refused by `code`, and the test asserts the expansion appears in **no** output, including the error text. Small on purpose — the point is that it is never parsed, not that it detonates. |
|
||||
| `xml-malformed.xml` | Known-negative: an unterminated tag must raise a typed `ExtractionError`, not leak `ParseError` and not yield zero concepts in silence. |
|
||||
|
||||
## The office fixtures
|
||||
|
||||
`two-line-krav.docx`, `no-styles-krav.docx` and `two-line-krav.xlsx` are
|
||||
|
|
|
|||
14
tests/fixtures/generic-feed.xml
vendored
Normal file
14
tests/fixtures/generic-feed.xml
vendored
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
<rss version="2.0">
|
||||
<channel>
|
||||
<title>Driftsmeldinger</title>
|
||||
<description>Meldinger om drift og vedlikehold.</description>
|
||||
<item>
|
||||
<title>Stengt tunnel</title>
|
||||
<description>Tunnelen er stengt for vedlikehold natt til fredag.</description>
|
||||
</item>
|
||||
<item>
|
||||
<title>Redusert framkommelighet</title>
|
||||
<description>Ett felt er stengt i anleggsperioden.</description>
|
||||
</item>
|
||||
</channel>
|
||||
</rss>
|
||||
54
tests/fixtures/sts-mini.xml
vendored
Normal file
54
tests/fixtures/sts-mini.xml
vendored
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
<standard>
|
||||
<front>
|
||||
<std-doc-meta>
|
||||
<title-wrap><full>Testnormal for fiksturbruk</full></title-wrap>
|
||||
</std-doc-meta>
|
||||
</front>
|
||||
<body>
|
||||
<sec id="s-forord" sec-type="foreword">
|
||||
<title>Forord</title>
|
||||
<p>Dette forordet baerer ingen nummerering.</p>
|
||||
</sec>
|
||||
<sec id="s-1">
|
||||
<label>1</label>
|
||||
<title>Bruksomraade</title>
|
||||
<p>Normalen gjelder for testing av ekstraktoren.</p>
|
||||
<sec id="s-1-1">
|
||||
<label>1.1</label>
|
||||
<title>Omfang</title>
|
||||
<p>Omfanget dekker hele arbeidet.</p>
|
||||
<sec id="s-1-1-1">
|
||||
<label>1.1.1</label>
|
||||
<title>Materialer</title>
|
||||
<p>Materialene skal vaere godkjente.</p>
|
||||
<sec id="s-1-1-1-a" sec-type="spec">
|
||||
<label>a)</label>
|
||||
<p>Betong skal ha fasthetsklasse B35.</p>
|
||||
</sec>
|
||||
<sec id="s-1-1-1-b" sec-type="spec">
|
||||
<label>b)</label>
|
||||
<p>Armering skal vaere av klasse B500NC.</p>
|
||||
</sec>
|
||||
</sec>
|
||||
</sec>
|
||||
</sec>
|
||||
<sec id="s-2">
|
||||
<label>2</label>
|
||||
<title>Tabeller</title>
|
||||
<p>Tabellen under er en enkelt tabellblokk.</p>
|
||||
<table-wrap id="tab-2-1">
|
||||
<label>Tabell 2-1</label>
|
||||
<table>
|
||||
<tbody>
|
||||
<tr><td>Prosess</td><td>Enhet</td></tr>
|
||||
<tr><td>Sprengning</td><td>kubikkmeter</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</table-wrap>
|
||||
<list list-type="alpha-lower">
|
||||
<list-item><p>Foerste punkt i lista.</p></list-item>
|
||||
<list-item><p>Andre punkt i lista.</p></list-item>
|
||||
</list>
|
||||
</sec>
|
||||
</body>
|
||||
</standard>
|
||||
14
tests/fixtures/xml-doctype-bomb.xml
vendored
Normal file
14
tests/fixtures/xml-doctype-bomb.xml
vendored
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
<?xml version="1.0"?>
|
||||
<!DOCTYPE lolz [
|
||||
<!ENTITY lol "SPRENGSTOFF">
|
||||
<!ENTITY lol2 "&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;">
|
||||
<!ENTITY lol3 "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;">
|
||||
<!ENTITY lol4 "&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;">
|
||||
]>
|
||||
<standard>
|
||||
<body>
|
||||
<sec id="s-1"><label>1</label><title>Uskyldig tittel</title>
|
||||
<p>&lol4;</p>
|
||||
</sec>
|
||||
</body>
|
||||
</standard>
|
||||
9
tests/fixtures/xml-malformed.xml
vendored
Normal file
9
tests/fixtures/xml-malformed.xml
vendored
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
<standard>
|
||||
<body>
|
||||
<sec id="s-1">
|
||||
<label>1</label>
|
||||
<title>Uavsluttet
|
||||
<p>Denne taggen lukkes aldri.</p>
|
||||
</sec>
|
||||
</body>
|
||||
</standard>
|
||||
Loading…
Add table
Add a link
Reference in a new issue