RED. Measured on R761 after the reader landed: non-whitespace preservation was
1 283 393 characters against the source's 1 283 395. Two characters, and a
percentage would have let them pass -- a ratio of 0.999998 reads like rounding.
Located by a prefix/suffix scan rather than a diff: the whole loss is one `x)`
marking a `<sec>` that carries a label and nothing else.
The mechanism is this reader's own. A label-only section holds its label as a
PREFIX for the body line beneath it, the way `li` is handled in the HTML
reader; when no body line follows -- the next thing emitted is the following
section's heading -- the pending prefix is overwritten by the next one and the
label never reaches the text.
`sts-empty-label.xml` is the case, hand-written and small: a lettered point
with a body, then one with none, then the next titled section.
pytest -q: 1 failed, 1566 passed, 1 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED, 12 failing. Two corrections to the previous commit's tests, both found by
running rather than by reading.
ONE, AND IT IS A REAL GAP THIS COMMIT CLOSES: `okf build` on an STS document
does not fail in the extractor at all -- it fails two layers away, in
`segmentation.observed_extractor_version`, with `no extractor version is known
for extractor_id 'xml'`. That coupling is deliberate and its own comment says
so: a row added to the extraction registry and not to `_STDLIB_EXTRACTOR_IDS`
"fails loudly on the first proposal for that type". No unit test of an
extractor can see it, because the missing thing is the version the plan is
KEYED to, not the text. The previous commit's tests would all have gone green
with the build still refusing every file. This test is the one that would not
have.
TWO: the expectation that the proposer strips a numbering token out of an ATX
title was wrong about `propose`, not about this reader. Measured: `_ATX` keeps
the whole title and fills `number` in addition, and `_NUMBERED` needs at least
one dot, so `1 Bruksomraade` gets no number while `1.1 Omfang` gets one and
keeps it. Both shapes are now asserted, along with the table-wrap arriving as
ONE `rule:table-block` candidate rather than one per row.
pytest -q: 12 failed, 1554 passed, 1 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED, 11 failing. Reproduced first, before any code: `okf build` on the folder
a publisher's own viewer delivers gives **110 of 110 unreadable, 0 plans, exit
2**, with `no extractor is registered for file extension '.xml'`. The
conservation identity `merged + coded rejections == N` is never written at all
-- the run aborts earlier on `FAILED - no segmentation plans` -- so the hole
was not even visible as a count.
The one xml file IS the whole product: R761 Prosesskoden:2025, the document
round 12 met as a 701-page PDF, in NISO-STS form. Measured on the file with
`xml.etree.ElementTree`: 7 715 `<sec>`, 2 761 with a `<title>`, 4 954 with a
`<label>` and no title, 10 `<table-wrap>`, 12 528 `<p>`, root `<standard>`, 0
`<!DOCTYPE` and 0 `<!ENTITY`. Its `<sec>`-nesting depths over the titled
sections are 28/118/500/1141/868/97/9 -- row for row the fasit's own
distribution. The ceiling is therefore structural rather than computed.
FOUR HAND-WRITTEN FIXTURES, none through `make_fixtures.py` and none
serialised by `ElementTree`: a library that writes and reads its own format
proves only that it agrees with itself. A known-positive STS mini, generic
non-STS xml, a `<!DOCTYPE` with an entity expansion, and a malformed file.
Two assertions that already existed are extended rather than duplicated: the
converter fence, because a file routed to the converter is read by a second
parser that never sees this reader's DTD refusal, and the evidence table,
because a row without a class is the failure that test exists for.
pytest -q: 11 failed, 1554 passed, 1 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
