feat(extract): the file that IS the product now reads, as a core stdlib type

GREEN on the twelve tests from the two commits before it. `.xml` joins
`_CORE_EXTRACTORS` beside `.html`, and 110 of 110 unreadable becomes a bundle.

A CORE type, not an `[extract]` one, because the parser is stdlib: putting it
behind the extra would make a pure-stdlib file type binary-dependent for no
gain. And never through the converter, which the existing fence test now says
in one more line: a file routed there is read by a second parser that never
sees this reader's DTD refusal, so the hardening would be true of the code and
false of the file.

TWO PATHS, NAMED RATHER THAN GUESSED. STS when the root is `<standard>` or the
document holds any `<sec>`; generic otherwise. Generic XML keeps its text in
document order and gets NO structure -- an element name is never promoted to a
heading, because an RSS feed is not a numbered standard.

THE GRAMMAR IS MARKDOWN, the same markdown the office rows and round 11's HTML
row reach the proposer through, and `propose.py` is untouched. `<sec>` with a
`<title>` becomes one ATX line carrying `<label>` + space + `<title>` at its
own nesting depth; `<sec>` with only a `<label>` becomes a body line with the
label in front, the way `li` is treated in HTML -- 4 954 of R761's 7 715 are
lettered points and one heading each would bury its own 2 761; `<table-wrap>`
becomes its label plus ONE markdown table through this package's own
`render_table`, separator line included, which is what makes it a block.

TWO CHOICES THAT ARE MEASUREMENTS, not preferences:

- Inline by ALLOWLIST, block by default -- the inverse of the HTML reader,
  because block-versus-inline is a property of HTML and XML has no such
  universal. Assuming block is safe (an extra break never removes text and a
  boundary needs a line that matches a grammar); assuming inline is not. The
  allowlist is load-bearing: that document carries 1 701 `<italic>` and
  1 396 `<bold>` inside its prose.
- The ATX ceiling is 6 and STS nesting reaches 7. The depth is CLIPPED, not
  dropped: 9 of the 2 761 titled sections sit at depth 7 and `#######` matches
  nothing, so dropping loses the section while clipping keeps the boundary and
  states the nesting one level too shallow.

A DTD IS REFUSED UNPARSED, and that is a guarantee about this code rather than
about the machine. Measured on this interpreter (3.14.0, pyexpat 2.7.3): an
external SYSTEM entity is refused by the stdlib and never fetched, but the
amplification limit that stops a billion-laughs comes from libexpat >= 2.4.0
and NOT from Python -- five levels still expand -- while `pyproject.toml`
requires only `>=3.10` and no lockfile pins an interpreter. `XMLParser` exposes
no `.parser` attribute on the C accelerator, so the handler route is not
portable either. NO new dependency: `defusedxml` and `lxml` both occur 0 times
in `uv.lock` and still do.

pytest -q: 1566 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 03:41:02 +02:00
commit 567a63d455
4 changed files with 276 additions and 4 deletions

View file

@ -46,7 +46,44 @@ one boundary rule:
`measured` since 2026-09-10, the row with the most measurement behind it and
no entry in the table at all -- with the limit
that travels with it -- one product, one format, one publisher, and a
generator's cut, not 828 documents anyone wrote. `pdf`/`docx`/`xlsx` only via
generator's cut, not 828 documents anyone wrote.
**`.xml` became a CORE type 2026-09-11** and it is the first row whose
ceiling is structural rather than recovered. A NISO-STS zip from a
publisher's own viewer was **110 of 110 unreadable, 0 plans, exit 2** --
`.xml` was in neither registry -- and the one xml file in it IS the whole
product: R761 Prosesskoden:2025, the document round 12 met as a 701-page
PDF, carrying 7 715 `<sec>`, **2 761 with a `<title>`**, 4 954 with a
`<label>` and no title, 10 `<table-wrap>`, root `<standard>`, 0 `<!DOCTYPE`.
Its `<sec>`-nesting depths over the titled sections are
28/118/500/1141/868/97/9, row for row the fasit's own. The reader is stdlib
(`xml.etree.ElementTree`) and adds NO dependency -- `defusedxml` and `lxml`
are 0 occurrences in `uv.lock` -- so it is core beside `.html` rather than
behind `[extract]`, which would make a pure-stdlib type binary-dependent.
**The output grammar is MARKDOWN**, the same the office rows reach the
proposer through, and `propose.py` is untouched: `<sec>` with a `<title>`
becomes one ATX line carrying `<label>` + space + `<title>` at its own
nesting depth, `<sec>` with only a `<label>` becomes a body line with the
label in front (never a heading -- 4 954 of 7 715 are lettered points and one
heading each buries the document's own 2 761), and `<table-wrap>` becomes its
label plus one markdown table through `render_table`. **`<label>` carries the
number and `<title>` carries the text** -- 2 of 2 761 titles begin with a
digit -- so emitting `<title>` alone scores 0 of 2 761 with nothing in the
code looking wrong. Inline is an ALLOWLIST and block is the default, the
inverse of the HTML reader, because block-versus-inline is a property of HTML
and XML has no such universal; the allowlist is load-bearing at 1 701
`<italic>` and 1 396 `<bold>` inside that document's prose. The ATX ceiling is
6 and STS nesting reaches 7, so the depth is CLIPPED and not dropped: 9 of
2 761 sit at depth 7 and `#######` matches nothing. A `<!DOCTYPE` is REFUSED
unparsed with its own code, a guarantee about the code rather than about the
machine -- measured on 3.14.0 with pyexpat 2.7.3, an external SYSTEM entity
is refused by the stdlib but the billion-laughs limit comes from libexpat
>= 2.4.0 and not from Python, while `pyproject.toml` requires only `>=3.10`.
XML that is not STS keeps its text in document order and gets NO invented
structure, and `.xml` never routes through the converter -- a second parser
that would never see that refusal. The registries are COUPLED: a row in
`_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
every proposal for the type, two layers away from the extractor.
`pdf`/`docx`/`xlsx` only via
the optional `[extract]` extra; without it those types are rejected
fail-fast. The extra ships `pdfplumber` for `pdf` (chosen on ONE measured
property: it keeps a requirement table's label and value on the same line