feat(extract): the file that IS the product now reads, as a core stdlib type
GREEN on the twelve tests from the two commits before it. `.xml` joins `_CORE_EXTRACTORS` beside `.html`, and 110 of 110 unreadable becomes a bundle. A CORE type, not an `[extract]` one, because the parser is stdlib: putting it behind the extra would make a pure-stdlib file type binary-dependent for no gain. And never through the converter, which the existing fence test now says in one more line: a file routed there is read by a second parser that never sees this reader's DTD refusal, so the hardening would be true of the code and false of the file. TWO PATHS, NAMED RATHER THAN GUESSED. STS when the root is `<standard>` or the document holds any `<sec>`; generic otherwise. Generic XML keeps its text in document order and gets NO structure -- an element name is never promoted to a heading, because an RSS feed is not a numbered standard. THE GRAMMAR IS MARKDOWN, the same markdown the office rows and round 11's HTML row reach the proposer through, and `propose.py` is untouched. `<sec>` with a `<title>` becomes one ATX line carrying `<label>` + space + `<title>` at its own nesting depth; `<sec>` with only a `<label>` becomes a body line with the label in front, the way `li` is treated in HTML -- 4 954 of R761's 7 715 are lettered points and one heading each would bury its own 2 761; `<table-wrap>` becomes its label plus ONE markdown table through this package's own `render_table`, separator line included, which is what makes it a block. TWO CHOICES THAT ARE MEASUREMENTS, not preferences: - Inline by ALLOWLIST, block by default -- the inverse of the HTML reader, because block-versus-inline is a property of HTML and XML has no such universal. Assuming block is safe (an extra break never removes text and a boundary needs a line that matches a grammar); assuming inline is not. The allowlist is load-bearing: that document carries 1 701 `<italic>` and 1 396 `<bold>` inside its prose. - The ATX ceiling is 6 and STS nesting reaches 7. The depth is CLIPPED, not dropped: 9 of the 2 761 titled sections sit at depth 7 and `#######` matches nothing, so dropping loses the section while clipping keeps the boundary and states the nesting one level too shallow. A DTD IS REFUSED UNPARSED, and that is a guarantee about this code rather than about the machine. Measured on this interpreter (3.14.0, pyexpat 2.7.3): an external SYSTEM entity is refused by the stdlib and never fetched, but the amplification limit that stops a billion-laughs comes from libexpat >= 2.4.0 and NOT from Python -- five levels still expand -- while `pyproject.toml` requires only `>=3.10` and no lockfile pins an interpreter. `XMLParser` exposes no `.parser` attribute on the C accelerator, so the handler route is not portable either. NO new dependency: `defusedxml` and `lxml` both occur 0 times in `uv.lock` and still do. pytest -q: 1566 passed, 1 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
5bb3b045c2
commit
567a63d455
4 changed files with 276 additions and 4 deletions
39
CLAUDE.md
39
CLAUDE.md
|
|
@ -46,7 +46,44 @@ one boundary rule:
|
|||
`measured` since 2026-09-10, the row with the most measurement behind it and
|
||||
no entry in the table at all -- with the limit
|
||||
that travels with it -- one product, one format, one publisher, and a
|
||||
generator's cut, not 828 documents anyone wrote. `pdf`/`docx`/`xlsx` only via
|
||||
generator's cut, not 828 documents anyone wrote.
|
||||
**`.xml` became a CORE type 2026-09-11** and it is the first row whose
|
||||
ceiling is structural rather than recovered. A NISO-STS zip from a
|
||||
publisher's own viewer was **110 of 110 unreadable, 0 plans, exit 2** --
|
||||
`.xml` was in neither registry -- and the one xml file in it IS the whole
|
||||
product: R761 Prosesskoden:2025, the document round 12 met as a 701-page
|
||||
PDF, carrying 7 715 `<sec>`, **2 761 with a `<title>`**, 4 954 with a
|
||||
`<label>` and no title, 10 `<table-wrap>`, root `<standard>`, 0 `<!DOCTYPE`.
|
||||
Its `<sec>`-nesting depths over the titled sections are
|
||||
28/118/500/1141/868/97/9, row for row the fasit's own. The reader is stdlib
|
||||
(`xml.etree.ElementTree`) and adds NO dependency -- `defusedxml` and `lxml`
|
||||
are 0 occurrences in `uv.lock` -- so it is core beside `.html` rather than
|
||||
behind `[extract]`, which would make a pure-stdlib type binary-dependent.
|
||||
**The output grammar is MARKDOWN**, the same the office rows reach the
|
||||
proposer through, and `propose.py` is untouched: `<sec>` with a `<title>`
|
||||
becomes one ATX line carrying `<label>` + space + `<title>` at its own
|
||||
nesting depth, `<sec>` with only a `<label>` becomes a body line with the
|
||||
label in front (never a heading -- 4 954 of 7 715 are lettered points and one
|
||||
heading each buries the document's own 2 761), and `<table-wrap>` becomes its
|
||||
label plus one markdown table through `render_table`. **`<label>` carries the
|
||||
number and `<title>` carries the text** -- 2 of 2 761 titles begin with a
|
||||
digit -- so emitting `<title>` alone scores 0 of 2 761 with nothing in the
|
||||
code looking wrong. Inline is an ALLOWLIST and block is the default, the
|
||||
inverse of the HTML reader, because block-versus-inline is a property of HTML
|
||||
and XML has no such universal; the allowlist is load-bearing at 1 701
|
||||
`<italic>` and 1 396 `<bold>` inside that document's prose. The ATX ceiling is
|
||||
6 and STS nesting reaches 7, so the depth is CLIPPED and not dropped: 9 of
|
||||
2 761 sit at depth 7 and `#######` matches nothing. A `<!DOCTYPE` is REFUSED
|
||||
unparsed with its own code, a guarantee about the code rather than about the
|
||||
machine -- measured on 3.14.0 with pyexpat 2.7.3, an external SYSTEM entity
|
||||
is refused by the stdlib but the billion-laughs limit comes from libexpat
|
||||
>= 2.4.0 and not from Python, while `pyproject.toml` requires only `>=3.10`.
|
||||
XML that is not STS keeps its text in document order and gets NO invented
|
||||
structure, and `.xml` never routes through the converter -- a second parser
|
||||
that would never see that refusal. The registries are COUPLED: a row in
|
||||
`_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
|
||||
every proposal for the type, two layers away from the extractor.
|
||||
`pdf`/`docx`/`xlsx` only via
|
||||
the optional `[extract]` extra; without it those types are rejected
|
||||
fail-fast. The extra ships `pdfplumber` for `pdf` (chosen on ONE measured
|
||||
property: it keeps a requirement table's label and value on the same line
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue