open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(description): an STS section's description is its own first spec point

Browse source 
K3-19 c. The NISO-STS reader records, per titled <sec>, the FIRST <p> of its
FIRST direct-child <sec sec-type="spec"> as `OutlineMark.description`. The
plan entry carries it (`description`, only where the source has one, so every
other row's plan keeps its bytes), `parse_segmentation_plan` refuses an empty,
multi-line or non-string value, and the door writes it as the concept's
`description` after the gate has seen it: it is document text persisted
outside the body the gate screens, so it is kept only on the non-blocking
floor and as the sanitized text.

SPEC SS 4.1 makes `description` RECOMMENDED and sets no length, in SS 4.1,
SS 8 or SS 11. The limit is ours and structural -- one paragraph, whole --
because a cut inside it writes a sentence the source never wrote. Measured on
R761: 2 026 of 2 761 titled sections carry a direct-child spec point; the
first <p> runs 17 / 109 / 273 / 521 / 942 characters (min / median / p90 /
p99 / max). A section with none gets no key; nothing is derived from the
title. A stated `--frontmatter description=...` replaces it.

The extracted text does not move: the description is read beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 03:22:44 +02:00
commit de7849e35b
6 changed files with 96 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

6
CLAUDE.md
View file

@ -130,7 +130,11 @@ one boundary rule:
one", which a name read from inside a document cannot obey. The `sources`
title is `<doc-number>` + `<year>`, then `<title-wrap>`, then the file name:
R761's `<full>` carries a COMMA, a flow terminator, so it is never written
and never cleaned up. The registries are COUPLED: a row in
and never cleaned up. A titled section's `description` is its own FIRST
spec point (first `<p>` of the first DIRECT-child `sec-type="spec"`, whole),
carried by the plan entry and screened by the gate: 2 026 of 2 761 on R761,
none invented for the rest. SS 4.1 sets no length, so the one-paragraph
limit is ours. The registries are COUPLED: a row in
`_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
every proposal for the type, two layers away from the extractor.
`pdf`/`docx`/`xlsx` only via