feat(description): an STS section's description is its own first spec point

K3-19 c. The NISO-STS reader records, per titled <sec>, the FIRST <p> of its
FIRST direct-child <sec sec-type="spec"> as `OutlineMark.description`. The
plan entry carries it (`description`, only where the source has one, so every
other row's plan keeps its bytes), `parse_segmentation_plan` refuses an empty,
multi-line or non-string value, and the door writes it as the concept's
`description` after the gate has seen it: it is document text persisted
outside the body the gate screens, so it is kept only on the non-blocking
floor and as the sanitized text.

SPEC SS 4.1 makes `description` RECOMMENDED and sets no length, in SS 4.1,
SS 8 or SS 11. The limit is ours and structural -- one paragraph, whole --
because a cut inside it writes a sentence the source never wrote. Measured on
R761: 2 026 of 2 761 titled sections carry a direct-child spec point; the
first <p> runs 17 / 109 / 273 / 521 / 942 characters (min / median / p90 /
p99 / max). A section with none gets no key; nothing is derived from the
title. A stated `--frontmatter description=...` replaces it.

The extracted text does not move: the description is read beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 03:22:44 +02:00
commit de7849e35b
6 changed files with 96 additions and 4 deletions

View file

@ -524,6 +524,30 @@ class _XmlTextExtractor:
"""An element's whole text, whitespace collapsed."""
return " ".join("".join(element.itertext()).split())
def _spec_point(self, section: Element) -> str | None:
"""The FIRST `<p>` of the FIRST direct-child `sec-type="spec"`, whole.
The limit is this package's, not the spec's: SPEC SS 4.1 asks for "a
single sentence" and sets no length anywhere. It is STRUCTURAL rather
than a character count, because a cut inside a paragraph writes a
sentence the source never wrote. Measured on the one STS document this
row has: 2 026 of 2 761 titled sections carry a direct-child spec
point; 264 of those points hold more than one `<p>` and 2 hold none;
the first `<p>` runs 17 / 109 / 273 / 521 / 942 characters at min /
median / p90 / p99 / max.
A DIRECT child only: a spec point belongs to the section it opens under,
and a container borrowing its first child's would describe a section by
a sentence about another one.
"""
for child in section:
if _local_name(child.tag) == "sec" and child.get("sec-type") == "spec":
for paragraph in child:
if _local_name(paragraph.tag) == "p":
return self._text_of(paragraph) or None
return None
return None
def _table(self, element: Element) -> bool:
"""A `<table-wrap>`: its label on a line, its rows as ONE table block.
@ -564,7 +588,12 @@ class _XmlTextExtractor:
heading = " ".join(part for part in parts if part)
self._emit("#" * level + " " + heading)
self.marks.append(
OutlineMark(line=len(self._lines) - 1, level=level, title=heading)
OutlineMark(
line=len(self._lines) - 1,
level=level,
title=heading,
description=self._spec_point(element),
)
)
skip = {id(title)} | ({id(label)} if label is not None else set())
elif label is not None:
@ -1036,6 +1065,10 @@ class OutlineMark:
line: int
level: int
title: str
#: The section's own first spec point, where the SOURCE declares one. Set
#: only by the NISO-STS reader; `None` on every bookmark mark, because a
#: bookmark declares a place and never a summary.
description: str | None = None
@dataclass(frozen=True)