feat(description): an STS section's description is its own first spec point

K3-19 c. The NISO-STS reader records, per titled <sec>, the FIRST <p> of its
FIRST direct-child <sec sec-type="spec"> as `OutlineMark.description`. The
plan entry carries it (`description`, only where the source has one, so every
other row's plan keeps its bytes), `parse_segmentation_plan` refuses an empty,
multi-line or non-string value, and the door writes it as the concept's
`description` after the gate has seen it: it is document text persisted
outside the body the gate screens, so it is kept only on the non-blocking
floor and as the sanitized text.

SPEC SS 4.1 makes `description` RECOMMENDED and sets no length, in SS 4.1,
SS 8 or SS 11. The limit is ours and structural -- one paragraph, whole --
because a cut inside it writes a sentence the source never wrote. Measured on
R761: 2 026 of 2 761 titled sections carry a direct-child spec point; the
first <p> runs 17 / 109 / 273 / 521 / 942 characters (min / median / p90 /
p99 / max). A section with none gets no key; nothing is derived from the
title. A stated `--frontmatter description=...` replaces it.

The extracted text does not move: the description is read beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 03:22:44 +02:00
commit de7849e35b
6 changed files with 96 additions and 4 deletions

View file

@ -1514,6 +1514,22 @@ def build_plan(
outline_marks=outline_marks,
outline_rule=outline_rule,
)
# A mark's description travels to the candidate whose span holds the mark's
# line -- the FIRST such mark, so a span that opens at zero (the front
# matter above the first section) still takes its own section's and no
# other. Only the NISO-STS reader sets one, so every other row's plan keeps
# its bytes: an absent key is the source saying nothing.
starts: list[int] = []
notes: list[str | None] = []
if outline_marks:
offsets = [0]
for line in text.splitlines(keepends=True):
offsets.append(offsets[-1] + len(line))
for mark in sorted(outline_marks, key=lambda item: item.line):
if mark.line < len(offsets) - 1:
starts.append(offsets[mark.line])
notes.append(mark.description)
cursor = 0
for candidate in subdivide(text, candidates, max_segment_chars):
entries.append(
{
@ -1550,6 +1566,12 @@ def build_plan(
"derived": _derived_names(candidate),
}
)
while cursor < len(starts) and starts[cursor] < candidate.start:
cursor += 1
if cursor < len(starts) and starts[cursor] < candidate.end:
description = notes[cursor]
if description is not None:
entries[-1]["description"] = description
return {
"version": "1",
"source_sha256": hashlib.sha256(source_bytes).hexdigest(),