open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

test(extract): a section below markdown's sixth level keeps its own level in the plan

Browse source 
K3-21 D, red. The NISO-STS reader clips `depth` to 6 once and writes that
one value into BOTH the ATX heading, where markdown has six levels, and the
`OutlineMark` the declared route builds its plan from, where nothing clips.
On one standard 9 of 2 761 titled sections sit at depth 7; the plan read them
at 6, and `--shell-parent` gave its two depth-7 shells the ancestor one level
too high.

`sts-deep.xml` (invented setting) reproduces it: the two depth-7 shells point
at the depth-5 section on e717b1c's code and must point at the depth-6 one.
Held: the mark carries depth 7; the heading stays `######` and the extracted
headings do not move one character; the shells point at their depth-6
ancestor.

2 of 3 red on 5970369; the green one is the text guard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 14:02:49 +02:00
commit 9822e5500c
2 changed files with 109 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

26
tests/fixtures/sts-deep.xml vendored Normal file
View file

@ -0,0 +1,26 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard>
<front>
<std-meta>
<title-wrap><main>Fikstur for dyp nesting</main></title-wrap>
</std-meta>
</front>
<body>
<sec sec-type="procedure" id="d1"><label>1</label><title>Nivaa en</title>
<sec sec-type="procedure" id="d11"><label>1.1</label><title>Nivaa to</title>
<sec sec-type="procedure" id="d111"><label>1.11</label><title>Nivaa tre</title>
<sec sec-type="procedure" id="d1111"><label>1.111</label><title>Nivaa fire</title>
<sec sec-type="procedure" id="d11111"><label>1.1111</label><title>Nivaa fem</title>
<sec sec-type="spec"><label>a)</label><p>Omfatter arbeid paa femte nivaa.</p></sec>
<sec sec-type="procedure" id="d111111"><label>1.11111</label><title>Nivaa seks</title>
<sec sec-type="spec"><label>a)</label><p>Omfatter arbeid paa sjette nivaa.</p></sec>
<sec sec-type="procedure" id="d1111111"><label>1.111111</label><title>Nivaa sju foerste</title></sec>
<sec sec-type="procedure" id="d1111112"><label>1.111112</label><title>Nivaa sju andre</title></sec>
</sec>
</sec>
</sec>
</sec>
</sec>
</sec>
</body>
</standard>

83
tests/test_depth_seven.py Normal file
View file

@ -0,0 +1,83 @@
"""A declared section below markdown's sixth level keeps its own level in the plan.
K3-21 D. The NISO-STS reader wrote ONE clipped level into two places: the ATX
heading it emits, where markdown has six levels and the clip is the grammar's,
and the `OutlineMark` the declared route builds its plan from, where nothing
clips and the source's own depth is the point (`OutlineMark`'s docstring:
the level is what the tree declares, "reported rather than fixed up"). On one
standard 9 of 2 761 titled sections sit at depth 7, and the plan read them at
6 -- so `--shell-parent` gave its two depth-7 shells the ancestor one level
too high.
The heading stays clipped: `#######` matches nothing, and the extracted TEXT
does not move one character. Only the mark's level does.
`sts-deep.xml` is hand-written in an invented setting and carries no sentence
from any source.
"""
from __future__ import annotations
from pathlib import Path
from llm_ingestion_okf import extract, propose
FIXTURE = Path(__file__).parent / "fixtures" / "sts-deep.xml"
#: Every flag `okf build` turns on by default.
BUILD_DEFAULTS = dict(
outline_run=3,
table_grid=True,
unit_fold=True,
keep_table_heading=True,
sheet_section_rows=True,
drop_wrapped_outline=True,
outline_gate=True,
first_span_from_zero=True,
close_span_gaps=True,
contents_name=True,
)
#: The extracted headings, as they were before this rule and after it.
HEADINGS = [
"# 1 Nivaa en",
"## 1.1 Nivaa to",
"### 1.11 Nivaa tre",
"#### 1.111 Nivaa fire",
"##### 1.1111 Nivaa fem",
"###### 1.11111 Nivaa seks",
"###### 1.111111 Nivaa sju foerste",
"###### 1.111112 Nivaa sju andre",
]
def test_the_mark_carries_the_declared_depth() -> None:
marks = extract.xml_outline(FIXTURE.name, FIXTURE.read_bytes())
assert [mark.level for mark in marks] == [1, 2, 3, 4, 5, 6, 7, 7]
def test_the_heading_stays_markdown_and_the_text_does_not_move() -> None:
data = FIXTURE.read_bytes()
text = extract.extract_text(FIXTURE.name, data)
lines = text.split("\n")
assert [line for line in lines if line.startswith("#")] == HEADINGS
for mark in extract.xml_outline(FIXTURE.name, data):
assert lines[mark.line] == "#" * min(mark.level, 6) + " " + mark.title
def test_a_depth_seven_shell_points_at_its_depth_six_ancestor() -> None:
data = FIXTURE.read_bytes()
text = extract.extract_text(FIXTURE.name, data)
plan = propose.build_plan(
FIXTURE,
text,
data,
okf_type="reference",
proposed_at="2026-01-01T00:00:00Z",
shell_parent=True,
**BUILD_DEFAULTS, # type: ignore[arg-type]
)
titles = {entry["segment_id"]: entry["title"] for entry in plan["entries"]}
parents = {entry["title"]: titles.get(entry.get("parent_id", "")) for entry in plan["entries"]}
assert parents["Nivaa sju foerste"] == "Nivaa seks"
assert parents["Nivaa sju andre"] == "Nivaa seks"