open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(propose,cli): the sheet's rows, the wrapped sentence, and a default that moved

Browse source 
K3 round 3, order 20260908T170037Z-3622420612-from-.claude, plus the PM
injection of 17:55Z carrying the operator's D3 answer.

TWO NEW RULES, both OFF by default.

`--sheet-section-rows` cuts an open table block at the rows that label its
sections: a RUN of at least three rows whose first cell is a bare numeric
label. It is the first rule here that opens a span INSIDE a table, and the
opposite direction from Arm E -- that arm decides how far a block extends,
this decides where it is cut inside. The spreadsheet is the one file type no
arm had ever moved: 1 concept -> 12 on the priced sheet, of which 11 are the
reference's 11 cost groups. The run guard buys exactly three candidates on the
corpus (the rows stating a computation basis), and each section carries its
own `source_sheet` + `source_rows`, verified on the artifact.

`--drop-wrapped-outline` declines an outline candidate whose line continues
onto the next one. Round 2 falsified the LENGTH axis on this case; this one is
not about size. Over the 12-document sample it separates 8 of 34 candidates --
the quoted regulation paragraphs and the risk-table rows -- and none of the 26
the operator kept. Position 4: 4 concepts -> 1, the reference. The cell is
`treffer` by count and lossy by bytes, and the report says so.

Whole corpus, one cached extraction: 1 of 39 documents changes under the first
rule (0 of 32 pdf, 0 of 5 docx), 5 of 39 under the second (all pdf). Arm B and
F2 are byte-identical against a frozen c3b645b, both halves by `diff -r`.

THE `okf build` DEFAULT MOVED, and this is the operator's decision executed,
not a side effect: no flag now means `--outline-run 3 --unit-fold`. Each arm
keeps an explicit opt-out (`--outline-run 0`, `--no-unit-fold`) and the pair
reproduces the pre-move bytes exactly. The proposer's own defaults do NOT
move: they are what the goldens and every published reproduction block are
pinned to, so the two layers now disagree on purpose and `cli.py` says where.
Two shipped expectations moved with it and both are stated in place. MEASURED
AND REPORTED BACK: the configuration made default scores 2 of 12 on the unit
worksheet, `docx` 0 of 3 -- Arm F's published 5 of 12 was measured with
`--table-grid` ON, and without it the fold has no joined table to fold.

Position 1 is declined for the second round on the axis the order named: 3 of
3 recovered chapters have body under them, so "heading-on-heading" separates 0
of 3.

The K2 ranking control round 2 could not measure: two bundles from frozen
c3b645b differing only in `--keep-table-heading`, 2 of 1108 files apart. The
priced concept ranks 96 of 629 in BOTH and is delivered at rank 10 in both, so
the flag buys 35 payload bytes and zero rank positions. Round 2's prediction
is falsified: the concept's NAME was already restored by the orphan-name
inheritance; the flag adds the heading LINE.

The stale S7 literal is restated where it stood (`c759a657...`, 171 614 B,
re-measured on this HEAD), so the next order cannot inherit the superseded one.

1397 -> 1414 tests. Report: docs/2026-09-08-k3-runde3-per-filtype.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 20:33:52 +02:00
commit 47f6619e6f
11 changed files with 1381 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

293
tests/test_cli_build.py
View file

@ -29,6 +29,7 @@ from __future__ import annotations
import shutil
import subprocess
import sys
from collections.abc import Sequence
from dataclasses import replace
from pathlib import Path
@ -82,12 +83,16 @@ def tree(root: Path) -> dict[str, bytes]:
}
def two_script_bundle(inbox: Path, out: Path) -> Path:
def two_script_bundle(inbox: Path, out: Path, proposer_flags: Sequence[str] = ()) -> Path:
"""The path this command replaces, run as it is documented, as subprocesses.
Re-implementing the loop here would compare the CLI against a copy of
itself. Driving the actual scripts is what makes the byte comparison mean
"the old path and the new path agree".
`proposer_flags` is passed through verbatim. It exists because the BUILD
default moved and the proposer's did not: the two paths still agree, and
saying which flags make them agree is the honest form of that claim.
"""
plans = out / "plans"
plans.mkdir(parents=True, exist_ok=True)
@ -109,6 +114,7 @@ def two_script_bundle(inbox: Path, out: Path) -> Path:
relative.with_suffix("").as_posix(),
"--proposed-at",
PROPOSED_AT,
*proposer_flags,
],
capture_output=True,
check=True,
@ -545,11 +551,18 @@ def test_the_arms_reach_the_proposer_from_the_build_command(tmp_path: Path) -> N
def test_the_build_default_is_unchanged_by_the_flags_existing(tmp_path: Path) -> None:
"""The control the test above rests on: same inbox, no flags, Arm B.
"""The control the test above rests on: same inbox, no flags, the DEFAULT.
Both halves over the same fixture. Without this, a change that turned an
arm ON by default would leave the assertion above green and move every
bundle every consumer builds.
**This expectation MOVED on 2026-09-08 and the move is stated rather than
quietly rewritten.** It was written in round 2 to pin the flagless build to
Arm B, so that a change turning an arm on by default could not hide behind
the assertion above. The operator then turned two arms on, deliberately,
and the titles below are Arm D plus Arm F on this fixture.
What the test still measures is unchanged and is the reason it stays: the
flagless build has ONE declared behaviour and a drifting default moves it.
`test_each_arm_in_the_default_has_an_explicit_opt_out` holds the other end,
where Arm B's titles are still asserted, from the opt-out.
"""
inbox = inbox_for_arms(tmp_path)
plans = tmp_path / "plans-plain"
@ -564,6 +577,236 @@ def test_the_build_default_is_unchanged_by_the_flags_existing(tmp_path: Path) ->
)
== 0
)
# Arm F folds `4.1 Loesmasser` into `4 Grunnforhold`; the sheet heading is
# still carried by the table block, because `--keep-table-heading` did NOT
# move and is still a flag.
assert plan_titles(plans) == ["4 Grunnforhold", "5 Prissammenstilling"]
# A sheet whose units are ROWS: one heading, one continuous table block, a
# preamble and a run of three numbered rows. Kept apart from ARM_DOCUMENTS so
# the round-2 assertions above keep measuring what they were written for --
# `epsilon.md`'s table has ONE numbered row, which is a quantity and not a run.
SHEET_DOCUMENT = {
"zeta.md": (
"## 6 Kostnadsoversikt\n\n"
"| Skjema | | |\n|----|----|----|\n"
"| Skjemaet fylles ut i sin helhet. | | |\n"
"| 01 | Felleskostnader | |\n"
"| 02 | Bygning | |\n"
"| 07 | Utendoers | |\n"
)
}
def inbox_for_sheet_sections(root: Path) -> Path:
inbox = root / "sheet-inbox"
inbox.mkdir(parents=True, exist_ok=True)
for name, body in SHEET_DOCUMENT.items():
(inbox / name).write_text(body, encoding="utf-8", newline="")
return inbox
def test_sheet_section_rows_reaches_the_proposer_from_the_build_command(tmp_path: Path) -> None:
"""The red test for round 3: D3 exists here and it changes the plan.
Asserted on the plan's TITLES, for the reason round 2's is: a rule that
produced the right number of concepts by cutting in the wrong places would
pass a count assertion. Each title carries the row's own label, which is
what makes the concept findable by the number a reader is holding.
"""
inbox = inbox_for_sheet_sections(tmp_path)
plans = tmp_path / "plans-sheet"
assert (
build(
inbox,
tmp_path / "bundle-sheet",
"--plans-dir",
str(plans),
"--proposed-at",
PROPOSED_AT,
"--sheet-section-rows",
)
== 0
)
assert plan_titles(plans) == [
"6 Kostnadsoversikt",
"01 Felleskostnader",
"02 Bygning",
"07 Utendoers",
]
def test_the_build_default_leaves_a_sheet_as_one_concept(tmp_path: Path) -> None:
"""The control the test above rests on: same inbox, no flag, one concept.
Without it, a change that turned D3 on by default would leave the assertion
above green while cutting every table a consumer has ever ingested.
"""
inbox = inbox_for_sheet_sections(tmp_path)
plans = tmp_path / "plans-sheet-plain"
assert (
build(
inbox,
tmp_path / "bundle-sheet-plain",
"--plans-dir",
str(plans),
"--proposed-at",
PROPOSED_AT,
)
== 0
)
assert plan_titles(plans) == ["6 Kostnadsoversikt"]
# A document whose numbering is RECOVERED and whose first numbered line is a
# wrapped sentence: the shape of quoted regulation text.
WRAPPED_DOCUMENT = {
"eta.md": (
"Innledning uten nummer.\n\n"
"1 Krav og kriterier etter denne bestemmelsen skal ha som maal aa\n"
"redusere anskaffelsens samlede klimaavtrykk.\n\n"
"2 Ordinaert kapittel\n\nDette kapittelet har sin egen kropp.\n\n"
"3 Tredje kapittel\n\nOgsaa dette er en kropp.\n"
)
}
def inbox_for_wrapped_outline(root: Path) -> Path:
inbox = root / "wrapped-inbox"
inbox.mkdir(parents=True, exist_ok=True)
for name, body in WRAPPED_DOCUMENT.items():
(inbox / name).write_text(body, encoding="utf-8", newline="")
return inbox
def test_drop_wrapped_outline_reaches_the_proposer_from_the_build_command(
tmp_path: Path,
) -> None:
"""The second red test for round 3, and its control is the pair below.
The build path had four flags and the proposer now has six. A rule that
only the proposer can run is a rule no bundle is ever built with, which is
the defect round 2 fixed for the arms.
"""
inbox = inbox_for_wrapped_outline(tmp_path)
plans = tmp_path / "plans-wrapped"
assert (
build(
inbox,
tmp_path / "bundle-wrapped",
"--plans-dir",
str(plans),
"--proposed-at",
PROPOSED_AT,
"--outline-run",
"3",
"--drop-wrapped-outline",
)
== 0
)
assert plan_titles(plans) == ["Ordinaert kapittel", "Tredje kapittel"]
def test_the_build_default_keeps_a_wrapped_outline_candidate(tmp_path: Path) -> None:
"""The control: Arm D alone recovers all three, including the sentence."""
inbox = inbox_for_wrapped_outline(tmp_path)
plans = tmp_path / "plans-wrapped-plain"
assert (
build(
inbox,
tmp_path / "bundle-wrapped-plain",
"--plans-dir",
str(plans),
"--proposed-at",
PROPOSED_AT,
"--outline-run",
"3",
)
== 0
)
assert plan_titles(plans) == [
"Krav og kriterier etter denne bestemmelsen skal ha som maal aa",
"Ordinaert kapittel",
"Tredje kapittel",
]
# --- the default moved (operator, 2026-09-08) ------------------------------
#
# Round 3. The operator answered the standing question with alternative (b):
# `okf build` with no flag is now Arm D plus Arm F, `--outline-run 3
# --unit-fold`. Two arms and not four -- `--table-grid` and
# `--keep-table-heading` stay flags until the K2 ranking control is measured.
#
# Every arm keeps an EXPLICIT opt-out, because a default nobody can turn off is
# not a default, it is a behaviour: `--outline-run 0` for Arm D (the number was
# always its own switch) and `--no-unit-fold` for Arm F.
def test_the_build_default_is_now_arm_d_plus_arm_f(tmp_path: Path) -> None:
"""The red test for the move: no flags must EQUAL the two flags, byte for byte.
Asserted on the bundle rather than on the parser, and byte for byte rather
than on a count: a default that reached the proposer as a different number
would still produce a plausible plan, and only the bytes can tell.
"""
inbox = inbox_for_arms(tmp_path)
default = tmp_path / "default-bundle"
explicit = tmp_path / "explicit-bundle"
assert build(inbox, default, "--proposed-at", PROPOSED_AT) == 0
assert (
build(
inbox,
explicit,
"--proposed-at",
PROPOSED_AT,
"--outline-run",
"3",
"--unit-fold",
)
== 0
)
assert tree(default) == tree(explicit)
def test_each_arm_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None:
"""The other half: a caller can still get the pre-move behaviour, and say so.
`--outline-run 0 --no-unit-fold` must reproduce Arm B exactly. Without
this, the move would be irreversible for every consumer who needs the old
bytes, and "off by default" would have become "unreachable".
"""
inbox = inbox_for_arms(tmp_path)
opted_out = tmp_path / "opt-out-bundle"
assert (
build(
inbox,
opted_out,
"--proposed-at",
PROPOSED_AT,
"--outline-run",
"0",
"--no-unit-fold",
)
== 0
)
plans = tmp_path / "plans-opt-out"
assert (
build(
inbox,
tmp_path / "opt-out-plans-bundle",
"--plans-dir",
str(plans),
"--proposed-at",
PROPOSED_AT,
"--outline-run",
"0",
"--no-unit-fold",
)
== 0
)
# Arm B's own titles, the ones round 2 pinned for the flagless build.
assert plan_titles(plans) == ["4 Grunnforhold", "4.1 Loesmasser", "5 Prissammenstilling"]
@ -572,12 +815,42 @@ def test_a_bundle_built_with_no_flags_is_byte_identical_to_the_shipped_one(
) -> None:
"""The byte control, on the tree the other build tests use.
The proposer gained a parameter and the CLI gained four. A default that
moved by a byte would break rebuild-equals-incremental for every consumer
who never passes a flag, and a title assertion cannot see that.
**This expectation MOVED on 2026-09-08 and the move is stated rather than
quietly rewritten.** Until then `okf build` with no flag was Arm B, so it
equalled the two-script path run with no flag either. The operator moved
the default to Arm D plus Arm F, so the equivalence now has to name the
arms on one side or the other, and both halves are asserted:
- the two-script path with the two arms equals the new default;
- the two-script path with no arm equals `--outline-run 0 --no-unit-fold`,
which is what makes the move reversible for a consumer who needs the old
bytes.
A single half would leave the other unmeasured, and the second is the one
that goes red if an opt-out stops opting out.
"""
inbox = inbox_with_subdirectories(tmp_path)
reference = two_script_bundle(inbox, tmp_path / "reference-2")
armed = two_script_bundle(
inbox, tmp_path / "reference-armed", proposer_flags=("--outline-run", "3", "--unit-fold")
)
bundle = tmp_path / "cli-bundle-2"
assert build(inbox, bundle, "--ingested-at", INGESTED_AT, "--proposed-at", PROPOSED_AT) == 0
assert tree(bundle) == tree(reference)
assert tree(bundle) == tree(armed)
plain = two_script_bundle(inbox, tmp_path / "reference-2")
opted_out = tmp_path / "cli-bundle-opt-out"
assert (
build(
inbox,
opted_out,
"--ingested-at",
INGESTED_AT,
"--proposed-at",
PROPOSED_AT,
"--outline-run",
"0",
"--no-unit-fold",
)
== 0
)
assert tree(opted_out) == tree(plain)

94
tests/test_propose_segments.py
View file

@ -1061,7 +1061,13 @@ def test_a_non_integer_outline_run_is_refused_by_argparse(
#: Arm C is False on purpose -- its help says "not defined in the K3 method
#: file" instead, and the point of the check is that each arm's attribution
#: sits in its OWN chunk.
ARM_ATTRIBUTION = {"outline-run": True, "table-grid": True, "max-segment-chars": False}
ARM_ATTRIBUTION = {
"outline-run": True,
"table-grid": True,
"sheet-section-rows": True,
"drop-wrapped-outline": True,
"max-segment-chars": False,
}
def test_each_arm_flag_carries_its_attribution_inside_its_own_option_chunk(
@ -1100,6 +1106,92 @@ def test_each_arm_flag_carries_its_attribution_inside_its_own_option_chunk(
assert squeezed.count("not defined upstream") == expected
# --- D3 round 3: a recovered heading that is really a wrapped sentence -----
#
# K3 position 4 quotes a regulation. Its numbered paragraphs (`1)`, `3)`, `4)`,
# `5)`) match Arm D's grammar exactly, so the arm recovers four "chapters" from
# text the operator wrote off in one line: "sitert forskriftstekst er ikke
# overskrifter". Round 2 tried to separate them by TITLE LENGTH and falsified
# it: a real chapter heading in the same sample is 88 characters and the
# quoted sentences are 86, 91, 92 and 100.
#
# The axis below is a different one and it is not about size. A heading is a
# complete line; a wrapped sentence continues on the next one. Measured over
# the 12-document sample: 8 of 34 outline candidates are followed by a line
# beginning in lower case -- the four quoted paragraphs at position 4 and four
# table rows at position 9, where the operator wrote "radene er ikke
# overskrifter". 0 of the 26 the operator kept.
#
# It judges RECOVERED candidates only. A dotted or ATX heading is the
# document's own claim about itself, and overruling that is a different
# decision from declining to invent one.
WRAPPED_OUTLINE = """Innledning til dokumentet uten nummer.
1 Krav og kriterier etter denne bestemmelsen skal ha som maal aa redusere
anskaffelsens samlede klimaavtrykk.
2 Ordinaert kapittel
Dette kapittelet har en kropp som begynner med stor forbokstav.
3 Tredje kapittel
Ogsaa dette er en kropp.
"""
DECLARED_HEADING_THAT_WRAPS = """## 2.1 Et erklaert kapittel
som fortsetter paa neste linje i samme setning.
## 2.2 Et annet kapittel
Kropp under det andre kapittelet.
"""
def test_a_wrapped_outline_candidate_is_not_a_heading() -> None:
"""The red test: the recovered candidate whose line wraps is dropped.
Only that one. The two whose next line opens a new sentence stay, which is
what keeps the rule from being "discard Arm D" wearing a flag.
"""
off = okf_propose_segments.find_candidates(WRAPPED_OUTLINE, outline_run=3)
assert [c.title for c in off] == [
"Krav og kriterier etter denne bestemmelsen skal ha som maal aa redusere",
"Ordinaert kapittel",
"Tredje kapittel",
]
on = okf_propose_segments.find_candidates(
WRAPPED_OUTLINE, outline_run=3, drop_wrapped_outline=True
)
assert [c.title for c in on] == ["Ordinaert kapittel", "Tredje kapittel"]
def test_a_declared_heading_that_wraps_is_left_alone() -> None:
"""The known-negative, and it is the rule's whole scope statement.
A document that numbers its own sections has declared them. The rule
exists because Arm D RECOVERS a numbering the document never declared, so
a recovered candidate that turns out to be a sentence was never a heading;
a declared one is a claim this tool does not get to overrule.
"""
off = okf_propose_segments.find_candidates(DECLARED_HEADING_THAT_WRAPS)
on = okf_propose_segments.find_candidates(
DECLARED_HEADING_THAT_WRAPS, drop_wrapped_outline=True
)
assert [c.title for c in off] == ["2.1 Et erklaert kapittel", "2.2 Et annet kapittel"]
assert off == on
def test_the_wrapped_rule_is_off_by_default() -> None:
"""The default control: same text, no flag, Arm D's own output."""
default = okf_propose_segments.find_candidates(WRAPPED_OUTLINE, outline_run=3)
explicit = okf_propose_segments.find_candidates(
WRAPPED_OUTLINE, outline_run=3, drop_wrapped_outline=False
)
assert default == explicit
assert len(default) == 3
# --- Arm E: the grid-rule join --------------------------------------------
#
# Arm E is not defined in `docs/2026-09-02-k3-k4-k5-metode.md` -- that file

194
tests/test_propose_sheet_sections.py Normal file
View file

@ -0,0 +1,194 @@
"""D3 round 3: section rows INSIDE a sheet's one table block, behind a flag.
The spreadsheet is the one file type no arm has ever moved. Its whole body
extracts as ONE continuous pipe-table block -- one heading and 101 rows -- so
every rule this module has proposes exactly one boundary for it, and the
operator's worksheet asks for one concept per numbered cost group. Those groups
are ROWS, not headings, which is why no heading rule can reach them and why
`--keep-table-heading` moved the first byte without moving the count.
The rule tested here is the opposite direction from `--table-grid`: that one
stops a converter's rule line from CLOSING a block, so one grid table proposes
one candidate instead of many; this one CUTS an open block at the rows that
label its sections. They read different grammars (a `+---+` rule line against a
pipe row's first cell) and compose in one order -- grid decides how far a block
extends, section rows decide where it is cut inside.
Generality is the whole point, so the fixtures below carry no word list and no
knowledge of any real document's labels: what makes a row a section row is that
its first cell is a bare numeric label and that it is one of a RUN of such rows.
The run is the same bounding device `CONTENTS_RUN` already uses in this module,
for the same reason: a single numbered row is a quantity, not a section.
"""
from __future__ import annotations
from pathlib import Path
import pytest
from llm_ingestion_okf import propose as okf_propose_segments
# One heading, one continuous table block, a preamble, a run of four numbered
# section rows, and a summing tail. The shape of a priced sheet, written from
# the grammar rather than copied from one.
SECTIONED_SHEET = """## Kostnadsoversikt {#sheet-1}
| Skjema | | |
|----|----|----|
| Skjemaet fylles ut i sin helhet. | | |
| Post | | SUM |
| 01 | Felleskostnader | |
| 11+12 | Rigging og drift | |
| 02 | Bygning | |
| 07 | Utendoers | |
| Tilbudt fastpris eksklusive avgift | | 100 |
"""
# The same block with the numbered run removed. Nothing here is a section row,
# so the flag must leave it exactly as it found it.
UNSECTIONED_SHEET = """## Kostnadsoversikt {#sheet-1}
| Skjema | | |
|----|----|----|
| Skjemaet fylles ut i sin helhet. | | |
| Post | | SUM |
| Felleskostnader | | 10 |
| Rigging og drift | | 20 |
| Tilbudt fastpris eksklusive avgift | | 100 |
"""
# A single row whose first cell is a number -- a computation basis, not a
# section. This is the known-negative the run threshold exists for, and it is
# the shape that actually occurs: a sheet states a quantity on its own row.
LONE_NUMERIC_ROW = """## Regningsarbeider {#sheet-2}
| Grunnlag | | |
|----|----|----|
| Beregningsgrunnlag massehaandtering | tonn | 16000 |
| Paaslag i prosent | | 10 |
"""
def test_a_run_of_numbered_rows_cuts_the_block_into_sections() -> None:
"""The rule, red first: one candidate per section row, named from the row.
Default: the whole sheet is ONE table block whose title is inherited from
the orphaned heading. On: the preamble keeps that block and each numbered
row opens its own candidate, running to the next section row or to the end
of the block -- so the LAST section carries the sheet's tail, which is the
honest consequence of a rule that cuts and never discards.
"""
off = okf_propose_segments.find_candidates(SECTIONED_SHEET)
assert [(c.title, c.rule) for c in off] == [
("Kostnadsoversikt {#sheet-1}", okf_propose_segments.RULE_TABLE_BLOCK)
]
on = okf_propose_segments.find_candidates(SECTIONED_SHEET, sheet_section_rows=True)
assert [(c.number, c.title, c.rule) for c in on] == [
(None, "Kostnadsoversikt {#sheet-1}", okf_propose_segments.RULE_TABLE_BLOCK),
("01", "01 Felleskostnader", okf_propose_segments.RULE_SHEET_SECTION),
("11+12", "11+12 Rigging og drift", okf_propose_segments.RULE_SHEET_SECTION),
("02", "02 Bygning", okf_propose_segments.RULE_SHEET_SECTION),
("07", "07 Utendoers", okf_propose_segments.RULE_SHEET_SECTION),
]
# The preamble block ends where the first section opens, and the last
# section reaches the end of the text.
assert on[0].end == on[1].start
assert on[1].end == on[2].start
assert on[-1].end == len(SECTIONED_SHEET)
# The tail row is inside the last section rather than in no concept at all.
assert "Tilbudt fastpris" in SECTIONED_SHEET[on[-1].start : on[-1].end]
def test_a_section_row_survives_the_orphan_check() -> None:
"""A stated exception, because without it the rule cannot fire at all.
The orphan check reads the lines UNDER a candidate's first line, which is
right for a heading -- a heading with nothing under it names nothing. A
section row carries its content in its own cells, so every one-row section
is bodiless by that test and all but the last would be dropped. The
exception is written once, on the rule, and it is what the count below
proves.
"""
on = okf_propose_segments.find_candidates(SECTIONED_SHEET, sheet_section_rows=True)
single_row = [c for c in on if c.rule == okf_propose_segments.RULE_SHEET_SECTION][:3]
assert len(single_row) == 3
for candidate in single_row:
assert len(SECTIONED_SHEET[candidate.start : candidate.end].splitlines()) == 1
def test_a_table_without_section_rows_is_untouched() -> None:
"""First known-negative: identical objects, not merely an equal count."""
off = okf_propose_segments.find_candidates(UNSECTIONED_SHEET)
on = okf_propose_segments.find_candidates(UNSECTIONED_SHEET, sheet_section_rows=True)
assert len(off) == 1
assert off == on
def test_a_lone_numbered_row_is_not_a_section() -> None:
"""Second known-negative, and the reason the rule counts a RUN.
`16000` is a quantity. A rule that read one numbered cell as a section
would cut a sheet at every stated basis, which is the ungated widening this
module has already refused once for outline candidates.
"""
off = okf_propose_segments.find_candidates(LONE_NUMERIC_ROW)
on = okf_propose_segments.find_candidates(LONE_NUMERIC_ROW, sheet_section_rows=True)
assert len(off) == 1
assert off == on
def test_the_flag_composes_with_keep_table_heading() -> None:
"""Both on: the heading keeps the preamble, the sections stay separate.
D1 decides where the FIRST concept starts; this rule decides where the
block is cut. They read different things and neither disables the other,
which is what makes the pair safe to expose as two flags rather than one.
"""
on = okf_propose_segments.find_candidates(
SECTIONED_SHEET, sheet_section_rows=True, keep_table_heading=True
)
assert [(c.number, c.rule) for c in on] == [
(None, okf_propose_segments.RULE_HEADING),
("01", okf_propose_segments.RULE_SHEET_SECTION),
("11+12", okf_propose_segments.RULE_SHEET_SECTION),
("02", okf_propose_segments.RULE_SHEET_SECTION),
("07", okf_propose_segments.RULE_SHEET_SECTION),
]
assert on[0].start == 0
assert on[0].end == on[1].start
def test_the_rule_names_itself_in_derived() -> None:
"""A proposal an operator disagrees with is traceable to the rule."""
plan = okf_propose_segments.build_plan(
Path("ark.xlsx"),
SECTIONED_SHEET,
b"ark",
okf_type="requirement",
proposed_at="1970-01-01T00:00:00Z",
sheet_section_rows=True,
)
sections = [e for e in plan["entries"] if e["title"].startswith("01")]
assert len(sections) == 1
assert sections[0]["derived"] == [
okf_propose_segments.PROPOSED_MARKER,
okf_propose_segments.RULE_SHEET_SECTION,
]
# The label becomes the directory and is stripped from the stem, the same
# way a section number does for a heading.
assert sections[0]["path"] == "01/felleskostnader.md"
def test_sheet_section_rows_takes_no_argument(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""A boolean at the CLI. The rule's one number is a module constant."""
source = tmp_path / "ark.md"
source.write_text(SECTIONED_SHEET, encoding="utf-8")
out = tmp_path / "ark.json"
with pytest.raises(SystemExit) as exit_info:
okf_propose_segments.main([str(source), "--out", str(out), "--sheet-section-rows", "3"])
assert exit_info.value.code == 2
assert "usage:" in capsys.readouterr().err

48
tests/test_provenance.py
View file

@ -28,6 +28,7 @@ import pytest
from llm_ingestion_okf.extract import extract_text, source_units
from llm_ingestion_okf.inbox import render_inbox_concept
from llm_ingestion_okf.propose import RULE_SHEET_SECTION, find_candidates
from llm_ingestion_okf.profiles import (
DEFAULT,
SEGMENTED_OKF_V0_2,
@ -229,6 +230,53 @@ def test_a_spreadsheet_concept_names_the_sheet_and_its_rows() -> None:
assert "source_rows: [1, 1]\n" in document
@requires_extract
def test_each_sheet_section_names_its_own_rows() -> None:
"""D3: a section's locator is the section's rows, not the sheet's.
The rule cuts one table block into one candidate per numbered row, so the
provenance layer is what decides whether those concepts are addressable at
all. It reads the unit table built AT EXTRACTION and the candidate's own
span, and neither knows about the rule — which is exactly why this has to
be measured rather than assumed: a locator that reported the whole sheet
for every section would look right in the frontmatter and point at nothing.
"""
data, text = _extract("prisark.xlsx")
units = source_units("prisark.xlsx", data, text)
candidates = find_candidates(text, sheet_section_rows=True)
sections = [c for c in candidates if c.rule == RULE_SHEET_SECTION]
assert [c.number for c in sections] == ["01", "02", "03", "04"]
located = []
for candidate in sections:
document = render_inbox_concept(
text,
okf_type="reference",
title=candidate.title,
source_file="prisark.xlsx",
source_bytes=data,
ingested_at="2026-09-08T00:00:00Z",
profile=SEGMENTED_OKF_V0_2,
units=units,
span=(candidate.start, candidate.end),
)
assert "source_sheet: Prisark\n" in document
located.append(
next(line for line in document.splitlines() if line.startswith("source_rows:"))
)
# Four sections, four DIFFERENT row locators, ascending, and each one is
# the workbook's own row number rather than a line in the extracted text.
# The last section stops at the second sheet's heading, which is the next
# mark: a span crossing two sheets would get no sheet and no rows at all,
# and the fixture is what proves it does not.
assert located == [
"source_rows: [3, 3]",
"source_rows: [4, 4]",
"source_rows: [5, 5]",
"source_rows: [6, 6]",
]
@requires_extract
def test_a_range_spanning_two_sheets_names_no_sheet_and_no_rows() -> None:
# A row number is only a place in the original once a sheet is named. A