feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id
Round 9: the four rests in STATE's NESTE that needed no operator decision.
CLAUSE 1 CLASSIFIED BY THE NUMBER, NOT THE TITLE. `_TRAILING_PAGE_NUMBER`
admitted a candidate into a contents run by asking whether the title ended in
an integer -- a question about the number. A drawing's dimension chain, a
schematic's labels, a door schedule, a coordinate column and a soil-layer
table all end in integers and name nothing. Measured over the 43-document
corpus: 68 candidates discarded over 11 of 39 readable documents, of which
19 over 5 documents are data rows.
That corrects round 8's own decomposition. Its "four misclassified numeric
tables and seven real contents listings" needs each document on one side, and
two of the eleven are both. Read across all 68 titles rather than the
three-title sample: 5 documents carry a data row, 8 carry a real entry.
`--contents-name` requires a NAME to survive stripping the page number. The
threshold is SWEPT, not chosen, and collapses at both ends: at an alphabetic
run of 1 a door schedule keeps a stray `V` and 13 of 19 are rescued; at 3 the
two-letter section name `VA` stops being a name, falls out of run membership,
and takes `RIB`, `MMI` and `Tittelfelt` below `CONTENTS_RUN` with it -- one
acronym costing four REAL entries. At 2: 16 of 19 rescued, 0 of 49 regressed.
The three not rescued carry a real word and are named rather than rounded off.
THE CONVERTER'S ANCHOR WAS IN THE CONCEPT ID. Pandoc writes a sheet as
`## <name> {#sheet-N}` and a titled slide as `## <title> {#slide-N}`. Because
a filename is reduced FROM the title, the anchor reached both. Operator
authorised the strip 2026-09-09 after the exposure was counted: 2 of 810
concepts on the previous default bundle, 2 of 1108 on Arm B, 1 of 26 on the
operator's folder. Two ids renamed, one of which `portfolio-optimiser` has
cited in writing; both are in the report so that message can be sent.
One rule in one function, read by BOTH title-forming sites -- a rule in only
one would leave the id and the title naming the same concept differently. The
known-negative is the point: `Mal for {kundenavn}` is a title an author wrote.
odt/rtf/pptx MEASURED END TO END FOR THE FIRST TIME, on hand-built documents,
because the corpus denominator is genuinely zero (86 files: 66 pdf, 10 docx,
4 xlsx, 2 zip, 2 smc, 2 doc). `_EVIDENCE` gains a third class rather than
stretching an existing one: `constructed` means the row has met a document,
but not one anyone wrote for their own purposes. odt 1 of 1 declared headings;
pptx 2 of 2 on a deck that declares slide titles and 0 of 2 on one that does
not -- round 7's reading of pptx was a fixture property, not the format; rtf
0 segments, because the container has no heading style and the author's title
is bold text. rtf is the one open finding.
ACCEPTANCE, all four. The 12-position reference is label-identical in BOTH
readings (pdf 7/8, docx 3/3, xlsx 0/1 or 1/1, sheet 10/12 or 11/12). One K2
bundle carrying both changes: 453 concepts / 865 md, hit@8 [1,1,1,1,1,None]
on it AND on Arm B, with the known-negative still reproducing on the new
bytes. `okf project` byte-equal to `okf build`, `diff -r` empty. Consumer
cost is a re-run: 436/832 -> 453/865, digest 21af4a1aa98315cf.
Three published numbers corrected: README's 596 tests (1515), README's "15
concepts out" for `okf project` (that was the O6 defect; it is 26), and O6's
print-mode method, which does not reproduce without --allowedTools.
Report: docs/2026-09-09-k3-runde9-restene.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
fbad5b0eab
commit
6ff18fd703
15 changed files with 964 additions and 67 deletions
123
tests/test_contents_name_part.py
Normal file
123
tests/test_contents_name_part.py
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
"""Clause 1 asks whether a title NAMES something, not whether a number ends it.
|
||||
|
||||
`_TRAILING_PAGE_NUMBER` reads a run of consecutive titles ending in an integer
|
||||
as a contents listing and discards it. That predicate cannot tell a contents
|
||||
entry from a run of DATA ROWS, because a drawing's dimension chain, a P&ID's
|
||||
schematic labels, a door schedule and a borehole log's coordinate column all
|
||||
end in integers too. Measured on the 43-document corpus, clause 1 dropped 68
|
||||
candidates over 11 of 39 readable documents, and 19 of those 68 over 5 of the
|
||||
11 were data rows whose titles -- `40.23`, `300 1`, `3000 V 1`, `619649.097`
|
||||
-- name nothing at all.
|
||||
|
||||
Round 8 removed the DAMAGE (`--close-span-gaps` carries a removed mark's text
|
||||
on the mark above), so what is left is classification. The repair is one
|
||||
clause: a title is a contents entry only if a NAME survives stripping the page
|
||||
number.
|
||||
|
||||
WHERE THE THRESHOLD SITS IS MEASURED, NOT CHOSEN. Swept over the corpus at an
|
||||
alphabetic run of >= 1, >= 2 and >= 3 characters, and it collapses at both
|
||||
ends: at >= 1 three data rows keep a stray single letter (`3000 V`) and stay
|
||||
misclassified (13 of 19 rescued); at >= 3 a real contents list breaks, because
|
||||
`VA` is a two-letter section name and dropping it out of run membership takes
|
||||
`RIB`, `MMI` and `Tittelfelt` below `CONTENTS_RUN` with it -- one acronym
|
||||
costing four real entries. At >= 2 the rescue is 16 of 19 and the regression 0
|
||||
of 49 real entries. This file holds BOTH sides, and the second is the one that
|
||||
matters: a rule that rescues a table by also rescuing contents lists has moved
|
||||
the defect rather than fixed it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from llm_ingestion_okf import cli
|
||||
from llm_ingestion_okf.propose import find_candidates
|
||||
|
||||
DEFAULT = dict(
|
||||
outline_run=cli.DEFAULT_OUTLINE_RUN,
|
||||
table_grid=cli.DEFAULT_TABLE_GRID,
|
||||
unit_fold=cli.DEFAULT_UNIT_FOLD,
|
||||
keep_table_heading=cli.DEFAULT_KEEP_TABLE_HEADING,
|
||||
sheet_section_rows=cli.DEFAULT_SHEET_SECTION_ROWS,
|
||||
drop_wrapped_outline=cli.DEFAULT_DROP_WRAPPED_OUTLINE,
|
||||
outline_gate=cli.DEFAULT_OUTLINE_GATE,
|
||||
first_span_from_zero=cli.DEFAULT_FIRST_SPAN_FROM_ZERO,
|
||||
close_span_gaps=cli.DEFAULT_CLOSE_SPAN_GAPS,
|
||||
contents_name=cli.DEFAULT_CONTENTS_NAME,
|
||||
)
|
||||
|
||||
#: The borehole-log shape, rebuilt from the corpus document's geometry: a
|
||||
#: coordinate column whose rows are `<northing>.<decimals>` and end in digits.
|
||||
#: Five consecutive rows, comfortably over `CONTENTS_RUN`, each with a body so
|
||||
#: the orphan check does not remove them first.
|
||||
DATA_ROWS = """# Geoteknisk borelogg Testdokument
|
||||
|
||||
Dette dokumentet har en kjent struktur: en koordinatkolonne der hver rad er
|
||||
et tall, og ingen av dem navngir noe som helst.
|
||||
|
||||
## 619482.244
|
||||
|
||||
Proevepunkt P20, naverboring, fire proever i sjiktet 0-4 meter.
|
||||
|
||||
## 619551.473
|
||||
|
||||
Proevepunkt P21, naverboring, tre proever i sjiktet 0-3 meter.
|
||||
|
||||
## 619575.846
|
||||
|
||||
Proevepunkt P22, naverboring, fem proever i sjiktet 0-5 meter.
|
||||
|
||||
## 619649.097
|
||||
|
||||
Proevepunkt P23, naverboring, to proever i sjiktet 0-2 meter.
|
||||
|
||||
## 619712.518
|
||||
|
||||
Proevepunkt P24, naverboring, seks proever i sjiktet 0-6 meter.
|
||||
"""
|
||||
|
||||
#: The known-negative, and it carries the exact boundary the sweep found: a
|
||||
#: real contents listing whose shortest entry is a TWO-letter section name.
|
||||
#: Every one of these must still be discarded.
|
||||
REAL_CONTENTS = """# Modellansvar Testdokument
|
||||
|
||||
Innhold
|
||||
|
||||
## Ansvar fagmodellansvarlig 5
|
||||
|
||||
## Kontaktpersoner 7
|
||||
|
||||
## RIB 9
|
||||
|
||||
## VA 11
|
||||
|
||||
## MMI 13
|
||||
|
||||
## Tittelfelt 15
|
||||
|
||||
Dette avsnittet staar under listen og gir den en kropp.
|
||||
"""
|
||||
|
||||
|
||||
def _titles(document: str) -> list[str]:
|
||||
return [candidate.title for candidate in find_candidates(document, **DEFAULT)]
|
||||
|
||||
|
||||
def test_a_run_of_numeric_data_rows_is_not_a_contents_listing() -> None:
|
||||
"""The known-positive: nothing survives stripping, so nothing is a name."""
|
||||
titles = _titles(DATA_ROWS)
|
||||
for row in ("619482.244", "619551.473", "619575.846", "619649.097", "619712.518"):
|
||||
assert row in titles, f"clause 1 discarded the data row {row!r} as a contents entry"
|
||||
|
||||
|
||||
def test_a_real_contents_listing_is_still_discarded() -> None:
|
||||
"""The known-negative, and the reason the threshold is 2 and not 3.
|
||||
|
||||
`VA` is two letters. At an alphabetic run of >= 3 it stops being a name,
|
||||
falls out of run membership, and the run around it drops below
|
||||
`CONTENTS_RUN` -- so the whole listing survives and four real entries are
|
||||
emitted as concepts. Measured on the corpus, that is exactly what happened.
|
||||
"""
|
||||
titles = _titles(REAL_CONTENTS)
|
||||
for entry in ("Ansvar fagmodellansvarlig", "Kontaktpersoner", "RIB", "VA", "MMI", "Tittelfelt"):
|
||||
assert not any(title.startswith(entry) for title in titles), (
|
||||
f"clause 1 emitted the contents entry {entry!r} as a concept"
|
||||
)
|
||||
169
tests/test_converter_attribute_titles.py
Normal file
169
tests/test_converter_attribute_titles.py
Normal file
|
|
@ -0,0 +1,169 @@
|
|||
"""A converter's own attribute is not part of a document's title.
|
||||
|
||||
Converting a spreadsheet, pandoc writes each sheet as
|
||||
`## <sheet name> {#sheet-<n>}`, and a deck whose slides carry title
|
||||
placeholders as `## <slide title> {#slide-<n>}`. The brace block is pandoc's
|
||||
attribute syntax -- an anchor it emits so its own HTML output can link to the
|
||||
section. It is not something the author of the spreadsheet or the deck wrote.
|
||||
|
||||
It reached the artifact twice over: into a concept's `title`, and from there
|
||||
into the concept's id, because a filename is REDUCED from the title. Measured
|
||||
2026-09-09 on the two bundles on disk, `2 of 810` concepts on the default K2
|
||||
bundle and `2 of 1108` on Arm B carried it, and the ids were
|
||||
`del-ii-bilag-7-prisskjema/prissammenstilling-sheet-1` and
|
||||
`del-ii-bilag-0-dokumentliste-del-ii/ark1-sheet-1`.
|
||||
|
||||
Round 7 measured this and left it, because the repair RENAMES concept ids
|
||||
another repository has cited in writing, and renaming those is not a
|
||||
measurement's call. The operator authorised the strip on 2026-09-09; the
|
||||
exposure is in the round-9 report with both ids side by side, before and
|
||||
after.
|
||||
|
||||
TWO TITLE SITES, ONE RULE. `propose` forms a segment's title from an ATX
|
||||
heading and `structure` derives a document title from its leading heading, so
|
||||
a rule living in one of them would strip the attribute on one path and leave
|
||||
it on the other. Both read `strip_converter_attribute`.
|
||||
|
||||
THE KNOWN-NEGATIVE IS THE POINT. The pattern must match pandoc's attribute
|
||||
form and nothing else: a heading that legitimately contains braces -- a
|
||||
placeholder in a template, a code identifier -- is a title the author wrote,
|
||||
and stripping it would be the same defect pointed the other way.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from llm_ingestion_okf import cli
|
||||
from llm_ingestion_okf.extract import extract_text, strip_converter_attribute
|
||||
from llm_ingestion_okf.propose import find_candidates
|
||||
from llm_ingestion_okf.structure import derive_document_structure
|
||||
|
||||
DEFAULT = dict(
|
||||
outline_run=cli.DEFAULT_OUTLINE_RUN,
|
||||
table_grid=cli.DEFAULT_TABLE_GRID,
|
||||
unit_fold=cli.DEFAULT_UNIT_FOLD,
|
||||
keep_table_heading=cli.DEFAULT_KEEP_TABLE_HEADING,
|
||||
sheet_section_rows=cli.DEFAULT_SHEET_SECTION_ROWS,
|
||||
drop_wrapped_outline=cli.DEFAULT_DROP_WRAPPED_OUTLINE,
|
||||
outline_gate=cli.DEFAULT_OUTLINE_GATE,
|
||||
first_span_from_zero=cli.DEFAULT_FIRST_SPAN_FROM_ZERO,
|
||||
close_span_gaps=cli.DEFAULT_CLOSE_SPAN_GAPS,
|
||||
contents_name=cli.DEFAULT_CONTENTS_NAME,
|
||||
)
|
||||
|
||||
SHEET_DOCUMENT = """## Prissammenstilling {#sheet-1}
|
||||
|
||||
Denne posten samler prisene for hele leveransen og er den posten et
|
||||
spoersmaal om pris maa naa.
|
||||
|
||||
## Ark1 {#sheet-2}
|
||||
|
||||
Denne posten lister dokumentene i del II og har sin egen kropp.
|
||||
|
||||
## Lysbilde med krav {#slide-3}
|
||||
|
||||
Dette lysbildet bar en tittelplassholder, saa konvertereren ga det et anker.
|
||||
"""
|
||||
|
||||
#: The known-negative. Braces the AUTHOR wrote, in the two shapes that occur:
|
||||
#: a template placeholder and a code identifier. Neither is pandoc's attribute
|
||||
#: form, and neither may be touched.
|
||||
AUTHORED_BRACES = """## Mal for {kundenavn} og leveransen
|
||||
|
||||
Denne overskriften har krollparenteser forfatteren skrev selv, og den staar
|
||||
paa niva 2 sammen med den neste -- clause 2 folds a level that occurs once
|
||||
into the level above, which would empty this fixture without saying so.
|
||||
|
||||
## Feltet {"id": 4} i nyttelasten
|
||||
|
||||
Dette avsnittet beskriver et felt og staar under sin egen overskrift.
|
||||
"""
|
||||
|
||||
|
||||
def test_the_attribute_is_stripped_from_a_segment_title() -> None:
|
||||
titles = [candidate.title for candidate in find_candidates(SHEET_DOCUMENT, **DEFAULT)]
|
||||
for leaked in ("{#sheet-1}", "{#sheet-2}", "{#slide-3}"):
|
||||
assert not any(leaked in title for title in titles), (
|
||||
f"the converter attribute {leaked} reached a concept title: {titles}"
|
||||
)
|
||||
assert "Prissammenstilling" in titles
|
||||
assert "Lysbilde med krav" in titles
|
||||
|
||||
|
||||
def test_the_attribute_is_stripped_from_a_derived_document_title() -> None:
|
||||
derived = derive_document_structure(
|
||||
"# Prissammenstilling {#sheet-1}\n\nEn kropp.\n", source_file="regneark.md"
|
||||
)
|
||||
title = derived.title
|
||||
assert title == "Prissammenstilling", f"structure derived the title {title!r}"
|
||||
|
||||
|
||||
def test_authored_braces_are_left_alone() -> None:
|
||||
"""The known-negative: only pandoc's attribute form is an attribute."""
|
||||
titles = [candidate.title for candidate in find_candidates(AUTHORED_BRACES, **DEFAULT)]
|
||||
assert any("{kundenavn}" in title for title in titles), titles
|
||||
assert any('{"id": 4}' in title for title in titles), titles
|
||||
assert strip_converter_attribute('Feltet {"id": 4} i nyttelasten') == (
|
||||
'Feltet {"id": 4} i nyttelasten'
|
||||
)
|
||||
assert strip_converter_attribute("Mal for {kundenavn}") == "Mal for {kundenavn}"
|
||||
|
||||
|
||||
def test_a_deck_that_declares_slide_titles_is_named_by_them() -> None:
|
||||
"""The slide form, on a REAL conversion rather than a hand-written string.
|
||||
|
||||
The committed `pptx` fixture emits `## Slide 1` / `## Slide 2`, and round 7
|
||||
read that as the format segmenting badly. Decomposed 2026-09-09 it is not:
|
||||
that deck's title shapes carry no `<p:ph type="title"/>` placeholder, so the
|
||||
converter has nothing to name a slide WITH. Given a deck that declares one,
|
||||
the converter writes `## <authored title> {#slide-N}` -- and the attribute
|
||||
is exactly what this module strips, so the concept ends up named by the
|
||||
title the author wrote.
|
||||
|
||||
Built in memory from the committed generator rather than added to
|
||||
`tests/fixtures/k2-office/`, which holds exactly three containers and
|
||||
asserts that it does.
|
||||
"""
|
||||
pytest.importorskip("pypandoc", reason="office conversion needs the [extract] extra")
|
||||
# Loaded BY PATH: the generator lives under `tests/fixtures/` and is not on
|
||||
# `sys.path`, so `importorskip` on its name skips silently -- a green run
|
||||
# over an empty set, which is the one outcome this file exists to prevent.
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"make_k2_office", Path(__file__).parent / "fixtures" / "make_k2_office.py"
|
||||
)
|
||||
assert spec is not None and spec.loader is not None
|
||||
make = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(make)
|
||||
|
||||
def titled(shape_id: int, name: str, text: str) -> str:
|
||||
return (
|
||||
"<p:sp><p:nvSpPr>"
|
||||
f'<p:cNvPr id="{shape_id}" name="{name}"/><p:cNvSpPr/>'
|
||||
'<p:nvPr><p:ph type="title"/></p:nvPr>'
|
||||
"</p:nvSpPr><p:spPr/>" + make._pptx_text_body(text) + "</p:sp>"
|
||||
)
|
||||
|
||||
parts = make.pptx_parts()
|
||||
parts["ppt/slides/slide1.xml"] = make._pptx_slide(
|
||||
titled(2, "Tittel", make.TITLE)
|
||||
+ make._pptx_shape(3, "Ingress", make.INTRO)
|
||||
+ make._pptx_table(4, "Kravtabell", tuple(make.PAIRS))
|
||||
)
|
||||
parts["ppt/slides/slide2.xml"] = make._pptx_slide(
|
||||
titled(2, "Undertittel", make.GRID_CAPTION)
|
||||
+ make._pptx_table(3, "Luminansmatrise", make.GRID)
|
||||
)
|
||||
data = make.build_container(parts, stored_first="[Content_Types].xml")
|
||||
|
||||
text = extract_text("tittelplassholder.pptx", data)
|
||||
headings = [line for line in text.splitlines() if line.startswith("#")]
|
||||
# The known-positive for the strip: the converter DOES write the anchor
|
||||
# here, so a green assertion below is not green over an empty set.
|
||||
assert any("{#slide-" in line for line in headings), headings
|
||||
|
||||
titles = [candidate.title for candidate in find_candidates(text, **DEFAULT)]
|
||||
assert titles == [make.TITLE, make.GRID_CAPTION], titles
|
||||
|
|
@ -41,12 +41,26 @@ import okf_consume_measure # noqa: E402
|
|||
#: --bundle-id k2-trinn1-20260903 --okf-version 0.2
|
||||
#: with no arm flag at all -- the package default, which is the point.
|
||||
#:
|
||||
#: Rebuilt 2026-09-11 for `--close-span-gaps`. Digest, from inside the bundle:
|
||||
#: Rebuilt 2026-09-09 for `--contents-name` (round 9). Digest, from inside the
|
||||
#: bundle:
|
||||
#: find . -type f -print0 | sort -z | xargs -0 shasum -a 256 | shasum -a 256
|
||||
#: -> 8c93e5e3222577a2b3352ca83af980e403d3a571c3a467b83c3d8170b1df2b69
|
||||
#: -> 21af4a1aa98315cf514c4cbc6b4a9b77ce63960224d6d7b31b34d55cc67fb2ad
|
||||
#: (The previous default, `K2-bundle-default-20260911`, was
|
||||
#: 8c93e5e3222577a2b3352ca83af980e403d3a571c3a467b83c3d8170b1df2b69 at 436
|
||||
#: concepts and stays on disk.)
|
||||
#: Two independent builds of it differ in NOTHING (`diff -rq`), including
|
||||
#: `log.md`, which carries the corpus path and never the bundle's own.
|
||||
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260911"
|
||||
#:
|
||||
#: CONCEPT IDS MOVED IN THIS REBUILD, and not only because the count did.
|
||||
#: Round 9 strips pandoc's `{#sheet-N}` / `{#slide-N}` anchor where a title is
|
||||
#: formed, and a concept's filename is reduced FROM its title, so TWO ids on
|
||||
#: this bundle are renamed:
|
||||
#: del-ii-bilag-7-prisskjema/prissammenstilling-sheet-1 -> .../prissammenstilling
|
||||
#: del-ii-bilag-0-dokumentliste-del-ii/ark1-sheet-1 -> .../ark1
|
||||
#: The first is an id `portfolio-optimiser` has cited in writing. The rename
|
||||
#: was authorised by the operator on 2026-09-09 after the exposure was counted:
|
||||
#: 2 of 810 concepts on the previous default and 2 of 1108 on Arm B.
|
||||
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260912"
|
||||
GOLD_SET = PROJECT_ROOT / ".claude/projects/2026-09-07-okf-consume-prepass/hit-at-k-questions.json"
|
||||
|
||||
requires_default_bundle = pytest.mark.skipif(
|
||||
|
|
@ -58,13 +72,18 @@ requires_default_bundle = pytest.mark.skipif(
|
|||
),
|
||||
)
|
||||
|
||||
#: Measured 2026-09-11 on the bundle above. The count moved 425 -> 436 with
|
||||
#: Measured 2026-09-09 on the bundle above. The count moved 425 -> 436 with
|
||||
#: `--sheet-section-rows --keep-table-heading`; `--first-span-from-zero` and
|
||||
#: `--close-span-gaps` each moved it by NOTHING, which is the point of both --
|
||||
#: they add no boundary, they only move a span's start or its end. Round 8's
|
||||
#: rule closed 43 631 characters (2.51 % of the corpus) that were in no
|
||||
#: segment, and the count is byte-for-byte the same 436.
|
||||
EXPECTED_CONCEPTS = 436
|
||||
#: segment, and the count was byte-for-byte the same 436.
|
||||
#:
|
||||
#: 436 -> 453 with round 9's `--contents-name`, which does add concepts: a run
|
||||
#: of data rows is no longer read as a contents listing and discarded, so the
|
||||
#: candidates it was taking with it survive. Corpus-wide, 429 -> 447 candidates
|
||||
#: over 32 -> 33 documents with a plan, and characters in no segment stay 0.
|
||||
EXPECTED_CONCEPTS = 453
|
||||
EXPECTED_HITS = 5
|
||||
#: Rank per question INDEX, `None` for the row that misses on every bundle and
|
||||
#: every configuration measured so far. The identity is the index; the question
|
||||
|
|
|
|||
|
|
@ -58,19 +58,24 @@ def test_the_readme_no_longer_claims_docx_and_xlsx_fail_fast() -> None:
|
|||
assert claim.lower() not in text, f"README still claims: {claim}"
|
||||
|
||||
|
||||
def test_the_readme_states_which_rows_are_unmeasured() -> None:
|
||||
"""An unmeasured row must not read as a supported one.
|
||||
def test_the_readme_states_which_rows_are_not_measured() -> None:
|
||||
"""A row that is not `measured` must not read as a supported one.
|
||||
|
||||
Three of the five office formats have denominator ZERO in the corpus this
|
||||
work was measured on. A consumer reading the README should be able to see
|
||||
that without reading the source.
|
||||
|
||||
Reads the CLASS from the table rather than the literal `unmeasured`: round
|
||||
9 moved those three rows to `constructed`, and a test pinned to one word
|
||||
would have gone green over an empty set the moment the word changed. Every
|
||||
class that is not `measured` must be named in the README, whichever it is.
|
||||
"""
|
||||
text = README.read_text(encoding="utf-8")
|
||||
unmeasured = {s.lstrip(".") for s, e in _EVIDENCE.items() if e == "unmeasured"}
|
||||
assert unmeasured, "the evidence table lists no unmeasured rows"
|
||||
for suffix in unmeasured:
|
||||
assert suffix in text, f"README does not mention the unmeasured row {suffix}"
|
||||
assert "unmeasured" in text.lower()
|
||||
weaker = {s.lstrip("."): e for s, e in _EVIDENCE.items() if e != "measured"}
|
||||
assert weaker, "the evidence table lists no rows weaker than measured"
|
||||
for suffix, evidence in weaker.items():
|
||||
assert suffix in text, f"README does not mention the {evidence} row {suffix}"
|
||||
assert evidence in text.lower(), f"README does not use the word {evidence}"
|
||||
|
||||
|
||||
def test_the_readme_still_states_what_stays_out() -> None:
|
||||
|
|
|
|||
|
|
@ -180,11 +180,17 @@ def test_evidence_class_is_asserted_not_commented() -> None:
|
|||
|
||||
assert set(_EVIDENCE) == set(_PANDOC_FORMATS), "every row needs an evidence class"
|
||||
assert {s for s, e in _EVIDENCE.items() if e == "measured"} == {".docx", ".xlsx"}
|
||||
assert {s for s, e in _EVIDENCE.items() if e == "unmeasured"} == {
|
||||
# Since 2026-09-09 the three office rows are `constructed`, not
|
||||
# `unmeasured`: each has now been put through end to end on a hand-built
|
||||
# document with a hand-written fasit, and none of them has a corpus file.
|
||||
# The set is asserted EMPTY rather than dropped -- a class with no members
|
||||
# is a fact about this package, and a future row can re-enter it.
|
||||
assert {s for s, e in _EVIDENCE.items() if e == "constructed"} == {
|
||||
".pptx",
|
||||
".odt",
|
||||
".rtf",
|
||||
}
|
||||
assert {s for s, e in _EVIDENCE.items() if e == "unmeasured"} == set()
|
||||
|
||||
|
||||
def test_the_unparsed_set_is_empty_now_that_every_row_has_a_reader() -> None:
|
||||
|
|
|
|||
|
|
@ -16,8 +16,10 @@ becoming a memory of a run.
|
|||
|
||||
WHAT IT DOES NOT DO. Three synthetic documents in one house style are not a
|
||||
corpus. A green run here says the row is exercised, never that the format is
|
||||
covered; `_EVIDENCE` therefore stays `unmeasured` and this suite asserts that
|
||||
it does.
|
||||
covered. Round 9 (2026-09-09) measured all three end to end for the first time
|
||||
and moved them `unmeasured` -> `constructed`, a third evidence class defined in
|
||||
`extract._EVIDENCE`: the row has met a document, but not one anyone wrote for
|
||||
their own purposes. This suite asserts that class rather than the old one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -113,4 +115,4 @@ def test_the_fasit_is_hand_counted_and_committed_beside_the_fixtures() -> None:
|
|||
|
||||
def test_three_synthetic_documents_do_not_make_a_row_measured() -> None:
|
||||
"""One house style, N = 3, written by us -- that is exercise, not coverage."""
|
||||
assert [_EVIDENCE[key] for key in (".pptx", ".odt", ".rtf")] == ["unmeasured"] * 3
|
||||
assert [_EVIDENCE[key] for key in (".pptx", ".odt", ".rtf")] == ["constructed"] * 3
|
||||
|
|
|
|||
|
|
@ -79,14 +79,19 @@ def test_a_run_of_numbered_rows_cuts_the_block_into_sections() -> None:
|
|||
of the block -- so the LAST section carries the sheet's tail, which is the
|
||||
honest consequence of a rule that cuts and never discards.
|
||||
"""
|
||||
# The title carries NO `{#sheet-1}` since 2026-09-09: the converter anchor
|
||||
# is stripped where the title is formed, because a concept's id is reduced
|
||||
# from its title and the two must not name the same concept differently.
|
||||
# The fixture keeps the anchor -- it is what pandoc writes, and removing it
|
||||
# from the input would test the strip against text that never has it.
|
||||
off = okf_propose_segments.find_candidates(SECTIONED_SHEET)
|
||||
assert [(c.title, c.rule) for c in off] == [
|
||||
("Kostnadsoversikt {#sheet-1}", okf_propose_segments.RULE_TABLE_BLOCK)
|
||||
("Kostnadsoversikt", okf_propose_segments.RULE_TABLE_BLOCK)
|
||||
]
|
||||
|
||||
on = okf_propose_segments.find_candidates(SECTIONED_SHEET, sheet_section_rows=True)
|
||||
assert [(c.number, c.title, c.rule) for c in on] == [
|
||||
(None, "Kostnadsoversikt {#sheet-1}", okf_propose_segments.RULE_TABLE_BLOCK),
|
||||
(None, "Kostnadsoversikt", okf_propose_segments.RULE_TABLE_BLOCK),
|
||||
("01", "01 Felleskostnader", okf_propose_segments.RULE_SHEET_SECTION),
|
||||
("11+12", "11+12 Rigging og drift", okf_propose_segments.RULE_SHEET_SECTION),
|
||||
("02", "02 Bygning", okf_propose_segments.RULE_SHEET_SECTION),
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue