feat(propose,consume,tools): the type that declares nothing, and the prefix that is not a word
Three of round 9's four measured holes, each closed with a rule chosen on a measurement rather than named as a limit. `rtf` GIVES 0 SEGMENTS -> 6 of 6 AUTHORED TITLES over N = 4. The container has no heading style, so the author's title is bold text. The grammar is markdown, not `rtf`: the converter already writes that title as `**...**` in the same output every office row produces, so no `rtf`-only heading form exists. Three parameters were swept over 47 readable documents and ONE carried -- refusing a line that ends in terminal punctuation takes false-positive lines from 9-12 to 1-2. A maximum title length (unlimited/40/60/80/120) and a must-stand-between-blank-lines clause are both FLAT, so neither is in the rule. The last false positive is closed by G1, the principle `_gate_outline` already carries: recovery yields to declaration. False positives are then 0 of the 31 declaring documents by construction, and 0 of 27 on the corpus. Reach: 2 of 39 corpus documents, both `docx`, 0 of 33 `pdf` and 0 of 2 `xlsx`. Behind `--bold-title`, default OFF pending the hit@8 measurement; the default bundle is byte-identical without it. BOTH ALTERNATIVES THE ORDER NAMED WERE MEASURED AND FELLED. A fourth hand-laid fixture DECLARES heading styles in a stylesheet and the converter discards them, emitting the same bold line -- so "read the declared headings out of the markdown" has nothing to read. `rtf` -> `docx` -> markdown yields 0 ATX headings on that same document, because the loss is in the `rtf` READER before any writer sees the style. Fixtures are hand-laid in `make_k2_office.py` with the fasit written first; they live in their own directory because Door B walks a drop directory recursively and `k2-office/` reads its N off the listing. THE PREFIX OVER-MATCH: THREE CANDIDATES MEASURED, ALL THREE FAILED ON ONE ROW. Re-measured on the pinned 453-concept bundle with the control run first: `under` occurs 79 times by equality and matches 172 by prefix, `undersjoisk` 0 and 172, `bilateral` 0 and 400 of 453, `standhaftig` 0 and 219. The two extra known-negatives were FOUND, not chosen -- every 4-character prefix ranked by document frequency, then a real word taken from the widest. A longer floor (5-8), a coverage share (0.5-0.8) and a long-words-only floor (>= 8) each cost row 1 its rank on the default bundle and the whole row on Arm B. Decomposed: row 1's token `prisene` reaches its gold document through `pris|sammenstilling` on four characters -- 0.57 of one word and 0.22 of the other -- so the over-match and the wanted match are one mechanism. THE FOURTH CANDIDATE IS THE ANSWER: the shared prefix must be a WORD the bundle uses. `pris` is; `bila` and `stan` are not. `bilateral` 400 -> 0 and 512 -> 0, `standhaftig` 219 -> 56 and 235 -> 33, every hit@8 row keeping rank 1 on BOTH bundles. `undersjoisk` stops at 162 because `under` IS a word here -- a genuine Norwegian morpheme, so that residual is a different answer, not a ceiling. ON by default (`--no-stem-prefix`), pinned with its own known-negative on the shipped bytes. THE SHIM: a path importer holds the object `module_from_spec` made, and `sys.modules[__name__] = _impl` never reaches it. Measured under both counting methods -- 3 of 76 public names by `vars()`. One line copies the public names into this file's globals; the dunder filter is load-bearing, because an unfiltered copy overwrites `__name__` before the next line uses it as the alias key. It restores attribute ACCESS and not patch-through, which is why the alias stays. A CHANGELOG note under 0.7.0 and a shim docstring line say so, since what the consumer asked for was the note. Suite 1515 -> 1535. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
6ff84d71c8
commit
191de89f41
16 changed files with 1100 additions and 22 deletions
59
tests/fixtures/k2-office-fasit.json
vendored
59
tests/fixtures/k2-office-fasit.json
vendored
|
|
@ -38,5 +38,64 @@
|
|||
"strings": 59,
|
||||
"container": "flat stream: title paragraph, intro paragraph, 20 table rows, caption paragraph, 4 table rows"
|
||||
}
|
||||
},
|
||||
"rtf_variants": {
|
||||
"_note": [
|
||||
"Hand-counted 2026-09-09 from the AUTHORED content in",
|
||||
"make_k2_office.py, BEFORE round 10's bold-title rule was measured",
|
||||
"against it -- never from a converter's output. `authored_titles` is",
|
||||
"what a reader of the source would call a section name; it is NOT a",
|
||||
"count of what any container manages to declare, which is the whole",
|
||||
"point: rtf has no heading style, so the author's title is bold text.",
|
||||
"`bold_runs_that_are_not_titles` is the known-negative denominator."
|
||||
],
|
||||
"counted_by": "hand, from tests/fixtures/make_k2_office.py",
|
||||
"counted_at": "2026-09-09",
|
||||
"documents": {
|
||||
"krav-rikt-tekstformat.rtf": {
|
||||
"authored_titles": 1,
|
||||
"titles": [
|
||||
"Kravspesifikasjon for tunnelbelysning"
|
||||
],
|
||||
"bold_runs_that_are_not_titles": 0,
|
||||
"declares_heading_style": false,
|
||||
"container": "title bold AND at 16pt, then intro, the 20-row table, the caption and the 4x4 grid"
|
||||
},
|
||||
"krav-rikt-tekstformat-fet-alene.rtf": {
|
||||
"authored_titles": 2,
|
||||
"titles": [
|
||||
"Kravspesifikasjon for tunnelbelysning",
|
||||
"Luminanskrav per trafikklasse"
|
||||
],
|
||||
"bold_runs_that_are_not_titles": 0,
|
||||
"declares_heading_style": false,
|
||||
"container": "both titles bold at BODY size, no point-size change anywhere"
|
||||
},
|
||||
"krav-rikt-tekstformat-fet-i-avsnitt.rtf": {
|
||||
"authored_titles": 1,
|
||||
"titles": [
|
||||
"Kravspesifikasjon for tunnelbelysning"
|
||||
],
|
||||
"bold_runs_that_are_not_titles": 1,
|
||||
"declares_heading_style": false,
|
||||
"container": "one bold title, then a sentence carrying one bold phrase that continues past it"
|
||||
},
|
||||
"krav-rikt-tekstformat-stil.rtf": {
|
||||
"authored_titles": 2,
|
||||
"titles": [
|
||||
"Kravspesifikasjon for tunnelbelysning",
|
||||
"Luminanskrav per trafikklasse"
|
||||
],
|
||||
"bold_runs_that_are_not_titles": 0,
|
||||
"declares_heading_style": true,
|
||||
"container": "a stylesheet declaring \\s1 and \\s2, applied to the two title paragraphs"
|
||||
}
|
||||
},
|
||||
"totals": {
|
||||
"documents": 4,
|
||||
"authored_titles": 6,
|
||||
"bold_runs_that_are_not_titles": 1,
|
||||
"documents_declaring_a_heading_style": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
5
tests/fixtures/k2-rtf-variants/krav-rikt-tekstformat-fet-alene.rtf
vendored
Normal file
5
tests/fixtures/k2-rtf-variants/krav-rikt-tekstformat-fet-alene.rtf
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
{\rtf1\ansi\ansicpg1252\deff0{\fonttbl{\f0\froman Times New Roman;}}\pard\sa180\b Kravspesifikasjon for tunnelbelysning\b0\par
|
||||
\pard\sa180 Dokumentet samler kravene til belysning i vegtunneler over 500 meter.\par
|
||||
\pard\sa180\b Luminanskrav per trafikklasse\b0\par
|
||||
\pard\sa180 Terskelluminansen er 145 candela og gjelder i hele overgangssonen.\par
|
||||
}
|
||||
4
tests/fixtures/k2-rtf-variants/krav-rikt-tekstformat-fet-i-avsnitt.rtf
vendored
Normal file
4
tests/fixtures/k2-rtf-variants/krav-rikt-tekstformat-fet-i-avsnitt.rtf
vendored
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
{\rtf1\ansi\ansicpg1252\deff0{\fonttbl{\f0\froman Times New Roman;}}\pard\sa180\b Kravspesifikasjon for tunnelbelysning\b0\par
|
||||
\pard\sa180 Dokumentet samler kravene til belysning i vegtunneler over 500 meter.\par
|
||||
\pard\sa180 Avvik fra kravene krever soknad om \b fravik\b0 fra byggherren for arbeidet starter.\par
|
||||
}
|
||||
5
tests/fixtures/k2-rtf-variants/krav-rikt-tekstformat-stil.rtf
vendored
Normal file
5
tests/fixtures/k2-rtf-variants/krav-rikt-tekstformat-stil.rtf
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
{\rtf1\ansi\ansicpg1252\deff0{\fonttbl{\f0\froman Times New Roman;}}{\stylesheet{\s0 Normal;}{\s1\b\fs32 heading 1;}{\s2\b\fs28 heading 2;}}\pard\s1\b\fs32 Kravspesifikasjon for tunnelbelysning\b0\fs24\par
|
||||
\pard\s0\sa180 Dokumentet samler kravene til belysning i vegtunneler over 500 meter.\par
|
||||
\pard\s2\b\fs28 Luminanskrav per trafikklasse\b0\fs24\par
|
||||
\pard\s0\sa180 Terskelluminansen er 145 candela og gjelder i hele overgangssonen.\par
|
||||
}
|
||||
110
tests/fixtures/make_k2_office.py
vendored
110
tests/fixtures/make_k2_office.py
vendored
|
|
@ -26,6 +26,11 @@ from pathlib import Path
|
|||
|
||||
HERE = Path(__file__).parent
|
||||
OUT = HERE / "k2-office"
|
||||
#: Round 10's rtf variants live in their OWN directory, and that is not tidiness.
|
||||
#: `test_k2_office_fixtures.py` reads N off `k2-office/` by listing it, because
|
||||
#: Door B walks a drop directory RECURSIVELY -- anything parked beside the three
|
||||
#: containers would enter that run and the denominator would stop being three.
|
||||
RTF_OUT = HERE / "k2-rtf-variants"
|
||||
|
||||
TITLE = "Kravspesifikasjon for tunnelbelysning"
|
||||
INTRO = "Dokumentet samler kravene til belysning i vegtunneler over 500 meter."
|
||||
|
|
@ -307,6 +312,95 @@ def _rtf_row(row: tuple[str, ...]) -> str:
|
|||
return "\\trowd\\trgaph108" + borders + "\n" + cells + "\\row\n"
|
||||
|
||||
|
||||
# --- the three rtf VARIANTS round 10 added ------------------------------------
|
||||
#
|
||||
# WHY THEY EXIST. Round 9 measured the `rtf` row on ONE document and it came
|
||||
# back at 0 of 0 declared headings, 0 concepts, 1368 of 1368 characters in no
|
||||
# segment. A repair proposed on N = 1 is a repair fitted to one file, so round
|
||||
# 10's order required N >= 3 before any rule was written. Three lawful sources
|
||||
# were offered; this is the first of them, and it is the one this repository's
|
||||
# committed-fixture policy allows: hand-laid here, in the same file, with the
|
||||
# fasit written BEFORE the measurement.
|
||||
#
|
||||
# WHAT EACH ONE VARIES, and it is one axis each:
|
||||
#
|
||||
# -fet-alene the title is bold at BODY size, with no point-size change
|
||||
# at all. Variant A (`krav-rikt-tekstformat.rtf`) sets its
|
||||
# title bold AND at 16pt, so without this document a rule
|
||||
# reading "bold" could be passing on the size instead.
|
||||
# -fet-i-avsnitt a bold PHRASE inside a running sentence, which must NOT
|
||||
# become a boundary. The document's own known-negative.
|
||||
# -stil the container DECLARES heading styles (`\\s1`, `\\s2`) in a
|
||||
# stylesheet. This is the control for the whole repair, and
|
||||
# it is a control that FAILED in an informative direction:
|
||||
# measured, the vendored converter discards the style and
|
||||
# emits the same bold line, so "read the declared style"
|
||||
# is not a route that exists for `rtf`. The fixture is kept
|
||||
# precisely because it pins that.
|
||||
#
|
||||
# The bodies are deliberately short. These documents answer where a boundary
|
||||
# is, not how much text survives -- variant A already carries the coverage and
|
||||
# fidelity counts for the container.
|
||||
|
||||
RTF_VARIANT_TITLE = "Kravspesifikasjon for tunnelbelysning"
|
||||
RTF_VARIANT_SECTION = "Luminanskrav per trafikklasse"
|
||||
RTF_VARIANT_BODY_ONE = "Dokumentet samler kravene til belysning i vegtunneler over 500 meter."
|
||||
RTF_VARIANT_BODY_TWO = "Terskelluminansen er 145 candela og gjelder i hele overgangssonen."
|
||||
#: The bold RUN in `-fet-i-avsnitt`, set inside a sentence that continues past
|
||||
#: it. Authored as emphasis, never as a title.
|
||||
RTF_VARIANT_EMPHASIS = "fravik"
|
||||
|
||||
|
||||
def _rtf_document(paragraphs: list[str]) -> bytes:
|
||||
"""Wrap hand-written paragraph bodies in the minimal rtf container."""
|
||||
header = "{\\rtf1\\ansi\\ansicpg1252\\deff0{\\fonttbl{\\f0\\froman Times New Roman;}}"
|
||||
return (header + "".join(paragraphs) + "}").encode("ascii")
|
||||
|
||||
|
||||
def rtf_bold_only_bytes() -> bytes:
|
||||
"""Title and section name bold at BODY size -- no point-size change."""
|
||||
return _rtf_document(
|
||||
[
|
||||
"\\pard\\sa180\\b " + _rtf_escape(RTF_VARIANT_TITLE) + "\\b0\\par\n",
|
||||
"\\pard\\sa180 " + _rtf_escape(RTF_VARIANT_BODY_ONE) + "\\par\n",
|
||||
"\\pard\\sa180\\b " + _rtf_escape(RTF_VARIANT_SECTION) + "\\b0\\par\n",
|
||||
"\\pard\\sa180 " + _rtf_escape(RTF_VARIANT_BODY_TWO) + "\\par\n",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def rtf_bold_inside_paragraph_bytes() -> bytes:
|
||||
"""One bold title, and one bold phrase mid-sentence that is not a title."""
|
||||
sentence = (
|
||||
"Avvik fra kravene krever soknad om "
|
||||
+ "\\b "
|
||||
+ _rtf_escape(RTF_VARIANT_EMPHASIS)
|
||||
+ "\\b0 "
|
||||
+ " fra byggherren for arbeidet starter."
|
||||
)
|
||||
return _rtf_document(
|
||||
[
|
||||
"\\pard\\sa180\\b " + _rtf_escape(RTF_VARIANT_TITLE) + "\\b0\\par\n",
|
||||
"\\pard\\sa180 " + _rtf_escape(RTF_VARIANT_BODY_ONE) + "\\par\n",
|
||||
"\\pard\\sa180 " + sentence + "\\par\n",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def rtf_styled_bytes() -> bytes:
|
||||
"""The container DECLARES heading styles. The converter discards them."""
|
||||
stylesheet = "{\\stylesheet{\\s0 Normal;}{\\s1\\b\\fs32 heading 1;}{\\s2\\b\\fs28 heading 2;}}"
|
||||
return _rtf_document(
|
||||
[
|
||||
stylesheet,
|
||||
"\\pard\\s1\\b\\fs32 " + _rtf_escape(RTF_VARIANT_TITLE) + "\\b0\\fs24\\par\n",
|
||||
"\\pard\\s0\\sa180 " + _rtf_escape(RTF_VARIANT_BODY_ONE) + "\\par\n",
|
||||
"\\pard\\s2\\b\\fs28 " + _rtf_escape(RTF_VARIANT_SECTION) + "\\b0\\fs24\\par\n",
|
||||
"\\pard\\s0\\sa180 " + _rtf_escape(RTF_VARIANT_BODY_TWO) + "\\par\n",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def rtf_bytes() -> bytes:
|
||||
body = [
|
||||
"{\\rtf1\\ansi\\ansicpg1252\\deff0",
|
||||
|
|
@ -323,11 +417,15 @@ def rtf_bytes() -> bytes:
|
|||
|
||||
if __name__ == "__main__":
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
RTF_OUT.mkdir(parents=True, exist_ok=True)
|
||||
written = {
|
||||
"krav-presentasjon.pptx": build_container(pptx_parts()),
|
||||
"krav-tekstdokument.odt": build_container(odt_parts(), stored_first="mimetype"),
|
||||
"krav-rikt-tekstformat.rtf": rtf_bytes(),
|
||||
OUT / "krav-presentasjon.pptx": build_container(pptx_parts()),
|
||||
OUT / "krav-tekstdokument.odt": build_container(odt_parts(), stored_first="mimetype"),
|
||||
OUT / "krav-rikt-tekstformat.rtf": rtf_bytes(),
|
||||
RTF_OUT / "krav-rikt-tekstformat-fet-alene.rtf": rtf_bold_only_bytes(),
|
||||
RTF_OUT / "krav-rikt-tekstformat-fet-i-avsnitt.rtf": rtf_bold_inside_paragraph_bytes(),
|
||||
RTF_OUT / "krav-rikt-tekstformat-stil.rtf": rtf_styled_bytes(),
|
||||
}
|
||||
for name, payload in sorted(written.items()):
|
||||
(OUT / name).write_bytes(payload)
|
||||
print(f"wrote {name} ({len(payload)} bytes)")
|
||||
for path, payload in sorted(written.items()):
|
||||
path.write_bytes(payload)
|
||||
print(f"wrote {path.parent.name}/{path.name} ({len(payload)} bytes)")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue