open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(propose,consume,tools): the type that declares nothing, and the prefix that is not a word

Browse source 
Three of round 9's four measured holes, each closed with a rule chosen on a
measurement rather than named as a limit.

`rtf` GIVES 0 SEGMENTS -> 6 of 6 AUTHORED TITLES over N = 4. The container has
no heading style, so the author's title is bold text. The grammar is markdown,
not `rtf`: the converter already writes that title as `**...**` in the same
output every office row produces, so no `rtf`-only heading form exists. Three
parameters were swept over 47 readable documents and ONE carried -- refusing a
line that ends in terminal punctuation takes false-positive lines from 9-12 to
1-2. A maximum title length (unlimited/40/60/80/120) and a
must-stand-between-blank-lines clause are both FLAT, so neither is in the rule.
The last false positive is closed by G1, the principle `_gate_outline` already
carries: recovery yields to declaration. False positives are then 0 of the 31
declaring documents by construction, and 0 of 27 on the corpus. Reach: 2 of 39
corpus documents, both `docx`, 0 of 33 `pdf` and 0 of 2 `xlsx`. Behind
`--bold-title`, default OFF pending the hit@8 measurement; the default bundle
is byte-identical without it.

BOTH ALTERNATIVES THE ORDER NAMED WERE MEASURED AND FELLED. A fourth hand-laid
fixture DECLARES heading styles in a stylesheet and the converter discards
them, emitting the same bold line -- so "read the declared headings out of the
markdown" has nothing to read. `rtf` -> `docx` -> markdown yields 0 ATX
headings on that same document, because the loss is in the `rtf` READER before
any writer sees the style. Fixtures are hand-laid in `make_k2_office.py` with
the fasit written first; they live in their own directory because Door B walks
a drop directory recursively and `k2-office/` reads its N off the listing.

THE PREFIX OVER-MATCH: THREE CANDIDATES MEASURED, ALL THREE FAILED ON ONE ROW.
Re-measured on the pinned 453-concept bundle with the control run first:
`under` occurs 79 times by equality and matches 172 by prefix, `undersjoisk` 0
and 172, `bilateral` 0 and 400 of 453, `standhaftig` 0 and 219. The two extra
known-negatives were FOUND, not chosen -- every 4-character prefix ranked by
document frequency, then a real word taken from the widest. A longer floor
(5-8), a coverage share (0.5-0.8) and a long-words-only floor (>= 8) each cost
row 1 its rank on the default bundle and the whole row on Arm B. Decomposed:
row 1's token `prisene` reaches its gold document through
`pris|sammenstilling` on four characters -- 0.57 of one word and 0.22 of the
other -- so the over-match and the wanted match are one mechanism.

THE FOURTH CANDIDATE IS THE ANSWER: the shared prefix must be a WORD the bundle
uses. `pris` is; `bila` and `stan` are not. `bilateral` 400 -> 0 and 512 -> 0,
`standhaftig` 219 -> 56 and 235 -> 33, every hit@8 row keeping rank 1 on BOTH
bundles. `undersjoisk` stops at 162 because `under` IS a word here -- a genuine
Norwegian morpheme, so that residual is a different answer, not a ceiling. ON
by default (`--no-stem-prefix`), pinned with its own known-negative on the
shipped bytes.

THE SHIM: a path importer holds the object `module_from_spec` made, and
`sys.modules[__name__] = _impl` never reaches it. Measured under both counting
methods -- 3 of 76 public names by `vars()`. One line copies the public names
into this file's globals; the dunder filter is load-bearing, because an
unfiltered copy overwrites `__name__` before the next line uses it as the alias
key. It restores attribute ACCESS and not patch-through, which is why the alias
stays. A CHANGELOG note under 0.7.0 and a shim docstring line say so, since
what the consumer asked for was the note.

Suite 1515 -> 1535.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 23:05:45 +02:00
commit 191de89f41
16 changed files with 1100 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

35
src/llm_ingestion_okf/cli.py
View file

@ -183,6 +183,15 @@ DEFAULT_CLOSE_SPAN_GAPS = True
#: them from a contents entry. It only ever REMOVES members from a run, so it
#: can only add concepts, never take one away. Opt-out `--no-contents-name`.
DEFAULT_CONTENTS_NAME = True
#: Round 10's rule for the type that declares nothing. `rtf` came back at 0 of
#: 0 declared headings, 0 concepts and 1368 of 1368 characters in no segment:
#: the container has no heading style, so the author's title is bold text.
#: The grammar is markdown, not `rtf` -- the converter already writes that
#: title as `**...**` in the same output every office row produces -- and it is
#: gated by the principle Arm D already carries, that recovery yields to
#: declaration. Three parameters were swept over 47 readable documents and one
#: carried (see `tests/test_bold_title.py`). Default set by measurement below.
DEFAULT_BOLD_TITLE = False
#: Round 3's two spreadsheet rules (D1 and D3), held back through rounds 5 and
#: 6 by a RETRIEVAL regression rather than by the reference: they take the
@ -250,6 +259,7 @@ def _propose_plans(
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
bold_title: bool = False,
pdf_headings: bool = False,
pdf_headings_reserve: bool = False,
ocr: bool = False,
@ -287,6 +297,7 @@ def _propose_plans(
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
bold_title=bold_title,
pdf_headings=pdf_headings,
pdf_headings_reserve=pdf_headings_reserve,
ocr=ocr,
@ -323,6 +334,7 @@ def build(
first_span_from_zero: bool = DEFAULT_FIRST_SPAN_FROM_ZERO,
close_span_gaps: bool = DEFAULT_CLOSE_SPAN_GAPS,
contents_name: bool = DEFAULT_CONTENTS_NAME,
bold_title: bool = DEFAULT_BOLD_TITLE,
pdf_headings: bool = DEFAULT_PDF_HEADINGS,
pdf_headings_reserve: bool = DEFAULT_PDF_HEADINGS_RESERVE,
ocr: bool = DEFAULT_OCR,
@ -394,6 +406,7 @@ def build(
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
bold_title=bold_title,
pdf_headings=pdf_headings,
pdf_headings_reserve=pdf_headings_reserve,
ocr=ocr,
@ -725,6 +738,27 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"pre-2026-09-09 contents-run predicate byte for byte"
),
)
build_parser.add_argument(
"--bold-title",
action="store_true",
default=DEFAULT_BOLD_TITLE,
help=(
"Read a line that is ONE bold span as a title, in a document that "
"declares no heading of its own. For the type whose container has "
"no heading style at all: `rtf` reached round 9 at 0 of 0 declared "
"headings, 0 concepts and 1368 of 1368 characters in no segment. "
"The grammar is markdown, so it reaches every type the converter "
"writes bold for, and it is inert for `pdf`, which never goes "
"through the converter. Measured over 47 readable documents: 0 "
"false positives on the 31 that declare, by the gate"
),
)
build_parser.add_argument(
"--no-bold-title",
action="store_false",
dest="bold_title",
help="The rule's explicit opt-out",
)
build_parser.add_argument(
"--pdf-headings",
choices=("none", "font", "font-reserve"),
@ -799,6 +833,7 @@ def main(argv: list[str] | None = None) -> int:
first_span_from_zero=args.first_span_from_zero,
close_span_gaps=args.close_span_gaps,
contents_name=args.contents_name,
bold_title=args.bold_title,
pdf_headings=args.pdf_headings == "font",
pdf_headings_reserve=args.pdf_headings == "font-reserve",
ocr=args.ocr,

134
src/llm_ingestion_okf/consume.py
View file

@ -745,9 +745,36 @@ def is_identifier(token: str) -> bool:
return _IDENTIFIER_RE.fullmatch(token) is not None
def tokens_match(left: str, right: str) -> bool:
def tokens_match(left: str, right: str, *, stems: frozenset[str] | None = None) -> bool:
"""Whether two tokens share a leading prefix of at least `MIN_SHARED_PREFIX`.
**THE SHARED PREFIX MUST BE A WORD** when `stems` is given, and that is
round 10's repair. The floor alone matched four characters that are not a
stem at all: measured on the pinned 453-concept bundle with the control run
first, `under` occurs 79 times by equality and matches 172 concepts by
prefix, while `undersjoisk` occurs 0 times and matches the same 172;
`bilateral` occurs 0 times and matches 400 of 453 through `bilag`;
`standhaftig` 0 and 219 through `standard`.
Three repairs were measured and all three failed on the same row: a longer
floor (5-8), a coverage share of the question word (0.5-0.8), and a floor
applied only to long words (>= 6, 8, 10, 12). Row 1's question token
`prisene` reaches its gold document through `pris|sammenstilling` on the
four characters `pris` -- 0.57 of the question word and 0.22 of the
document word -- so the over-match and the wanted match are the same
mechanism seen from two sides, and no threshold on length or coverage
separates them.
What separates them is that `pris` is a word and `bila` is not. With
`stems`, `bilateral` falls to 0 on both bundles and every hit@8 row keeps
its rank. `undersjoisk` still reaches 162 because it shares `under`, which
IS a word here -- a genuine Norwegian morpheme, so that residual is a
different answer rather than a ceiling.
The vocabulary is the BUNDLE's own, which makes a payload depend on the
corpus the way `rarity_weights` already does. Passing `None` reproduces the
pre-round-10 matcher exactly.
Symmetric, and it degrades to equality for short tokens: two 4-character
tokens match only if they are the same word.
@ -778,7 +805,15 @@ def tokens_match(left: str, right: str) -> bool:
shared = 0
while shared < limit and left[shared] == right[shared]:
shared += 1
return shared >= MIN_SHARED_PREFIX
if shared < MIN_SHARED_PREFIX:
return False
if stems is None:
return True
# Equality first, and only inside this branch. A token always answers to
# itself, whatever the corpus contains -- but the check must NOT move above
# the floor, where it would make `veg`/`veg` match and change the shipped
# rule for every token shorter than `MIN_SHARED_PREFIX`.
return left == right or left[:shared] in stems
#: One declared vocabulary family, spelled once: within it, any term answers to
@ -846,7 +881,12 @@ def searchable_text(concepts: Sequence["Concept"]) -> list[str]:
]
def rarity_weights(question_tokens: Sequence[str], corpus: Sequence[str]) -> dict[str, float]:
def rarity_weights(
question_tokens: Sequence[str],
corpus: Sequence[str],
*,
stems: frozenset[str] | None = None,
) -> dict[str, float]:
"""What one hit on each question token is worth, from the bundle alone.
`log(N / df)`: `N` concepts, and `df` the number of them bearing the token
@ -880,7 +920,7 @@ def rarity_weights(question_tokens: Sequence[str], corpus: Sequence[str]) -> dic
for text in corpus:
candidate_tokens = normalise(text)
for token in counts:
if any(tokens_match(token, other) for other in candidate_tokens):
if any(tokens_match(token, other, stems=stems) for other in candidate_tokens):
counts[token] += 1
return {
token: math.log(total / count) if count else math.log(total)
@ -931,6 +971,7 @@ def _overlap(
*,
cost_vocabulary: bool = False,
weights: Mapping[str, float] | None = None,
stems: frozenset[str] | None = None,
) -> float:
"""What the candidate text answers of the question.
@ -944,7 +985,7 @@ def _overlap(
return sum(
1 if weights is None else weights.get(token, 1.0)
for token in question_tokens
if any(tokens_match(token, other) for other in candidate_tokens)
if any(tokens_match(token, other, stems=stems) for other in candidate_tokens)
or (bridged and in_cost_vocabulary(token))
)
@ -978,6 +1019,7 @@ def document_scores(
profile: BundleProfile = DEFAULT_PROFILE,
cost_vocabulary: bool = False,
weights: Mapping[str, float] | None = None,
stems: frozenset[str] | None = None,
) -> dict[str, float]:
"""One score per top-level document, from the indexes and the paths alone.
@ -1030,6 +1072,7 @@ def document_scores(
concept_id.replace("/", " "),
cost_vocabulary=bridge,
weights=weights,
stems=stems,
),
)
for relative in indexes:
@ -1042,7 +1085,13 @@ def document_scores(
continue
record(
document,
_overlap(question_tokens, entry.label, cost_vocabulary=bridge, weights=weights),
_overlap(
question_tokens,
entry.label,
cost_vocabulary=bridge,
weights=weights,
stems=stems,
),
)
return {
document: totals[document] / units[document] ** DOCUMENT_PRIOR_EXPONENT
@ -1087,6 +1136,28 @@ RRF_K = 60
DEFAULT_TIE_SHARED_RANK = True
#: Round 10. `MIN_SHARED_PREFIX = 4` matches on four characters whether or not
#: they are a stem. Measured on the pinned 453-concept bundle, control first:
#: `bilateral` occurs 0 times by equality and matches 400 of 453 through
#: `bilag`; `standhaftig` 0 and 219 through `standard`; `undersjoisk` 0 and 172
#: through `under`. Requiring the shared prefix to occur as a token in the
#: bundle's own concepts takes the first to 0 and the second to 56 on the
#: default bundle (0 and 33 on Arm B) with every hit@8 row keeping rank 1 on
#: BOTH bundles.
#:
#: ON by measurement, not by taste, and the measurement is that the other three
#: candidates are not available: a longer floor (5-8), a coverage share
#: (0.5-0.8) and a floor for long words only (>= 8) each cost row 1 its rank on
#: the default bundle and the whole row on Arm B. Row 1 reaches its gold
#: document through `pris|sammenstilling` on the four characters `pris`, so the
#: over-match and the wanted match are one mechanism; only "is the prefix a
#: word" separates them.
#:
#: LIKE `--tie-shared-rank`, THIS MOVES A PAYLOAD WITH NO BUNDLE CHANGING. A
#: consumer pinned to the previous excerpt order needs `--no-stem-prefix`.
DEFAULT_STEM_PREFIX = True
def concept_scores(
concepts: Sequence[Concept],
question: str,
@ -1096,6 +1167,7 @@ def concept_scores(
weights: Mapping[str, float] | None = None,
lookup: bool = True,
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
stems: frozenset[str] | None = None,
) -> list[tuple[Concept, float, int]]:
"""Every concept, ordered best first, fused from three signals by RRF.
@ -1161,13 +1233,20 @@ def concept_scores(
titles[concept.concept_id],
cost_vocabulary=bridge,
weights=weights,
stems=stems,
)
)
for concept in concepts
},
{
concept.concept_id: float(
_overlap(question_tokens, concept.body, cost_vocabulary=bridge, weights=weights)
_overlap(
question_tokens,
concept.body,
cost_vocabulary=bridge,
weights=weights,
stems=stems,
)
)
for concept in concepts
},
@ -1212,8 +1291,10 @@ def concept_scores(
# was, at the price of one more pass over the same two fields.
else {
concept.concept_id: int(
_overlap(question_tokens, titles[concept.concept_id], cost_vocabulary=bridge)
+ _overlap(question_tokens, concept.body, cost_vocabulary=bridge)
_overlap(
question_tokens, titles[concept.concept_id], cost_vocabulary=bridge, stems=stems
)
+ _overlap(question_tokens, concept.body, cost_vocabulary=bridge, stems=stems)
)
for concept in concepts
}
@ -1489,6 +1570,7 @@ def build_payload(
rarity_weight: bool = False,
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
withheld_titles: bool = False,
stem_prefix: bool = DEFAULT_STEM_PREFIX,
) -> dict[str, object]:
"""One bundle plus one question, cut to one contract-conformant payload.
@ -1534,8 +1616,19 @@ def build_payload(
)
for concept_id in concept_ids
]
# The bundle's OWN vocabulary, and the reason the rule is a set rather than
# a threshold: `pris` is a word here and `bila` is not, which is what
# separates a Norwegian compound from four coincidental characters. One
# pass, over the same text the ranking reads.
stems = (
frozenset(token for text in searchable_text(concepts) for token in normalise(text))
if stem_prefix
else None
)
weights = (
rarity_weights(normalise(question), searchable_text(concepts)) if rarity_weight else None
rarity_weights(normalise(question), searchable_text(concepts), stems=stems)
if rarity_weight
else None
)
ranked = concept_scores(
concepts,
@ -1546,10 +1639,12 @@ def build_payload(
profile=profile,
cost_vocabulary=cost_vocabulary,
weights=weights,
stems=stems,
),
cost_vocabulary=cost_vocabulary,
weights=weights,
tie_shared_rank=tie_shared_rank,
stems=stems,
)
titles_by_id = {concept.concept_id: concept.title for concept in concepts}
matched = sum(1 for _, _, lexical in ranked if lexical > 0)
@ -1705,6 +1800,24 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
dest="tie_shared_rank",
help="The rule's explicit opt-out, reproducing the pre-2026-09-10 order",
)
parser.add_argument(
"--stem-prefix",
action="store_true",
default=DEFAULT_STEM_PREFIX,
help=(
"require a shared PREFIX to be a word the bundle uses, so four "
"coincidental characters no longer match. Measured on the pinned "
"453-concept bundle: `bilateral` occurs 0 times and reached 400 of "
"453 through `bilag`; with this it reaches 0, and every hit@8 row "
"keeps rank 1 on both bundles. ON since 2026-09-09"
),
)
parser.add_argument(
"--no-stem-prefix",
action="store_false",
dest="stem_prefix",
help="The rule's explicit opt-out, reproducing the pre-round-10 matcher",
)
parser.add_argument(
"--withheld-titles",
action="store_true",
@ -1750,6 +1863,7 @@ def main(argv: list[str] | None = None) -> int:
reserve_top_rank=args.reserve_top_rank,
rarity_weight=args.rarity_weight,
tie_shared_rank=args.tie_shared_rank,
stem_prefix=args.stem_prefix,
withheld_titles=args.withheld_titles,
)
except ConsumeError as error:

100
src/llm_ingestion_okf/propose.py
View file

@ -123,6 +123,18 @@ RULE_TABLE_GRID = "rule:table-grid"
#: the two compose in one order: Arm E decides how far a block extends, this
#: decides where it is cut inside.
RULE_SHEET_SECTION = "rule:sheet-section"
#: Round 10 only, and its axis is a fifth one. Arm C names SIZE, Arm D what the
#: DOCUMENT declared, Arm E what the CONVERTER emitted, D3 what the SHEET
#: labelled; this names what the AUTHOR set in bold where the container gave
#: them no heading style to declare with. `rtf` is the row that forced it -- it
#: has no heading style at all -- but the grammar is markdown, not `rtf`, and
#: the rule reaches every type the converter writes bold for.
#:
#: It is a RECOVERY, like Arm D, so it carries Arm D's gate: a document that
#: declares a heading of its own admits none of these. Measured over 47
#: readable documents, that gate is what takes the false-positive count to 0
#: of the 31 that declare.
RULE_BOLD_TITLE = "rule:bold-title"
RULE_NAMES = (
RULE_HEADING,
RULE_TABLE_BLOCK,
@ -131,6 +143,7 @@ RULE_NAMES = (
RULE_OUTLINE,
RULE_TABLE_GRID,
RULE_SHEET_SECTION,
RULE_BOLD_TITLE,
)
#: How many characters of context each side of a quote anchor carries. Enough
@ -246,6 +259,20 @@ _SHEET_SECTION_LABEL = re.compile(r"^\d+(?:[+./-]\d+)*$")
# A pipe that pandoc did not escape. Splitting a row on a bare `|` would cut a
# cell containing a literal pipe in half and misread the FIRST cell of the row
# after it, which is the only cell this rule judges.
#: A line that is ONE bold span and nothing else. The whole discriminator of
#: round 10's rule: bold inside a paragraph is `text **bold** text`, which is
#: not a whole line, so the anchors are what separate a title from emphasis.
#: Both markers are read because the converter writes `**` and a hand-authored
#: markdown document may carry `__`.
_BOLD_LINE = re.compile(r"^\s*(?:\*\*|__)(?P<title>\S.*?)(?:\*\*|__)\s*$")
#: What a bold line must NOT end in. The one parameter of three that carried a
#: measurement: over 47 readable documents it takes false-positive lines from
#: 9-12 down to 1-2, and the candidates it drops are a contract cover page's
#: sentence fragments (`er inngatt mellom:`, `Sted og dato:`). A heading names
#: a section; a fragment set in bold ends the way a sentence does.
_TERMINAL_PUNCTUATION = (".", ":", ",", ";", "!", "?")
_UNESCAPED_PIPE = re.compile(r"(?<!\\)\|")
@ -442,6 +469,40 @@ def declares_headings(candidates: Iterable[Candidate]) -> bool:
return any(candidate.rule == RULE_HEADING for candidate in candidates)
def _gate_bold_title(
marked: list[tuple[int, Candidate]], joined: set[int]
) -> tuple[list[tuple[int, Candidate]], set[int]]:
"""G1 for round 10's rule: a document that declares admits no bold titles.
The same principle `_gate_outline` carries and the same predicate,
`declares_headings`, so there is one definition of "this document declared
a heading of its own" and not two that can drift. Applied to `marked` --
BEFORE the orphan pass -- for the reason stated there: the second pass
closes each span at the NEXT mark, so removing a mark here lets the
preceding span reach through the text it used to open, where filtering the
finished entries would leave that text in no segment at all.
There is NO G2 half. Arm D's share clause exists because a recovered
outline can carry a document whose declarations are incidental; a single
bold line in a document that already declares is not that case, and the one
false positive measured over 47 documents is exactly it. Adding a share
threshold here would be a knob no measurement asks for.
`declares_headings` counts `RULE_HEADING` alone, so admitting bold titles
can never change what this gate -- or the outline gate below it -- decides.
"""
if not declares_headings(candidate for _, candidate in marked):
return marked, joined
kept: list[tuple[int, Candidate]] = []
remap: dict[int, int] = {}
for position, entry in enumerate(marked):
if entry[1].rule == RULE_BOLD_TITLE:
continue
remap[position] = len(kept)
kept.append(entry)
return kept, {remap[p] for p in joined if p in remap}
def _gate_outline(
marked: list[tuple[int, Candidate]], joined: set[int], end_of_text: int, length: int
) -> tuple[list[tuple[int, Candidate]], set[int]]:
@ -549,6 +610,7 @@ def find_candidates(
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
bold_title: bool = False,
) -> list[Candidate]:
"""Every boundary the mechanical rules propose, in document order.
@ -591,6 +653,14 @@ def find_candidates(
REMOVE members from a run, which is why it only ever rescues candidates and
never discards one the shipped rule kept.
`bold_title` is round 10's gate and it is OFF at False, where the branch is
not even evaluated. On, a line that is ONE bold span, does not end in
terminal punctuation and is not made only of stop words proposes a
boundary -- but ONLY in a document that declares no heading of its own.
It exists for `rtf`, whose container has no heading style at all, and it
reads markdown rather than `rtf`: the converter already writes the author's
bold title as `**...**` in the same output every office row produces.
`close_span_gaps` is OFF at False, where a mark removed after its
neighbour's span was closed takes that text out of the plan entirely. On,
a span runs to the next SURVIVING candidate's start and the last one runs
@ -752,6 +822,30 @@ def find_candidates(
)
continue
if bold_title:
bold = _BOLD_LINE.match(line)
if bold is not None:
title = bold.group("title")
# An inner marker means the line is two or more bold RUNS with
# text between them, not one title set in bold.
nested = "**" in title or "__" in title
fragment = title.rstrip().endswith(_TERMINAL_PUNCTUATION)
if not nested and not fragment and not _is_stop_word_only(title):
marked.append(
(
index,
Candidate(
title=title,
level=1,
number=None,
rule=RULE_BOLD_TITLE,
start=offsets[index],
end=end_of_text,
),
)
)
continue
atx = _ATX.match(line)
numbered = _NUMBERED.match(line)
if atx is None and numbered is None:
@ -788,6 +882,8 @@ def find_candidates(
)
)
if bold_title:
marked, joined = _gate_bold_title(marked, joined)
if outline_gate:
marked, joined = _gate_outline(marked, joined, end_of_text, len(text))
@ -1232,6 +1328,7 @@ def build_plan(
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
bold_title: bool = False,
) -> dict[str, Any]:
"""The artifact. Every entry PROPOSED, the plan itself never adjudicated."""
taken: set[str] = set()
@ -1249,6 +1346,7 @@ def build_plan(
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
bold_title=bold_title,
)
for candidate in subdivide(text, candidates, max_segment_chars):
entries.append(
@ -1326,6 +1424,7 @@ def run(
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
bold_title: bool = False,
pdf_headings: bool = False,
pdf_headings_reserve: bool = False,
ocr: bool = False,
@ -1402,6 +1501,7 @@ def run(
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
bold_title=bold_title,
)
# Nothing to propose is an OUTCOME, and it is not an artifact. An empty
# plan cannot be replayed -- `process_inbox` refuses one, because a plan