open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(propose,cli): typography as a reserve, and the two of our own numbers it took to measure it

Browse source 
K3 round 5. Three questions, three answers, and two of them correct a figure
this repository published.

RETRIEVAL FIRST, because it could have reversed a default. hit@8 over the six
questions on BOTH K2 bundles -- Arm B at 629 concepts and the shipped default
at 492 -- is 5 of 6 with ranks 1,1,1,1,1,- on each, so 0 of 6 rows lost. The
order's rule reverses `--unit-fold` at >= 2 of 6; it does not fire, and the
default stands. The gold sets shrink (49->26, 20->17, 43->36, 11->18) while
every rank holds at 1, which is the fold merging concepts rather than removing
a document from the top.

TWO PUBLISHED NUMBERS CORRECTED, both ours. The S7 candidate ranks 96 of 629
and 159 of 492 were measured with the cost vocabulary passed to
`concept_scores` and NOT to `document_scores`, while `build_payload` passes it
to both; scored the way the shipped payload scores it, the same concept is 10
of 629 and 19 of 492. And round 4 attributed its non-delivery to the default
move -- measured here, it is not delivered on the Arm B bundle either, for a
different reason (knapsack eviction at 68 046 bytes of a 120 000 budget, versus
`below_k`). That column had been inherited from round 3's own build, never
re-measured.

`--pdf-headings font-reserve`, OFF, and the hypothesis behind it is falsified
by its own condition rather than by a score: position 7, the one position the
flag exists for, has THREE outline runs, so the reserve is silent there at
every minimum. It changes 0 of 12 cells on the reference and reaches 4 of 39
corpus documents, none of them rated. Built anyway because it was authorised
and because the condition is now measured rather than assumed. The predicate
lives in one place (`propose.heading_reserve_applies`) and the door receives it
as a callable, like `gate`: a plan indexes the exact string it was proposed
against, so a reserve firing on one side only would make every document it
touches a coded rejection.

The `xlsx` re-reading is confirmed on the artifact -- 11 `rule:sheet-section`
units plus 1 `rule:table-block` ingress -- but the number alone makes the cell
worse (distance 1 -> 2), because the criterion counts that ingress as a table
that should have been merged. A hit needs both halves ratified, and the
reference is the operator's.

`--sheet-section-rows` as a default: three cells better and none worse on the
twelve positions, but the K2 control moves -- row 1's gold document splits 1 ->
12 concepts and its best concept ranks 2 instead of 1. Condition not met,
default not moved.

Default build byte-identical before and after (`diff -r`, 30 md files).
Suite 1441 -> 1449; three of the eight were red first.

Report: docs/2026-09-08-k3-runde5-hitat8-og-skriftakse.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 00:25:51 +02:00
commit b01492b7f5
10 changed files with 743 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

44
src/llm_ingestion_okf/cli.py
View file

@ -70,6 +70,7 @@ from __future__ import annotations
import argparse
import sys
import tempfile
from functools import partial
from pathlib import Path
from .corpus import LOG_NAME, CorpusReport, load_plans, measure
@ -77,6 +78,7 @@ from .errors import IngestError
from .inbox import walk_inbox
from .profiles import SEGMENTED_OKF_V0_2, STRUCTURED_V1, BundleProfile
from .propose import ProposerError
from .propose import heading_reserve_applies
from .propose import run as propose_run
__all__ = ["DEFAULT_STAMP", "build", "main", "measure"]
@ -150,6 +152,18 @@ DEFAULT_UNIT_FOLD = True
DEFAULT_PDF_HEADINGS = False
DEFAULT_OCR = False
#: Round 5's third value on the SAME axis, and off for a third reason: not
#: caution, not construction, but a reachable set with no reference in it.
#: The reserve reads typography only where the outline gate admits no run, so
#: over the 43-document corpus it can act on 4 of 39 readable documents -- and
#: on the twelve-position reference it changes NOTHING, because the only
#: positions where it fires are one PDF whose glyphs carry no ToUnicode
#: mapping and four office documents the PDF reader never touches. The
#: position it was built for numbers its own chapters, so the reserve is
#: silent there by construction. Measured in
#: `docs/2026-09-08-k3-runde5-hitat8-og-skriftakse.md`.
DEFAULT_PDF_HEADINGS_RESERVE = False
#: The timestamp written when the caller passes none, for the ingest stamp and
#: the proposal stamp alike. ONE constant: two independently-defaulted literals
#: drift, and the drift shows up only as two bundles differing in a field
@ -171,6 +185,7 @@ def _propose_plans(
sheet_section_rows: bool = False,
drop_wrapped_outline: bool = False,
pdf_headings: bool = False,
pdf_headings_reserve: bool = False,
ocr: bool = False,
) -> tuple[int, int, int]:
"""Propose a plan per dropped file. Returns (written, nothing, failed).
@ -203,6 +218,7 @@ def _propose_plans(
sheet_section_rows=sheet_section_rows,
drop_wrapped_outline=drop_wrapped_outline,
pdf_headings=pdf_headings,
pdf_headings_reserve=pdf_headings_reserve,
ocr=ocr,
)
except ProposerError as exc:
@ -234,6 +250,7 @@ def build(
sheet_section_rows: bool = False,
drop_wrapped_outline: bool = False,
pdf_headings: bool = DEFAULT_PDF_HEADINGS,
pdf_headings_reserve: bool = DEFAULT_PDF_HEADINGS_RESERVE,
ocr: bool = DEFAULT_OCR,
) -> CorpusReport:
"""Folder in, bundle out. The whole command, minus argument parsing.
@ -252,6 +269,13 @@ def build(
"""
if proposed_at is None:
proposed_at = ingested_at
# Bound to THIS run's outline minimum, once, so the proposer and the door
# cannot be handed two different thresholds for the same question.
reserve = (
partial(heading_reserve_applies, outline_run=outline_run)
if pdf_headings_reserve and not pdf_headings
else None
)
if not segments:
report = measure(
inbox,
@ -259,6 +283,7 @@ def build(
ingested_at=ingested_at,
profile=STRUCTURED_V1,
pdf_headings=pdf_headings,
heading_reserve=reserve,
ocr=ocr,
)
_write_log(bundle, report, profile=STRUCTURED_V1)
@ -292,6 +317,7 @@ def build(
sheet_section_rows=sheet_section_rows,
drop_wrapped_outline=drop_wrapped_outline,
pdf_headings=pdf_headings,
pdf_headings_reserve=pdf_headings_reserve,
ocr=ocr,
)
print(
@ -308,6 +334,7 @@ def build(
profile=SEGMENTED_OKF_V0_2,
root_frontmatter_values={"okf_version": okf_version, "bundle_id": bundle_id},
pdf_headings=pdf_headings,
heading_reserve=reserve,
ocr=ocr,
)
_write_log(bundle, report, profile=SEGMENTED_OKF_V0_2)
@ -507,8 +534,14 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
)
build_parser.add_argument(
"--pdf-headings",
choices=("none", "font"),
default="font" if DEFAULT_PDF_HEADINGS else "none",
choices=("none", "font", "font-reserve"),
default=(
"font"
if DEFAULT_PDF_HEADINGS
else "font-reserve"
if DEFAULT_PDF_HEADINGS_RESERVE
else "none"
),
help=(
"how a PDF's headings are recovered before any arm reads the text. "
"none (the default) is today's extraction: a PDF carries no heading "
@ -518,7 +551,11 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"heading in the same markdown the office path produces, so the "
"existing heading rule reads it and no PDF-only grammar exists. A "
"CONJUNCTION, measured: adding weight as a disjunct took precision "
"from 0.786 to 0.524"
"from 0.786 to 0.524. font-reserve is font applied ONLY to a "
"document whose own numbering the outline arm finds nothing of -- "
"typography as a second source where there is no first one, never "
"on top of one. Three values of one option, so no caller can ask "
"for two at once"
),
)
build_parser.add_argument(
@ -566,6 +603,7 @@ def main(argv: list[str] | None = None) -> int:
sheet_section_rows=args.sheet_section_rows,
drop_wrapped_outline=args.drop_wrapped_outline,
pdf_headings=args.pdf_headings == "font",
pdf_headings_reserve=args.pdf_headings == "font-reserve",
ocr=args.ocr,
)
except (IngestError, OSError, ValueError) as exc:

4
src/llm_ingestion_okf/corpus.py
View file

@ -36,7 +36,7 @@ import argparse
import json
import sys
import time
from collections.abc import Mapping
from collections.abc import Callable, Mapping
from dataclasses import dataclass, replace
from pathlib import Path
@ -281,6 +281,7 @@ def measure(
profile: BundleProfile = STRUCTURED_V1,
root_frontmatter_values: Mapping[str, str] | None = None,
pdf_headings: bool = False,
heading_reserve: Callable[[str], bool] | None = None,
ocr: bool = False,
) -> CorpusReport:
"""Run the corpus through the door and count what happened.
@ -304,6 +305,7 @@ def measure(
root_frontmatter_values=root_frontmatter_values,
segmentations=plans,
pdf_headings=pdf_headings,
heading_reserve=heading_reserve,
ocr=ocr,
)
elapsed = time.monotonic() - started

20
src/llm_ingestion_okf/inbox.py
View file

@ -743,6 +743,7 @@ def process_inbox(
segmentation: SegmentationPlan | None = None,
segmentations: Mapping[str, SegmentationPlan] | None = None,
pdf_headings: bool = False,
heading_reserve: Callable[[str], bool] | None = None,
ocr: bool = False,
) -> InboxResult:
"""Convert every file dropped in `inbox_dir` into an OKF concept.
@ -962,6 +963,23 @@ def process_inbox(
pdf_headings=pdf_headings,
ocr=ocr,
)
# The heading RESERVE, supplied as a predicate rather than decided
# here: the condition is the proposer's outline grammar, and the
# door does not own that grammar. A callable keeps the dependency
# pointing the way the layers do -- the same shape `gate` already
# has -- while guaranteeing the two sides ask ONE question. Both
# `text` and `units` move together, because a locator built from
# one rendering cannot address the other.
reading_fonts = pdf_headings
if heading_reserve is not None and not pdf_headings and heading_reserve(text):
reading_fonts = True
text = extract_text(
source_name(path),
source_bytes,
renderer=_resolve_renderer(profile, path.name),
pdf_headings=True,
ocr=ocr,
)
# Computed from the SAME text the plan's offsets index, so the
# locator and the offset can never disagree about which rendering
# they describe. `None` when the profile names no provenance:
@ -972,7 +990,7 @@ def process_inbox(
source_name(path),
source_bytes,
text,
pdf_headings=pdf_headings,
pdf_headings=reading_fonts,
ocr=ocr,
)
if profile.provenance is not None

29
src/llm_ingestion_okf/propose.py
View file

@ -310,6 +310,26 @@ def outline_lines(text: str) -> list[tuple[int, int, str]]:
return found
def heading_reserve_applies(text: str, *, outline_run: int) -> bool:
"""Whether this text needs a SECOND heading source, having no run of its own.
The font reader's reserve condition, and the only place it is decided. The
proposer and the door both call this, because a plan indexes the exact
string it was proposed against: a reserve that fired on one side and not
the other would make every document it touched a coded rejection.
It reads the gate AS CONFIGURED rather than a fixed minimum -- raising the
arm's threshold widens the reserve, which is the same document property
seen through the same threshold. At `outline_run` 0 the gate admits nothing
at all, so the reserve is unconditional; that combination is round 4's
"font instead of Arm D", measured at 1 of 8, and a caller reaching it gets
it deliberately.
"""
if outline_run <= 0:
return True
return not outline_runs(outline_lines(text), outline_run)
def outline_runs(
entries: list[tuple[int, int, str]], minimum: int
) -> list[list[tuple[int, int, str]]]:
@ -1119,6 +1139,7 @@ def run(
sheet_section_rows: bool = False,
drop_wrapped_outline: bool = False,
pdf_headings: bool = False,
pdf_headings_reserve: bool = False,
ocr: bool = False,
) -> int:
if max_segment_chars < 0:
@ -1164,6 +1185,14 @@ def run(
# against another is refused by `assert_plan_applies`, which is the
# right outcome and a confusing one to debug.
text = extract_text(source.name, source_bytes, pdf_headings=pdf_headings, ocr=ocr)
# The reserve, and the reason it re-extracts rather than post-processes:
# the font reader works on the PDF's glyph geometry, which the joined
# text no longer carries. Skipped outright when the font reader is
# already on -- `font` and `font-reserve` are two values of one option,
# never a pair to combine.
if pdf_headings_reserve and not pdf_headings:
if heading_reserve_applies(text, outline_run=outline_run):
text = extract_text(source.name, source_bytes, pdf_headings=True, ocr=ocr)
except IngestError as exc:
raise ProposerError(f"cannot extract text from {source.name}: {exc}") from exc