open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(profiles): a profile may name a per-suffix renderer

Browse source 
Arm E, capability only. A profile MAY name a renderer per suffix; no
domain-aware renderer is written here, that stays a Non-Goal, and `_RENDERERS`
is empty on purpose so the emptiness reads as a decision rather than an
omission.

THE LAYERING IS THE DESIGN, not an implementation detail. `extract.py` is the
extraction registry and must not import the contract layer, or the dependency
runs backwards and the registry stops standing on its own. So `extract_text`
gains a keyword-only `renderer: Callable[[str], str] | None`, knowing nothing
about profiles, and `inbox.py` -- which already holds the profile at that call
site -- resolves a NAME to a function. A test asserts extract.py still contains
no reference to the profile layer, because that constraint is the whole reason
the parameter is shaped this way.

The renderer runs AFTER extraction, never instead of it, so it never has to
re-implement a reader and the two cannot drift. The default is identity, which
is what keeps the five byte-pinned goldens byte-pinned -- asserted per suffix
rather than once.

An unknown renderer NAME is refused rather than falling back to identity: a
silent fallback would produce a bundle that looks rendered and is not, which is
the failure mode this arm exists to make visible. That needed a registered code
(`unknown_renderer`) and its test -- slightly beyond the step's named files,
but the capability cannot ship without defining what an unknown name does.

`tests/test_profile.py`'s exact-field-set assertion went red, as the plan's risk
table predicted. Updated deliberately with the reason recorded: that assertion
exists so a field cannot arrive without someone deciding it should, and its red
run is the mechanism working.

Suite 917 -> 926. All five goldens byte-identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-02 14:19:18 +02:00
commit 55a09d6c8e
7 changed files with 213 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

19
src/llm_ingestion_okf/extract.py
View file

@ -331,7 +331,9 @@ _OPTIONAL_EXTRACTORS: dict[str, Callable[[bytes], str]] = {
}
def extract_text(filename: str, data: bytes) -> str:
def extract_text(
filename: str, data: bytes, *, renderer: Callable[[str], str] | None = None
) -> str:
"""Convert one dropped file's bytes to OKF concept text, dispatched by type.
`filename` supplies the extension (case-insensitive); `data` is the raw
@ -339,11 +341,24 @@ def extract_text(filename: str, data: bytes) -> str:
without the extra, and any unregistered extension, fail fast with a typed
:class:`ExtractionError`. Extracting a `pdf` also emits an
:class:`ExtractionWarning`: drawn content has no text to recover.
`renderer`, when given, is applied to the EXTRACTED TEXT before it is
returned -- after extraction, never instead of it, so a renderer never has
to re-implement a reader and the two cannot drift. It is a plain callable
rather than anything profile-shaped ON PURPOSE: this module is the
extraction registry and must not import the contract layer, or the
dependency would run backwards and the registry would stop standing on its
own. Resolving a profile's NAMED renderer to a function is the caller's
job, in the layer that already holds the profile.
The default is identity, which is what keeps every existing byte-pinned
golden byte-pinned.
"""
suffix = Path(filename).suffix.lower()
extractor = _CORE_EXTRACTORS.get(suffix) or _OPTIONAL_EXTRACTORS.get(suffix)
if extractor is not None:
return extractor(data)
text = extractor(data)
return renderer(text) if renderer is not None else text
if suffix in _UNPARSED_OPTIONAL_EXTENSIONS:
raise _extra_missing(suffix)
raise ExtractionError(