feat(extract): Door B extraction registry (Phase 2 step 1)

First, guard-independent step of Phase 2: a stdlib-only registry mapping a
dropped file's extension to its text extractor, with the fail-fast gates that
keep binary parsing out of core. `extract_text(filename, data)` dispatches
(case-insensitively) to:

- `md`/`txt` — utf-8-sig passthrough (BOM never leaks, baseline parity with
  Door A's read_csv);
- `csv` — the Phase 1 `render_table` (renderer reused, not duplicated);
- `json` — verbatim inside `render_fenced_block`;
- `html`/`htm` — text via `html.parser`, `script`/`style` stripped, tags as
  word boundaries (spec B3: adequate for v1, richer is out of scope).

`pdf`/`docx`/`xlsx` are `[extract]`-gated; until that extra ships a parser they
fail fast with a typed error naming the extra — never a silent skip, never a
bundled parser in core. New `ExtractionError(IngestError)` carries four stable
codes (`extractor_unknown`, `extractor_extra_missing`, `extractor_decode_error`,
`extractor_empty_csv`); a non-UTF-8 file is a typed corrupt-input failure, never
a leaked UnicodeDecodeError. `extract_text` returns text content only — LF
framing and concept frontmatter are the materializer's job (step 2).

No runtime dependency and no guard call yet (the guard pin and 0.4.0 land with
the persist gate in steps 4–5). TDD: test_extract.py + the four codes in the
test_error_codes.py registry precede the implementation; mypy --strict, ruff,
and the `sanitize|quarantine|lexicon` boundary grep-gate all clean; the Phase 1
golden suite still passes byte-for-byte.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
This commit is contained in:
Kjell Tore Guttormsen 2026-07-24 20:18:23 +02:00
commit db93de4aef
5 changed files with 301 additions and 0 deletions

View file

@ -19,6 +19,7 @@ import pytest
import llm_ingestion_okf.connectors as connectors
from llm_ingestion_okf.connectors import read_csv, read_http, read_sql, safe_resolve
from llm_ingestion_okf.errors import (
ExtractionError,
IngestError,
ManifestError,
MaterializationError,
@ -26,6 +27,7 @@ from llm_ingestion_okf.errors import (
RenderError,
SourceError,
)
from llm_ingestion_okf.extract import extract_text
from llm_ingestion_okf.manifest import load_manifest, load_manifest_bytes
from llm_ingestion_okf.materialize import materialize_bundle
from llm_ingestion_okf.render import sql_value_to_text
@ -291,6 +293,33 @@ def test_unsupported_cell_type(value: object) -> None:
assert code_of(excinfo) == "unsupported_cell_type"
# --- ExtractionError codes ---
def test_extractor_unknown() -> None:
with pytest.raises(ExtractionError) as excinfo:
extract_text("archive.zip", b"")
assert code_of(excinfo) == "extractor_unknown"
def test_extractor_extra_missing() -> None:
with pytest.raises(ExtractionError) as excinfo:
extract_text("doc.pdf", b"binary")
assert code_of(excinfo) == "extractor_extra_missing"
def test_extractor_decode_error() -> None:
with pytest.raises(ExtractionError) as excinfo:
extract_text("note.txt", b"\xffbad")
assert code_of(excinfo) == "extractor_decode_error"
def test_extractor_empty_csv() -> None:
with pytest.raises(ExtractionError) as excinfo:
extract_text("empty.csv", b"")
assert code_of(excinfo) == "extractor_empty_csv"
# --- MaterializationError codes ---