open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(extract): resolve the vendored converter and refuse an unpinned version

Browse source 
`_pandoc.py` hands back a converter whose identity is known, or refuses.

The wheel is not enough on its own. pypandoc searches PATH before its own
bundled binary and keeps the highest version found, so on this host the
vendored 3.9 was silently bypassed for the system 3.10.2 -- measured a third
time before writing this. The resolver reads the installed package's own
`files/pandoc` path and asserts the reported version against a frozen
PANDOC_VERSION literal, raising `extractor_binary_version` naming both,
`extractor_binary_missing` when the wheel carries no binary, and
`extractor_extra_missing` when the extra is absent.

A mismatch is refused rather than used with a warning: extraction is
deterministic within a converter version and not across one, and a byte-pinned
fixture cannot tell "a different converter ran" from "we introduced a defect".

Two defects found by measuring rather than by the suite:

1. The first implementation asked `pypandoc.get_pandoc_version()`, which
   answers from a module global that `clean_pandocpath_cache()` does not reset.
   It therefore reported whichever binary was probed FIRST in the process --
   3.10.2 for the bundled 3.9 binary. The suite was green because nothing in it
   probed the host binary first. Now `_get_pandoc_version(path)` probes the
   argument, with no cache and no search in the way, and a regression test
   poisons the cache before resolving. Negative control: that test fails on the
   old mechanism.
2. The module docstring named the process-spawning API in prose, which is
   enough to fail the model-free gate -- the gate is a grep. Reworded. The gate
   now proves the narrower "no model vendor is reachable from src/", stated in
   the module rather than glossed.

os.environ is restored on both the success and the failure path, and a
pre-existing override is put back rather than deleted.

Suite 887 -> 895. mypy --strict clean (pypandoc joins the guard's
ignore_missing_imports override; every value it returns is coerced here).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-02 14:07:20 +02:00
commit b9372ad7e8
3 changed files with 305 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

7
pyproject.toml
View file

@ -103,6 +103,13 @@ python_version = "3.10"
module = ["llm_ingestion_guard", "llm_ingestion_guard.*"]
ignore_missing_imports = true
# `pypandoc` ships no py.typed marker either. Only `_pandoc.py` imports it, and
# every value it hands back is coerced to `str`/`Path` there before it reaches
# the rest of the package -- the same discipline as the guard adapter above.
[[tool.mypy.overrides]]
module = ["pypandoc", "pypandoc.*"]
ignore_missing_imports = true
# Install CHANNEL for the guard, which is not on a package index yet. It is
# uv-specific, and it reaches further than a dev-only setting: a consumer
# installing this package from git WITH UV picks the guard up from this tag