Kjell Tore Guttormsen
169dfdf550
The converter binary is vendored as a pinned wheel in the [extract] extra rather than resolved from PATH. Three measurements retired the alternative: the xlsx and pptx readers exist only from pandoc 3.8.3 while Debian 12 ships 2.17.1.1 and Ubuntu 24.04 ships 3.1.3; the docx reader changed its output in at least 22 of the 3.x releases; and Quarto and pypandoc-binary already made this call. Two consequences are recorded because they are easy to get wrong: the single-runtime-dependency rule is untouched, since the extra sits outside it by construction, and the model-free gate keeps subprocess in its alternation, since src/ imports a package instead of spawning a process. One trap is recorded because it is invisible when it fires: pypandoc searches PATH before its own bundled binary and takes the highest version it finds, so on a host with a newer system pandoc the vendored wheel is bypassed without a word and the determinism guarantee goes with it. Judgement lives in tools/, outside the run path, as versioned data. The verdict carries its dwell time, because a ratified flag with no per-item time cannot be falsified. Rejecting is one keystroke and correcting optional, because requiring a typed correction was measured to reduce flagging. The model leg is off by default so its value can be measured. A sixth profile carries both okf_version and bundle_id rather than extending SEGMENTED_V1, which would have rested on an unverifiable claim about whether another repository had adopted it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
94 lines
4.9 KiB
Markdown
94 lines
4.9 KiB
Markdown
# Office intake: the three decisions, and what measured them
|
||
|
||
Decision record for the intake work planned 2026-09-01. The plan itself is
|
||
local-only; these three choices are not, because each of them changes something
|
||
a consumer can observe.
|
||
|
||
## 1. The converter binary is vendored, not resolved from PATH
|
||
|
||
**Decision: operator, 2026-09-01, after research.**
|
||
|
||
`pandoc` converts `docx`, `xlsx`, `pptx`, `odt` and `rtf` to text, and it enters
|
||
as a **pinned wheel inside the existing `[extract]` extra** rather than as a
|
||
system binary the library shells out to.
|
||
|
||
The first design considered was the opposite — spawn a system `pandoc` from one
|
||
allowlisted module — and three independent measurements retired it:
|
||
|
||
- The `xlsx` and `pptx` readers exist only from pandoc **3.8.3**. Debian 12
|
||
ships 2.17.1.1 and Ubuntu 24.04 ships 3.1.3, so a `PATH` binary cannot deliver
|
||
two of the five formats on current stable distributions. Those table rows
|
||
would fail by environment rather than by design.
|
||
- The `docx` reader changed its emitted output in **at least 22 of the 3.x
|
||
releases**, including one change shipped in 3.6.2 and reverted in 3.6.4. A
|
||
byte-exact golden over an unpinned binary is a promise about someone else's
|
||
machine.
|
||
- The ecosystem already made this decision: Quarto bundles pandoc to eliminate
|
||
version-mismatch risk, and `pypandoc-binary` ships it as a platform wheel.
|
||
|
||
Two consequences follow, and both are load-bearing:
|
||
|
||
**The single-runtime-dependency rule is unchanged.** The wheel lives in the
|
||
`[extract]` extra, which `pyproject.toml` already places outside that rule by
|
||
construction. The default install still declares exactly one runtime dependency.
|
||
|
||
**The model-free gate keeps `subprocess` in its alternation.** Because the
|
||
conversion goes through an imported package rather than a process this library
|
||
spawns, `src/llm_ingestion_okf/` needs no `subprocess` of its own and the gate
|
||
stays exactly as it was. What the gate proves has narrowed and should be stated
|
||
honestly: it proves no model vendor is reachable from `src/`, not that no
|
||
process is spawned anywhere in the dependency tree.
|
||
|
||
**One measured trap, recorded because it is invisible when it fires.**
|
||
`pypandoc` searches `PATH` **before** its own bundled binary and selects the
|
||
highest version it finds. On a machine with a newer system pandoc, the vendored
|
||
wheel is silently bypassed and every determinism guarantee with it. The binary
|
||
must therefore be resolved explicitly and its version asserted against a frozen
|
||
literal — a check that costs nothing and is the only thing standing between the
|
||
pin and a bundle built by a converter nobody chose.
|
||
|
||
## 2. Judgement lives outside the run path, as versioned data
|
||
|
||
**Decision: operator, 2026-09-01.**
|
||
|
||
Segment boundaries are proposed by a stdlib, model-free tool; a `tools/`-side
|
||
command raises those proposals into reasoned candidates; a human ratifies or
|
||
corrects; the ratification is written to a versioned plan file, and the run path
|
||
only ever reads that file. `src/` remains model-free.
|
||
|
||
The measured shape of the loop matters more than the shape of the diagram:
|
||
|
||
- **The verdict carries its dwell time.** A ratified flag with no per-item time
|
||
is unfalsifiable. The published case is 300 000+ insurance decisions averaging
|
||
1.2 seconds each, ~80 % of the appealed ones later overturned. Recording the
|
||
time is what makes "adjudicated" a fact rather than a claim, and it is the
|
||
same field that measures what adjudication costs.
|
||
- **Rejecting is one keystroke; correcting is optional.** Measured over 2 784
|
||
annotations: people flagged *fewer* errors when flagging required typing the
|
||
correction. An interface that charges for disagreement manufactures the
|
||
rubber-stamping its output then claims to have avoided.
|
||
- **The model leg is off by default,** so its value can be measured rather than
|
||
assumed. Pre-annotation has been measured lowering a good annotator's accuracy
|
||
from 98.1 % to 95.8 %.
|
||
|
||
## 3. A new profile, never a change to an existing one
|
||
|
||
**Decision: operator, 2026-09-01.**
|
||
|
||
A segmented bundle could not declare which upstream spec it targets:
|
||
`SEGMENTED_V1` names `bundle_id`, `OKF_V0_2` names `okf_version`, and the two
|
||
never intersect. Rather than add the key to `SEGMENTED_V1`, a sixth profile —
|
||
`SEGMENTED_OKF_V0_2` — carries both.
|
||
|
||
This follows the standing rule that upstream support is additive, expressed as a
|
||
new profile and never as a migration. It also avoids resting on an unverified
|
||
premise: extending `SEGMENTED_V1` would have been safe only if nobody had
|
||
adopted it, and "nobody has adopted it yet" is a claim about other repositories
|
||
that this one cannot check.
|
||
|
||
## What these three do not settle
|
||
|
||
The acceptance rubric these decisions are measured against (K1–K7) is owned at
|
||
programme level, spans this library and its consumers, and is not closed here.
|
||
This work produces numbers on K1–K5 with their denominators; the thresholds that
|
||
turn a number into a verdict belong to whoever owns the rubric.
|