llm-ingestion-okf/docs/plan/office-intake.md

Office intake: the three decisions, and what measured them

Decision record for the intake work planned 2026-09-01. The plan itself is local-only; these three choices are not, because each of them changes something a consumer can observe.

1. The converter binary is vendored, not resolved from PATH

Decision: operator, 2026-09-01, after research.

pandoc converts docx, xlsx, pptx, odt and rtf to text, and it enters as a pinned wheel inside the existing [extract] extra rather than as a system binary the library shells out to.

The first design considered was the opposite — spawn a system pandoc from one allowlisted module — and three independent measurements retired it:

• The xlsx and pptx readers exist only from pandoc 3.8.3. Debian 12 ships 2.17.1.1 and Ubuntu 24.04 ships 3.1.3, so a PATH binary cannot deliver two of the five formats on current stable distributions. Those table rows would fail by environment rather than by design.
• The docx reader changed its emitted output in at least 22 of the 3.x releases, including one change shipped in 3.6.2 and reverted in 3.6.4. A byte-exact golden over an unpinned binary is a promise about someone else's machine.
• The ecosystem already made this decision: Quarto bundles pandoc to eliminate version-mismatch risk, and pypandoc-binary ships it as a platform wheel.

Two consequences follow, and both are load-bearing:

The single-runtime-dependency rule is unchanged. The wheel lives in the [extract] extra, which pyproject.toml already places outside that rule by construction. The default install still declares exactly one runtime dependency.

The model-free gate keeps subprocess in its alternation. Because the conversion goes through an imported package rather than a process this library spawns, src/llm_ingestion_okf/ needs no subprocess of its own and the gate stays exactly as it was. What the gate proves has narrowed and should be stated honestly: it proves no model vendor is reachable from src/, not that no process is spawned anywhere in the dependency tree.

One measured trap, recorded because it is invisible when it fires. pypandoc searches PATH before its own bundled binary and selects the highest version it finds. On a machine with a newer system pandoc, the vendored wheel is silently bypassed and every determinism guarantee with it. The binary must therefore be resolved explicitly and its version asserted against a frozen literal — a check that costs nothing and is the only thing standing between the pin and a bundle built by a converter nobody chose.

2. Judgement lives outside the run path, as versioned data

Decision: operator, 2026-09-01.

Segment boundaries are proposed by a stdlib, model-free tool; a tools/-side command raises those proposals into reasoned candidates; a human ratifies or corrects; the ratification is written to a versioned plan file, and the run path only ever reads that file. src/ remains model-free.

The measured shape of the loop matters more than the shape of the diagram:

• The verdict carries its dwell time. A ratified flag with no per-item time is unfalsifiable. The published case is 300 000+ insurance decisions averaging 1.2 seconds each, ~80 % of the appealed ones later overturned. Recording the time is what makes "adjudicated" a fact rather than a claim, and it is the same field that measures what adjudication costs.
• Rejecting is one keystroke; correcting is optional. Measured over 2 784 annotations: people flagged fewer errors when flagging required typing the correction. An interface that charges for disagreement manufactures the rubber-stamping its output then claims to have avoided.
• The model leg is off by default, so its value can be measured rather than assumed. Pre-annotation has been measured lowering a good annotator's accuracy from 98.1 % to 95.8 %.

3. A new profile, never a change to an existing one

Decision: operator, 2026-09-01.

A segmented bundle could not declare which upstream spec it targets: SEGMENTED_V1 names bundle_id, OKF_V0_2 names okf_version, and the two never intersect. Rather than add the key to SEGMENTED_V1, a sixth profile — SEGMENTED_OKF_V0_2 — carries both.

This follows the standing rule that upstream support is additive, expressed as a new profile and never as a migration. It also avoids resting on an unverified premise: extending SEGMENTED_V1 would have been safe only if nobody had adopted it, and "nobody has adopted it yet" is a claim about other repositories that this one cannot check.

What these three do not settle

The acceptance rubric these decisions are measured against (K1–K7) is owned at programme level, spans this library and its consumers, and is not closed here. This work produces numbers on K1–K5 with their denominators; the thresholds that turn a number into a verdict belong to whoever owns the rubric.

Two cross-repo decisions ratified after these three

The three above were this repository's own to make. The two below were not: they span this library and a consumer, so they were ratified at programme level on 2026-09-02 and are recorded here because a contract that lives only in a consumer's plan file is a contract this side cannot be held to. They enter the office-intake work as amendments to it, not as new scope.

4. Concept IDs are bundle-local; identity across bundles is a tuple

Decision: programme level, 2026-09-02.

A concept ID is bundle-local and stable. Identity across bundles is the tuple (bundle_id, concept_id) — never the concept ID alone. The producer already stamps bundle_id into every concept's frontmatter, so a consumer reads the whole tuple from a single document and needs the root index.md only as a fallback for an older bundle.

There is no cross-bundle link form in v0.2. A consumer that holds several bundles partitions per bundle; nothing in this library emits a syntax for pointing from inside one bundle into another, and adding one is out of scope until a consumer states the need.

The timing is the reason this is written down now rather than later. The moment a golden bundle carrying okf_version is byte-pinned, the IDs inside it stop being adjustable — retrofitting a different denotation afterwards would move a pinned golden, which is the one thing byte-pinning exists to prevent.

5. The adjudication state has a fixed wire form

Decision: programme level, 2026-09-02.

Segment concepts carry their adjudication state in a frontmatter key adjudication, whose value set is closed: proposed or adjudicated. A value outside that set is an error, not an extension point.

When the value is adjudicated, three keys are written alongside it:

Key	Form
adjudicated_by	the adjudicator's identifier
adjudicated_at	ISO 8601
adjudication_dwell_s	integer seconds

The state is projected into the index as the facet adjudication. All of it is written only under the new profile; SEGMENTED_V1 output stays byte-identical.

Two things make this a contract rather than a naming preference. First, the dwell time travels with the verdict: a ratified flag with no per-item time is unfalsifiable, and the same field is what makes adjudication throughput measurable at all. Second, the consumer side has a rule this side must not break — absence of the key means the state unknown (an older bundle), never absent. A producer that emitted the key inconsistently would collapse that distinction, and the consumer's ability to see an unratified concept and discount it is the whole point of surfacing the state.