open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(plan): inherit the external negative finding on edge inference

Browse source 
A deep-research report arrived mid-session (.claude broadcast 20260826T223230Z)
on a near-identical problem shape -- numbered regulatory requirements with
cross-references. Two of its numbers bear on this work: index-selection
strategy contributed +38.0 points of accuracy, and edge inference gave NO
accuracy gain at 2.8x the cost.

The second is a negative finding worth inheriting rather than re-measuring. It
does not condemn what landed today: references and parent are EXTRACTED from
explicit tokens, and the one proposed relation is structural and costs a single
pass. It draws a line for later -- no semantic edge inference without measuring
that 2.8x against our own corpus first.

The report also states that no published source gives per-query token counts
for structured versus flat context, and none reports an indexed superseded-by
facet. That reframes today's 3.3x-6.4x cost dial: it is the tradeoff nobody has
published, which is a reason to measure it properly rather than to hide it.

Treated as a premise, not a result. An external number changes what is worth
trying next, not what this repo has proved.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-27 00:41:41 +02:00
commit 69001542b2
1 changed files with 36 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

36
docs/plan/structure-derivation.md
View file

@ -112,6 +112,42 @@ bytes to the O2 mapping and orphan every bundle this door has already written.
Which stamp Door B should write is a separate question from this order's, and
answering it here would have answered it silently. **Open.**
### D-S7. Structural edge inference only — never semantic
Added the same day, from external evidence that arrived mid-session
(`.claude` broadcast `20260826T223230Z`, deep-research report on the OKF track).
A multimodal regulatory-document system on a near-identical problem shape
(numbered requirements, cross-references, tables and figures spanning pages,
arXiv 2606.29399) reports two numbers that bear directly on this work:
- index-selection strategy contributed **+38.0 percentage points** of accuracy —
which is what a faceted index is;
- **edge inference gave no accuracy gain at 2.8x the cost**, and structural
edges (References, Specifies) dominated single-hop questions while semantic
edges only surfaced on composite multi-hop ones.
The second is a negative finding worth inheriting rather than re-measuring. It
does not condemn what is built here: `references` and `parent` are EXTRACTED
from explicit tokens rather than inferred, and the one proposed relation —
supersession from a shared number plus ordered versions — is structural and
costs a single pass. It does draw a line: **do not extend this module to
semantic edge inference**, and if anyone proposes it, the cost side of that
2.8x must be measured against our own corpus before any of it is built.
Treated as a premise, not a result. An external number is somebody else's
measurement; it changes what is worth trying next, not what this repo has
proved.
### D-S8. The cost dial is an original contribution, not only a cost
The same report states that no published source gives per-query token counts
for structured versus flat versus full-verbatim context, and that no source
reports an indexed superseded-by facet at all. The 3.3x–6.4x dial measured
below is therefore a number the literature does not have, and the `supersedes`
facet is not a re-implementation of a known technique. That reframes the cost
finding: it is the tradeoff nobody has published, which is a reason to measure
it properly on a real corpus rather than a reason to hide it.
## Verification
All commands are runnable and were run. Suite: 615 → 695.