docs(plan): record the structure-derivation decisions, mutations and cost

Six decisions, each with the failure it prevents: confidence carried in the
format, certainty stated per field, supersession never inferred from one
document, the index as a projection rather than an append, facets as a new
profile, and Door B's ownership stamp left deliberately open.

Six findings reported as findings rather than as gaps, per the order: the cost
of closing the metadata gap is 3.3x-6.4x the flat index and whether that keeps
the OKF arm's cost advantage cannot be answered from this repo -- it needs
their bake-off re-run on their corpus; a reference to a parent-level number
does not resolve to a child, and the fix needs a real corpus before 'unique'
can be defended; OKF_V0_2's verified/status/stale_after are still expressible
and unwritten for the reason already recorded.

Every number carries its denominator, and the supersedes 0/55 carries a
known-positive control (50/55 on a corpus that has shared numbers) proving the
query can find what it reported missing.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-27 00:39:48 +02:00
commit cf65635648
3 changed files with 248 additions and 0 deletions

View file

@ -24,6 +24,14 @@ one boundary rule:
return the same wrong shape, so the breakage is document geometry, not a
library choice. PDFs enter as prose, and drawn content (figures) does not
survive extraction at all, which every `pdf` extraction warns about.
Under the `STRUCTURED_V1` profile Door B additionally DERIVES structure —
title (leading heading → `title` key → `path.stem`), document number,
hierarchy, and cross-references — writes it into the concept frontmatter, and
projects it into a faceted index entry. Every inferred field is named in a
`derived` list; an unmarked heuristic is worse than none. The index is a
PROJECTION recomputed from the whole bundle each round, which is what makes
rebuild-from-scratch equal an incremental update byte for byte. `DEFAULT` is
untouched and byte-identical. Record: `docs/plan/structure-derivation.md`.
- **Door C — external bundle import:** third-party OKF bundles are assessed
per concept via the guard's `okf.import_bundle`; only concepts clearing the
guard's non-blocking floor are merged/indexed here. Two invariants, both