Kjell Tore Guttormsen
cf65635648
Six decisions, each with the failure it prevents: confidence carried in the format, certainty stated per field, supersession never inferred from one document, the index as a projection rather than an append, facets as a new profile, and Door B's ownership stamp left deliberately open. Six findings reported as findings rather than as gaps, per the order: the cost of closing the metadata gap is 3.3x-6.4x the flat index and whether that keeps the OKF arm's cost advantage cannot be answered from this repo -- it needs their bake-off re-run on their corpus; a reference to a parent-level number does not resolve to a child, and the fix needs a real corpus before 'unique' can be defended; OKF_V0_2's verified/status/stale_after are still expressible and unwritten for the reason already recorded. Every number carries its denominator, and the supersedes 0/55 carries a known-positive control (50/55 on a corpus that has shared numbers) proving the query can find what it reported missing.
226 lines
11 KiB
Markdown
226 lines
11 KiB
Markdown
# Structure derivation: numbering, hierarchy, cross-references, supersession
|
||
|
||
Answers order `20260826T124341Z-181848459` (from `.claude`, 2026-08-26). The
|
||
premise it waited on — the Door B / Door A capability gap — was closed in
|
||
`dc9ea59`: the answer was title derivation, not a door change and not a profile
|
||
on Door B alone.
|
||
|
||
The operator's requirement, quoted rather than paraphrased:
|
||
|
||
> "Og akkurat dette MÅ llm-ingestion-okf kunne lage automatisk og så ideelt som
|
||
> mulig når dokumenter legges inn i en okf bundle innboks (som kan skje en
|
||
> eller flere ganger, altså additivt)"
|
||
|
||
where "akkurat dette" is the form `.claude` described: documents that are
|
||
numbered, hierarchical, cross-referencing, and versioned such that one edition
|
||
supersedes another.
|
||
|
||
## What was measured, and what the measurement does and does not say
|
||
|
||
ms-ai-architect ran a pre-registered bake-off on 2026-08-26 over 55 documents
|
||
and 40 gold questions. Every arm retrieved 40/40, so the measurement could not
|
||
separate the arms on retrieval quality. The only axis that separated them was
|
||
trap exposure: 18/20 for the OKF-index arm against 8/20 for a frontmatter
|
||
head-scan over the same content. The reason was measured independently on both
|
||
sides — the flat `DEFAULT` index carries title/date/status/supersedes 0 times
|
||
while the documents in the same bundle carry them 55/55/55/5.
|
||
|
||
That is evidence against `DEFAULT`'s **index**, not against OKF. The metadata is
|
||
in the bundle; the index throws it away.
|
||
|
||
## Decisions
|
||
|
||
### D-S1. Every derived fact is marked, in the format
|
||
|
||
`derived` is a flat flow list naming exactly the fields this library inferred.
|
||
A field present and absent from that list was stated by the producer. The
|
||
format carries it, so this was not a case where "the format cannot express
|
||
confidence" — a flat list of strings round-trips through this library's
|
||
line-oriented parser and through the guard's T2 grammar alike.
|
||
|
||
Rationale: an unmarked heuristic is worse than no heuristic, because the
|
||
consumer cannot know when to doubt it. With the marker, a consumer that trusts
|
||
nothing derived can still use everything declared.
|
||
|
||
### D-S2. Certainty is stated per field, not per document
|
||
|
||
| Field | Source | Certainty |
|
||
|---|---|---|
|
||
| `title` | producer's `title` key | declared |
|
||
| | leading `# ` heading | **derived** |
|
||
| | filename stem | **derived** |
|
||
| `number` | producer's `number` key | declared |
|
||
| | leading number token of filename, then of title | **derived** |
|
||
| `parent` | arithmetic on `number` (drop last dotted part) | inherits `number`'s |
|
||
| `references` | producer's `references` key | declared |
|
||
| | bundle-local link targets + number mentions in the body | **derived** |
|
||
| `supersedes` | producer's `supersedes` key | declared |
|
||
| | same number + ordered `version` | **derived** |
|
||
| `version`, `status`, `date` | producer only | declared |
|
||
|
||
The number grammar is alpha-prefixed (`N500`, `V720`, `R610.4`) or dotted
|
||
numeric (`4.2.1`). A **bare integer is deliberately not a number**: admitting
|
||
`12-things.md` or `2026-notes.md` would stamp a document number on most of a
|
||
second brain that never had one.
|
||
|
||
A parent our own grammar could never admit is not emitted at all (`4.2` would
|
||
drop to `4`). An unresolved list that never clears is one a consumer learns to
|
||
ignore.
|
||
|
||
### D-S3. Supersession is never inferred from one document
|
||
|
||
Whether A supersedes B is a fact about a pair. `derive_document_structure`
|
||
records only what the producer declared; the bundle-level resolver may propose
|
||
a chain from a shared number plus ordered versions, and every edge it proposes
|
||
is marked derived. A group whose members do not all carry a version proposes
|
||
**nothing** — that is exactly the case where a guess would be indistinguishable
|
||
from a fact, and supersession is the relation a consumer is most likely to act
|
||
on.
|
||
|
||
### D-S4. The index is a projection, recomputed from the whole bundle
|
||
|
||
This is the answer to the additive requirement, and it is a design decision
|
||
rather than an algorithm: `resolve_structure` is a pure function of the whole
|
||
document set, and Door B rewrites the managed region of the index from it every
|
||
round. Nothing is diffed, so there is no diffing algorithm to prove correct.
|
||
|
||
Three of the order's four additive requirements fall out by construction:
|
||
|
||
- **rebuild == incremental** — both are the same function of the same files;
|
||
- **idempotence** — the concept name is the identity, so re-dropping a document
|
||
replaces its entry rather than doubling it;
|
||
- **a round-3 document can supersede a round-1 document** — the round-1 entry is
|
||
rewritten, which an append-only index could never do.
|
||
|
||
The fourth is explicit: an unresolved pointer is rendered with a `?` suffix
|
||
rather than omitted. During build-up, pointing at something not dropped yet is
|
||
the normal state; the dangerous version of it is the one that leaves no trace.
|
||
|
||
### D-S5. Facets arrive as a new profile (`STRUCTURED_V1`), never on `DEFAULT`
|
||
|
||
`DEFAULT` states commons' ingest-spec §6 index layer. Changing its rendered
|
||
bytes from here would be this repo editing a contract it does not own (O2), and
|
||
it would churn every golden fixture Door B has written. `entry_pattern` **is**
|
||
`link_pattern` when a policy carries no facets, so `DEFAULT` and `STRICT_V1`
|
||
match the same lines and emit the same bytes; a test asserts Door B's output is
|
||
byte-identical with and without the new parameter.
|
||
|
||
### D-S6. Door B keeps writing the literal `generated: true`
|
||
|
||
Not routed through `profile.ownership.stamp(...)`, which would move `DEFAULT`'s
|
||
bytes to the O2 mapping and orphan every bundle this door has already written.
|
||
Which stamp Door B should write is a separate question from this order's, and
|
||
answering it here would have answered it silently. **Open.**
|
||
|
||
## Verification
|
||
|
||
All commands are runnable and were run. Suite: 615 → 695.
|
||
|
||
```
|
||
.venv/bin/python -m pytest -q # 695 passed
|
||
.venv/bin/ruff check . && .venv/bin/ruff format --check .
|
||
.venv/bin/mypy --strict src/
|
||
```
|
||
|
||
### Mutation testing, with proof the mutation was applied
|
||
|
||
Each run asserts the anchor was present and that exactly one `# MUTANT` marker
|
||
landed in the file before the suite was read; a run whose anchor was absent
|
||
reports "NO RESULT READ" rather than a green suite. The first attempt reverted
|
||
mutations with `git checkout --`, which discarded uncommitted work; the harness
|
||
now snapshots the file text instead.
|
||
|
||
| # | Mutation | Result |
|
||
|---|---|---|
|
||
| M1 | index entry drops its facet tail | 6 failed |
|
||
| M2 | `unresolved` always returns empty | 2 failed |
|
||
| M3 | title derivation not marked derived | 3 failed |
|
||
| M4 | resolver iterates unsorted | **survived — equivalent** |
|
||
| M5 | self-reference guard removed | 1 failed |
|
||
| M6 | unnamed facet key accepted | 1 failed |
|
||
| M7 | index append-only (never removes a managed line) | 3 failed |
|
||
| M8 | reprojection claims any managed line | 1 failed |
|
||
| M9 | derived block emitted unsorted | **survived — equivalent** |
|
||
| M9b | *both* sorts removed | 1 failed (after adding the ordering test) |
|
||
| M10 | unresolved marker dropped | 2 failed |
|
||
| M11 | facet validation skipped | 1 failed |
|
||
| M12 | derived title not threaded to the entry | 1 failed |
|
||
|
||
M4 and M9 are equivalent mutants, not test gaps: `edges` is sorted before it is
|
||
returned, and `documents` is populated from `sorted(glob(...))`, so the loop
|
||
order cannot reach the output. M9 nonetheless showed the ordering property was
|
||
pinned only as a side effect of a byte comparison, so a test that states it
|
||
directly was added; M9b confirms it kills the mutation that removes both sorts.
|
||
|
||
### Cost, with the denominator stated
|
||
|
||
Measured on a **synthetic** 55-document corpus shaped like the one the bake-off
|
||
described — numbered, hierarchical, cross-referencing, versioned. This is a
|
||
measurement of **this library's emission**, not a re-run of anyone's bake-off,
|
||
and it is synthetic precisely because a private consumer's documents do not
|
||
come into this repo.
|
||
|
||
Facet coverage in the index, 55 documents, same counting query both sides:
|
||
|
||
| Facet | `DEFAULT` | `STRUCTURED_V1` |
|
||
|---|---|---|
|
||
| number / parent / status / date / version / references / derived | 0/55 | 55/55 |
|
||
| supersedes | 0/55 | 0/55 (corpus has no shared numbers) |
|
||
|
||
The `supersedes` zero is a property of that corpus, not of the code. Known
|
||
positive control, same query and same counter, on a corpus where documents do
|
||
share a number: **50/55** — 5 chains of 11 versions give 10 edges each, which
|
||
is the arithmetic the code should produce.
|
||
|
||
Index size for the same 55 documents, by facet key set — the dial a consumer
|
||
sets:
|
||
|
||
| Facet keys | Index chars | vs flat |
|
||
|---|---|---|
|
||
| none (`DEFAULT`) | 1 540 | 1.00x |
|
||
| status, supersedes, derived | 5 122 | 3.33x |
|
||
| number, status, supersedes, derived | 6 002 | 3.90x |
|
||
| number, parent, status, date, supersedes, derived | 7 817 | 5.08x |
|
||
| all eight (`STRUCTURED_V1`) | 9 797 | 6.36x |
|
||
|
||
## What was not achieved, and what would be needed
|
||
|
||
Reported as the order asked — what was tried, what happened, what would have to
|
||
change — rather than as "not possible".
|
||
|
||
1. **The cost of closing the gap is 3.3x–6.4x the flat index.** The bake-off's
|
||
OKF arm won on cost (6 031 characters against 21 879 for the head-scan it
|
||
lost to on traps). Whether a faceted index keeps that advantage **cannot be
|
||
answered from here**: it needs their bake-off re-run against
|
||
`STRUCTURED_V1`, on their corpus, with their questions. We own the emission
|
||
measurement; the arm comparison is theirs. The facet key set is the dial,
|
||
and a three-key set is the cheapest configuration that still carries status
|
||
and supersession.
|
||
|
||
2. **A reference to a parent-level number does not resolve to a child.** In the
|
||
synthetic corpus, `N130` in the body of one document does not resolve to a
|
||
document numbered `N130.2`; it is reported unresolved (`N130?`). This is
|
||
honest but incomplete — the fix is a prefix-resolution rule
|
||
(`subject` resolves to the unique document whose number starts with it), and
|
||
it was left out because "unique" has to be defined against a real corpus
|
||
before it can be defended. Deferred, not forgotten.
|
||
|
||
3. **`OKF_V0_2`'s `verified` / `status` / `stale_after` are still declared and
|
||
never written.** The order asked whether the alignment doc's "expressible"
|
||
reading is still right. It is, and for the reason already recorded: their
|
||
v0.2 values are block lists of mappings, which this library's value model
|
||
cannot round-trip until D1's structured reader lands. `STRUCTURED_V1`'s
|
||
`status` is a **scalar** facet and is a different key in a different profile;
|
||
it is not that field arriving early.
|
||
|
||
4. **Door B's ownership stamp is still the literal `true`** (D-S6). Open, and
|
||
deliberately not decided inside this order.
|
||
|
||
5. **A faceted index cannot be judged against a directory**
|
||
(`entries_match_directory`) in the same call, because `STRUCTURED_V1` does
|
||
not set it. Not attempted; no consumer has asked.
|
||
|
||
6. **No profile is applied to Door C.** Import writes concepts verbatim and
|
||
derives nothing, unchanged. Deriving structure for content another sender
|
||
wrote would put our inference into their bytes; if that is wanted it needs a
|
||
decision about whose claim the frontmatter makes.
|