open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(k3): round 20, a directory every concept shares and a section whose body is its heading

Browse source 
The report for 5ce8efe/9f8a1bc (the known-positive regression round 19
introduced, repaired in consume) and f7fd0d4/edd3b70 (`--shell-parent`, off).
Deviations first: the base suite is 1 658 passed / 1 skipped; the premise's
token counts reproduce as 11 and 6, not 15 and 17; a consumer session shared
the hit@k instrument's payload paths (3 of 16 payloads were its own, caught by
`bundle_mismatch`) and rebuilt N200 during the session (controlled with three
exports on one bundle); 35 of 710 shells have no ancestor holding text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 05:38:35 +02:00
commit 4595dde1bb
1 changed files with 231 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

231
docs/2026-09-11-k3-runde20-delt-katalog-og-arvet-kontekst.md Normal file
View file

@ -0,0 +1,231 @@
# K3 round 20: a directory every concept shares, and a section whose body is its heading
**Date:** 2026-09-11 · **Base:** `7eeafb3` · **Commits:** `5ce8efe` (red),
`9f8a1bc`, `f7fd0d4` (red), `edd3b70` · **Measured from:** frozen `git
archive` exports of `9f8a1bc` and `edd3b70` (Python 3.11.14), never the
working tree and never the `okf` on PATH.
Two deliveries, in the order they were set. **A** repairs a ranking regression
round 19 introduced: once a one-document bundle named its directory after the
document, a question naming the document stopped finding the concept that
names it. **B** adds `okf build --shell-parent`, off by default: a concept
whose body is its heading alone points at the nearest ancestor that holds
text. A was measured and committed before B was built, so B's rows are
measured against a bundle where the known-positive already stands.
## 0. Where the measurement differs from the order, first
- **The base suite is 1 658 passed, 1 skipped, rc 0**, not "1 known red". Run
from a scratch working directory one more test fails
(`tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit`
opens `examples/...` relative to the working directory); every suite number
below is from the repository root.
- **The shell count's path moved.** The consumer's `build/ferdig/r761-2025` now
holds its own route's bundle (0 of 2 728 shells). The generic bundle the order
counted is its `-generisk` sibling: **710 of 2 761**, 22 M. N100, N200 and
N500: **0 of 446, 0 of 1 133, 0 of 270**, as stated.
- **The known-positive's token counts**, with the shipped matcher (stem prefix
on) over the 2 761-concept bundle: `r761` in **0** titles and **2 761** titles
with the id; `prosesskoden` **11 → 2 761** (the injected premise said 15);
`standard` **6 → 6** (said 17 → 17). The mechanism stands; these are the
numbers.
- **Two things moved under this session, and both were measured before a
number was used.** (1) The hit@k instrument writes its payloads to fixed
paths, and a consumer session running the same instrument writes the same
paths. 3 of the 16 payloads first moved out were that session's: `okf check`
refused them with `bundle_mismatch` and the other session's bundle id. Every
payload below comes from a scratch copy of the instrument whose only change is
its payload path, and the instrument's own printed rows agree with it. (2)
The N200 bundle was rebuilt by its owner during the session: **1 137 of
1 137** files rewritten, files carrying U+00AD **116 → 0**. Digests from
before and after are not comparable. The control is the base, A and B
exports on the same current bundle: identical on **5 of 5**.
- **Acceptance B's "all 710 carry `parent`" cannot be met by this rule:**
**35 of 710** shells have no ancestor that holds text. **675 of 710** carry
it.
- **The premise correction holds.** The canonical `SPEC.md` has no `derived`
or `references` frontmatter key: `references/` is a directory convention
(§ 6.3, l. 479), and § 5.1 l. 343 reads "Lineage is expressed through links,
not a dedicated field." `derived:` in our bundles is this package's own key.
§ 6.1 l. 458-460: a link's kind, "parent/child" among them, "is conveyed by
the surrounding prose, not by the link itself" -- so `parent:` is an
extension under § 4.1, and not a field the spec defines.
## 1. A: a directory every concept id shares
**The mechanism.** `concept_scores`' first signal reads a concept's title
together with the segments of its id, and the overlap counts each question
token once. On a one-document bundle every id starts with the same directory.
When the question names the document, every concept answers those tokens
through the id, except the one whose title already carried them, which gains
nothing. The one concept distinguished by naming the document loses exactly
that distinction, and a concept answering nothing but the directory stops
being a guess the cut withholds. Measured on R761: the known-positive (KP)
went from rank 1 to not delivered at the default `k` and 13 at `k` = 50,
reproduced here on the base export with the instrument, S1-S6 unmoved.
**Four forms, measured on the base export with a scratch-patched
`consume.py`, every rank a delivered rank:**
| form | KP (default / `k` 50) | S1-S6 | K2 default, 6 payloads | K2 Arm B, 6 | N100/N200/N500, 15 | five-document bundle, 5 |
|---|---|---|---|---|---|---|
| off (control) | – / 13 | 6/6 | 6 of 6 identical | 6 of 6 | 15 of 15 | 5 of 5 |
| drop each concept's document directory | 1 / 1 | 6/6 | **0 of 6; row 6 rank 5 → not delivered** | **1 of 6; row 6 5 → 7** | 15 of 15 | 5 of 5 |
| drop the directories every id in the same document shares | 1 / 1 | 6/6 | **0 of 6; row 6 not delivered** | **1 of 6; row 6 → 7** | 15 of 15 | 5 of 5 |
| **drop the directories every id in the bundle shares (shipped)** | **1 / 1** | **6/6** | **6 of 6** | **6 of 6** | **15 of 15** | **5 of 5** |
The control reproduces the instrument's base row rank for rank and count for
count. **Across documents the document directory carries information**: the
two per-document forms cost a K2 row, so the shipped rule touches only the
prefix every id shares. On a one-document bundle that prefix is constant by
construction. Where ids share none, which covers every multi-document bundle
measured here, the signal reads the same string as before. `shared_id_prefix`
returns directory segments only, never the leaf. The third signal, the
document score, still reads the whole id, and on a one-document bundle it is
one value for every concept. Under `--rarity-weight` a token every concept
carries weighs `log(1) = 0`, which is the same answer.
**What the rule does not reach.** On a multi-document bundle whose question
names one document, the same saturation still happens inside that document.
The per-document forms would repair it, and they cost K2 row 6. No real
bundle of that shape was measured.
## 2. B: a section whose body is its heading
**The shape is faithful, not an extractor defect.** A process code states its
lettered points once, on the section that owns them, and every section nested
below inherits them. A nested section carries a title and nothing else, and
the XML reader is right to write it as a heading with nothing under it. The
directory tree is two levels deep (`<document>/<number>/<stem>`), so the
parent is not in the path either. `SegmentEntry.parent_id` already existed. It
was validated against the plan's own ids and written by the door as `parent:`,
but the proposer never set it: **0** `parent:` lines in the base bundle.
**Which ancestor.** The order left the route to measurement. Against the
document's own `<sec>` nesting, over the 710 shells:
| route | names the same ancestor as the nesting | where it misses |
|---|---|---|
| **plan level and order (shipped)** | **708 of 710** | `36.73211` and `36.73212` sit at depth 7, which a markdown heading clips to 6, and point at `36.732` instead of `36.7321`: still an ancestor, one level too high |
| section-number prefix | 686 of 710 | `12` begins with `1`; and an unnumbered document has no number to read |
9 sections sit at depth 7, and 6 of them are shells. The level route reads the
plan, so it serves any outline route and not only NISO-STS. The test holds the
same parents through the bookmark arm's route. **An empty ancestor is passed
over**, so the pointer lands on text in one step. 531 of the 675 point at
their direct parent. **A shell with no ancestor holding text gets none**,
which applies to 35. The rule runs after the plan is laid out and adds no
boundary.
**Three forms, one built:**
- **(i) Copy the ancestor's text in.** Measured by the consumer through its own
chain, not this package's (`vegnormal-okf`,
`docs/2026-09-11-r761-d1-n-veien.md` § 4a-4c, 5). Shells went to 0 of 2 728,
and inherited text was 93.6 % of the process bodies' bytes. Median file size
went up 7.8×, to 5 543 B, and the bundle from 22 M to 28 M. hit@1 went 6/6 →
2/6 and consumption 3 of 6. Read precisely: 3 of those 5 misses had their
answer in front matter the consumer's layout put in one concept, which is
not copy-in. In the other two, copy-in grew each excerpt until the budget
held 4-8, and the right process fell to withheld place 504 and 1 069.
- **(ii) Point, copy nothing. Built.** The existing `parent:` key names the
ancestor's `segment_id`, which every concept of the run carries.
- **(iii) Copy only the nearest ancestor's first lettered point.** Not built.
The one data point for it is the consumer's S6, where an inherited scope
line answered the question with the parent never delivered. Its size and
ranking cost are not measured here.
**Known cost, not repaired.** The index is a projection of the frontmatter,
and `structure` reads `parent` as a document NUMBER. A segment id answers to
none, so every such parent renders unresolved in the index (`parent: p1?`),
even though the concept it names is in the bundle. The same key already
carries both meanings for an adjudicator's declared parent (`inbox.py`).
**Default: off, by measurement.** `okf consume` reads no `parent` key: 0 lines
quote it, and the same query finds `title` on 3. So no payload ranks
differently, and the hit@k rows below are identical with and without the
flag. The flag moves the bytes of every bundle holding a heading-only section.
A consumer reaches the pointer only by opening the concept file, and the
payload does not carry it.
## 3. V1 FERDIG
**A** (`9f8a1bc`), R761 built with round 19's flags, hit@k by the instrument's
rule:
| row | base `7eeafb3` | after | denominator |
|---|---|---|---|
| KP rank, default `k` / `k` 50 | not delivered / 13 | **1 / 1** | 1 |
| hit@1 · 8 · 50, S1-S6, default `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| hit@1 · 8 · 50, S1-S6, `k` 50 | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KN (fabricated section) | no hit | **no hit**, same top-1 | 1 |
| delivered per question, default `k` | 7 8 8 8 8 8 8 7 | 7 8 7 8 8 8 7 7 | 8 |
| `okf check` | — | **16 of 16 payloads, 16 rules, 0 findings, rc 0** | 16 |
**B** (`edd3b70`), the same build with and without `--shell-parent`:
| row | flag off | flag on | denominator |
|---|---|---|---|
| shells | 710 | 710 | 2 761 |
| shells carrying `parent` | 0 | **675** | 710 |
| concepts with text carrying `parent` | 0 | **0** | 2 051 |
| `parent` = the `<sec>` nesting's ancestor | — | **708** | 710 |
| hit@1 · 8 · 50, S1-S6, both `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KP rank, both `k` | 1 / 1 | **1 / 1** | 1 |
| consumption, one `claude -p` per question | not re-run | **6 of 6 correct; answer first cited 4 of 6** | 6 |
| `okf check` | — | **16 of 16, 16 rules, 0 findings, rc 0** | 16 |
| `du -sh` / bytes of concept files | 22 M / 3 263 257 | **22 M / 3 272 360** (+9 103) | — |
| median concept file | 840 B | **840 B** | 2 761 |
| files differing | — | **1 350** (675 concepts + 675 directory indexes) | — |
The flag-off build is byte-identical to A's bundle (`diff -r`, 0 lines). In
the consumption runs, S4 cited the terminology section first and S6 cited
main process 88 above its answer. No run self-reported an extra pre-pass. The
generic bundle before this round scored 6 of 6 correct and 5 of 6 first-cited
in the consumer's own run. One draw each is not a measured difference.
## 4. Nothing else moved
| row | denominator | result |
|---|---|---|
| K2 pinned default bundle | 6 payloads | **identical 6 of 6** after A and after B; ranks `(1,1,1,1,1,5)` |
| K2 Arm B bundle | 6 payloads | **identical 6 of 6**; ranks `(1,1,1,1,1,5)` |
| N100 / N500 | 5 + 5 payloads | **identical 10 of 10** against the base |
| N200 | 5 payloads | identical 5 of 5 after A; after the owner's rebuild, **base = A = B on 5 of 5** |
| five-document bundle (round 17's known-negative bundle) | 5 payloads | **identical 5 of 5**; the known-negative rank 1, unchanged from round 17 |
| five-document folder, rebuilt | 52 md | `diff -r` differs only in `log.md`'s converter path; with `--shell-parent` **0 shells of 26, 0 files differ** |
| `tests/test_default_bundle_pin.py` | 7 tests | **7 passed** in both exports, gold set copied in |
| suite, working tree | — | 1 658 → **1 662** (A) → **1 667** (B) passed, 1 skipped, after `git add` and after commit |
| suite, frozen exports | — | 1 661 and 1 666 passed, 1 skipped, **1 failed** = the known export artefact `test_segmented_golden.py::test_the_four_existing_goldens_are_untouched` |
| gates | — | `ruff check src tests tools`, `ruff format --check .` (125 files), `mypy --strict src` (21 files): clean, rc captured directly |
| `okf check` rules | `len(RULES)` | **16**, `contract_check.py` untouched |
## Honesty limits
- **A's KP row is ONE question**, and the form was chosen over four candidates
with that question's answer visible. The choice rests on the K2 row the other
forms lose, not on KP, which all three repairs fix.
- **A changes the ranking on any bundle whose ids all share a leading
directory**, which is what a one-document build produces. The old order is
reproducible by no flag. Measured on 1 such bundle; the 7 other bundles
measured share no prefix and are byte-identical.
- **B is measured on ONE document with shells (N = 1).** "Every outline
document" is a claim about the code, which reads the plan's level and order;
the fixture test and the five-document folder are the only other rows, and
the folder has no shells at all.
- **The 2 of 710 wrong ancestors come from ATX's six levels**, not from the
rule. Carrying the unclipped depth would fix them and was not done.
- **Form (i) was measured by the consumer through its own chain; form (iii)
was not measured by anyone.** (ii) was chosen because it moves no ranking
and D1 measured (i) moving it down, not because (ii) was shown better for a
reader. With `consume` blind to the key, (ii)'s value to a reader is
unmeasured.
- **Consumption is one non-deterministic draw per question**, six runs on a
subscription (`ANTHROPIC_API_KEY` unset), judged against the fasit section
by reading the answer. The answer texts are not reproduced here.
- **Conformance is the floor and never the proof.** `okf check` is green on
all 32 payloads, and a payload whose right answer was not delivered would
pass it as well.
- **The instrument's shared paths and the N200 rebuild were caught by
controls,** a known-positive checker rule and three exports on one bundle.
Nothing guarantees that no other movement went unseen.