Kjell Tore Guttormsen
b6a8c8bd89
A concept named its source file by basename and, when segmented, carried a
`source_offset` into the text THIS LIBRARY extracted. Following that pointer
needed the corpus directory, the extractor and its exact transitive version --
none of which the bundle carries. Hand-walked on a real K2 concept: six steps,
four of them requiring knowledge from outside the bundle, to learn that a
requirement sits on pages 12-13 of a 20-page document.
The address is spec's: `sources: [{ resource, title }]`, where `resource` is
the dropped file's inbox-relative path (SPEC v0.2 5.1:303-306 -- "an absolute
URL, a bundle-relative path, or a path into a `references/` subdirectory").
The locator is ours, and it has to be: 5.1 has no field for a place within a
resource, and the pinned guard (1.3.0) rejects every route to putting one
inside a `sources` entry -- a non-allowlisted key by name, a nested flow list
as "scalar leaves only", and quoting as an unsupported form. So the locator is
top-level keys shaped like `source_offset`, and a path carrying a flow
terminator is refused fail-fast rather than mangled.
The unit table is built AT EXTRACTION, where the extracted text and the
original's structure are known to agree: pdf -> `source_pages` from
pdfplumber's own page numbers (a page that yielded no text does not renumber
the ones after it), xlsx -> `source_sheet` + `source_rows`, everything else ->
`source_lines`. `source_offset` stays.
Two measurements changed the design before it shipped. A `paragraphs` key for
docx would name a number the document does not have: `<w:p>` counts of
108/27/65/176/57 against converted-markdown lines of 75/33/67/144/63, not one
pair agreeing -- so the key is `source_lines` and says what it indexes. And an
empty spreadsheet row renders exactly like a table separator: the content-based
rule ate 8 empty rows on the K2 price sheet and reported its last row as 92
against a workbook that says 100. The separator is now found by position, and
`tomrad.xlsx` keeps that red.
One profile moves. `provenance` is a policy object, `None` everywhere but
`SEGMENTED_OKF_V0_2`; the other five shipped profiles are byte-identical.
K2 rebuilt from a frozen src copy: 629 concepts, 1108 files, name set identical,
0 ids moved, 479 files byte-identical, 629 changed and 0 lines removed anywhere.
629/629 now carry an address and a locator. New ref
`sha256-tree:665563a2f74423fcbcc8e4f0b0954ee73b73985ac0418de4f6987bd162a1f7c8`;
`2f82fcfe...` is stale. The pre-pass payload does not grow by one byte
(209 092 B before and after, 18 changed lines: the ref and eight per-concept
digests) -- because an excerpt carries the body, not the frontmatter, which is
also why the consumer still cannot cite "file X page 12" from a payload alone.
Report: docs/2026-09-08-proveniens-k2.md. 1339 tests, ruff and mypy clean.
Co-Authored-By: Claude <claude-opus-5>
420 lines
21 KiB
Markdown
420 lines
21 KiB
Markdown
# Provenance to the original: an address and a locator per format, measured on 629 concepts
|
||
|
||
**Date:** 2026-09-08 · **Order:** `20260908T104812Z-6143923560-from-.claude` ·
|
||
**Instrument:** `src/llm_ingestion_okf/{extract,inbox,profiles}.py` plus the
|
||
`SEGMENTED_OKF_V0_2` profile · **Corpus:** the same 43-document public tender
|
||
folder every K2 measurement in this repository runs on.
|
||
|
||
A consuming agent is expected to falsify a hypothesis against the ORIGINAL
|
||
document. Before this step it could not: a concept named its source file by
|
||
basename, digested it, and — when segmented — carried a `source_offset` that
|
||
indexes the text THIS LIBRARY extracted, not the document anybody wrote. The
|
||
corpus directory, the extractor and its exact version were all required to
|
||
follow that pointer, and none of the three is in the bundle.
|
||
|
||
The corpus is external and no document body from it appears here. Documents are
|
||
named by shape, and everything else is a count.
|
||
|
||
---
|
||
|
||
## 0. What IS measured, and what is NOT
|
||
|
||
**Measured.** The provenance keys present on all 629 K2 concepts before and
|
||
after, per source format with a denominator; the three questions the order asks
|
||
per format ("can a consumer find the file, verify it, open the right place?");
|
||
one hand-walked known-positive from `source_offset` back to a page of the
|
||
original, counted in steps; the guard's verdict on seven candidate `sources`
|
||
forms, known-positive first; a full K2 rebuild with the file-set,
|
||
byte-diff and content identity; the payload a consumer receives; and the
|
||
viewer's own route table.
|
||
|
||
**Known-positive, run first.** The rebuild reproduces the bundle this
|
||
repository has been measuring on all week before anything else is read from it:
|
||
`merged + coded rejections = 39 + 4 = 43 = N`, exit 0; **629 concepts**;
|
||
**1 108 files, and the file-set is identical name for name** — 0 concept ids
|
||
moved. The 479 files that are not concepts (478 `index.md` plus `log.md`) are
|
||
byte-identical to the previous bundle.
|
||
|
||
**NOT measured.**
|
||
|
||
- **That a locator is CORRECT for any document not in this corpus.** The page
|
||
numbers are the extractor's own page numbers and the sheet rows were checked
|
||
against the workbook XML; the line numbers for the converted formats are
|
||
lines of OUR extraction and are labelled as such (§ 4).
|
||
- **That any model reads the new keys.** Whether an agent handed a payload
|
||
actually opens the original is `portfolio-optimiser`'s measurement, not this
|
||
one — and it cannot be, because the payload does not carry these keys at all
|
||
today (§ 6).
|
||
- **The viewer's per-requirement anchor.** One route table and one JS bundle
|
||
were read (§ 7). Whether `?reference=` positions a requirement needs a
|
||
browser, and the app lazy-loads chunks this measurement did not fetch.
|
||
- **A `.doc`, `.zip` or `.smc` row.** Those three types are rejected before any
|
||
concept exists, so they have no provenance to carry and no denominator here.
|
||
|
||
---
|
||
|
||
## 1. Before: what a concept said, per format, with denominators
|
||
|
||
Measured on the bundle built at `56c1205` (`/tmp/okf-kravnummer-20260908/k2-new`),
|
||
which is what HEAD produced — the following commit `116d3e1` changed the
|
||
consumption side only.
|
||
|
||
```
|
||
python3 - <<'PY' # frontmatter key presence over every non-index .md
|
||
... # the script is /tmp/o3/measure1.py; it parses the line-oriented block
|
||
PY
|
||
```
|
||
|
||
| | concepts | `source_file` | `source_sha256` | `source_offset` | `sources` |
|
||
|---|---|---|---|---|---|
|
||
| `.pdf` (32 of 33 files) | 586 | 586 | 586 | 576 | **0** |
|
||
| `.docx` (5 of 5) | 41 | 41 | 41 | 40 | **0** |
|
||
| `.xlsx` (2 of 2) | 2 | 2 | 2 | 2 | **0** |
|
||
| `.doc` / `.zip` / `.smc` | 0 | — | — | — | — |
|
||
| **total** | **629** | 629 | 629 | 618 | **0** |
|
||
|
||
The three questions the order asks, per format:
|
||
|
||
| format | (a) find the original file? | (b) verify it? | (c) open the right PLACE? |
|
||
|---|---|---|---|
|
||
| `.pdf` | **no** — a bare basename, no directory and no URL | **yes** — `source_sha256` over the dropped bytes | **no** — `source_offset` is a character position in the joined page text; `_extract_pdf` joins pages with a blank line and drops empty ones, so the page number is gone |
|
||
| `.docx` | **no** — same | **yes** | **no** — offset only; no paragraph, no heading path |
|
||
| `.xlsx` | **no** — same | **yes** | **partly** — the sheet name survives inside the concept TITLE (`Prissammenstilling {#sheet-1}`), as prose; no row, and no key a consumer can read |
|
||
|
||
Column (b) is a yes with a caveat worth stating: the digest verifies a file you
|
||
have already found by other means. It cannot help you find it.
|
||
|
||
### The known-positive: one hand-walk from an offset back to a page
|
||
|
||
Taken on a real segmented concept — a fire-safety requirement, `source_offset:
|
||
[24966, 25869]`, `source_sha256: 973e9490…` — using a script that shares no
|
||
code with the bundle builder:
|
||
|
||
1. Know the corpus directory. **Not in the bundle.** (`log.md` names it once,
|
||
for the run, not per concept — and `log.md` is a run log, not a concept.)
|
||
2. Read the file's bytes; confirm `sha256` matches. ✔
|
||
3. Know that the extractor is `pdfplumber` behind `[extract]`. **Not in the
|
||
bundle.**
|
||
4. Know its transitive text pin, `pdfminer.six==20260107`. **Not in the
|
||
bundle**, and the extracted text is pinned to it by a frozen literal in
|
||
`tests/test_extract.py`.
|
||
5. Re-extract, slice `[24966:25869]`. The requirement text comes back. ✔
|
||
6. To learn the PAGE, re-open the PDF page by page, rebuild the same join, and
|
||
bisect the offset. **Pages 12–13.**
|
||
|
||
**Six steps, four of which need knowledge the bundle does not carry.** The
|
||
document has 20 pages; nothing in the concept says which two.
|
||
|
||
---
|
||
|
||
## 2. What the spec has, and what it does not
|
||
|
||
SPEC v0.2 § 5.1:303-306, verbatim:
|
||
|
||
> `resource`: REQUIRED within an entry. Names either a concrete artifact a
|
||
> consumer can follow (an absolute URL, a bundle-relative path, or a path
|
||
> into a `references/` subdirectory, §6) or a population or scope descriptor
|
||
> it cannot (for example `all queries in BigQuery project X`).
|
||
|
||
A dropped file's inbox-relative path is precisely "a bundle-relative path", so
|
||
the ADDRESS needs no key of ours. `docs/okf-nokkelinventar.md` had already
|
||
named that re-alignment for `source_file` (Del B, `sources[].resource` as a
|
||
bundle-relative or `references/` path, § 5.1:303-305 + § 6.3); this step
|
||
executes it.
|
||
|
||
§ 5.1 lists exactly six entry keys — `resource`, `id`, `title`, `author`,
|
||
`usage_count`, `last_modified` — and **none of them is a location within the
|
||
resource.** A page, a sheet row and a line have no spec key, at any level.
|
||
|
||
### Why the locator cannot live inside a `sources` entry
|
||
|
||
Not a preference — a measured gate. Against the pinned guard (1.3.0), with a
|
||
known-positive first:
|
||
|
||
```
|
||
.venv/bin/python -c 'from llm_ingestion_guard import okf; okf.parse_frontmatter(DOC)'
|
||
```
|
||
|
||
| candidate | verdict |
|
||
|---|---|
|
||
| `sources: [{ id: a, resource: fixture }]` (our Door A golden) | **PASS** — known-positive |
|
||
| `sources: [{ resource: a.pdf, title: A }]` | **PASS** |
|
||
| `sources: [{ resource: a.pdf, sha256: abc }]` | REJECT — `mapping key 'sha256' is not on the OKF mapping allowlist under 'sources'` |
|
||
| `sources: [{ resource: a.pdf, pages: [1, 2] }]` | REJECT — `a flow mapping admits scalar leaves only, not '['` |
|
||
| `sources: [{ resource: a.xlsx, sheet: Ark1, rows: [1, 9] }]` | REJECT — same |
|
||
| `sources: [{ resource: "Del II, Bilag.pdf" }]` | REJECT — `a quoted scalar inside a flow mapping is not a supported form` |
|
||
| `sources: [{ resource: Del II Bilag 3.3.1 - Brannkonsept.pdf }]` | **PASS** — spaces are fine unquoted |
|
||
|
||
Two of those rows decide the design. A locator inside the entry has **no
|
||
expressible form** that survives the gate, so it would produce bundles this
|
||
project emits and could never read back through Door C. And because quoting is
|
||
also refused, a path containing a flow terminator cannot be escaped — it is
|
||
refused fail-fast instead (`inbox_source_file_unaddressable`), which is this
|
||
library's standing rule of validation over repair.
|
||
|
||
**The spec deviation, in four parts.**
|
||
|
||
| | |
|
||
|---|---|
|
||
| **spec says** | `sources[].resource` is the address (§ 5.1:303-306); there is no field for a place within it |
|
||
| **we do** | write the address in spec form, and the locator as top-level keys of ours — `source_pages`, `source_sheet`, `source_rows`, `source_lines` — in the shape `source_offset` already had |
|
||
| **measurement** | the guard rejects every route to putting a locator inside the entry (table above), and § 5.1's six keys contain no locator to reuse |
|
||
| **re-align** | if upstream ever gives `sources` a location field, these four keys move into the entry and become a rename. Until then they are ours, named as ours, and they say what they index |
|
||
|
||
---
|
||
|
||
## 3. The rule
|
||
|
||
**A unit table is built AT EXTRACTION**, where the extracted text and the
|
||
original's structure are the only moment they are known to agree, and it is
|
||
stored as `(start offset, original unit number, scope)` triples. A locator is
|
||
then a lookup, never a guess about text whose page breaks are already gone.
|
||
|
||
- `.pdf` → **`source_pages: [from, to]`**, from `pdfplumber`'s own page numbers.
|
||
A page that yielded no text leaves the table without renumbering the pages
|
||
after it, so the number is the document's, not "the n-th page that produced
|
||
text".
|
||
- `.xlsx` → **`source_sheet: <name>` + `source_rows: [from, to]`**. The sheet
|
||
comes from the heading the converter writes per sheet; the row restarts at
|
||
every sheet and skips the separator line **by position, never by content**
|
||
(§ 4).
|
||
- `.docx` and every text type → **`source_lines: [from, to]`**, lines of the
|
||
extracted text (§ 4).
|
||
- `source_offset` **stays**, unchanged. It is what an existing consumer joins
|
||
on, and a locator that replaced it would break them in order to help them.
|
||
|
||
`sources` is written whether or not a locator can be computed: the address
|
||
answers "which document", the locator answers "where in it", and a consumer is
|
||
owed the first even when the second is unavailable.
|
||
|
||
**One profile moves.** `provenance` is a policy object on `BundleProfile`,
|
||
defaulted to `None`, and set on `SEGMENTED_OKF_V0_2` alone. `sources` is a v0.2
|
||
key so a v0.1 profile must not name it; `DEFAULT` and `STRICT_V1` state
|
||
contracts owned in other repositories; `OKF_V0_2` is Door A's, where `sources`
|
||
already comes from the manifest. Five shipped profiles are byte-identical, and
|
||
a test asserts the field set so a sixth cannot arrive unnoticed.
|
||
|
||
---
|
||
|
||
## 4. Two things the measurement CHANGED before they shipped
|
||
|
||
Both were found by running the rule against the real corpus, and both would
|
||
have shipped a confident wrong number.
|
||
|
||
**A `paragraphs` key for `.docx` would have named a number the document does
|
||
not have.** The order asks for a paragraph or heading index. Measured on the
|
||
five K2 `.docx` files — `<w:p>` elements against converted-markdown lines:
|
||
|
||
| document | `<w:p>` | md lines | headings |
|
||
|---|---|---|---|
|
||
| 1 | 108 | 75 | 2 |
|
||
| 2 | 27 | 33 | 0 |
|
||
| 3 | 65 | 67 | 4 |
|
||
| 4 | 176 | 144 | 2 |
|
||
| 5 | 57 | 63 | 0 |
|
||
|
||
Not one pair agrees, and the heading-path alternative is empty for 2 of the 5.
|
||
So the key is `source_lines` and it says what it indexes: lines of the text
|
||
this library extracted. For `md`/`txt` that text IS the dropped file, and the
|
||
number is the original's own line.
|
||
|
||
**An empty spreadsheet row renders exactly like a table separator.** The first
|
||
implementation skipped the separator by matching the LINE (`^\|[-|: ]+\|$`).
|
||
On the K2 price sheet that rule also ate **8 empty rows**, and reported the
|
||
sheet's last row as **92** against a workbook that says **100** — every row
|
||
after the first blank one shifted, silently, for the whole sheet. The rule is
|
||
now positional: the second table line of a sheet is the separator, and nothing
|
||
else is. A new hand-laid fixture (`tomrad.xlsx`, four rows with the third
|
||
empty) is what keeps that red.
|
||
|
||
After the fix, against the workbook XML:
|
||
|
||
| spreadsheet | locator rows | `<row>` elements | contiguous from 1 |
|
||
|---|---|---|---|
|
||
| K2 document list | 1–39 | 39 | yes |
|
||
| K2 price sheet | 1–100 | 100 | yes |
|
||
|
||
Exact, both. The honesty limit: this holds as far as one converted line per
|
||
`<row>` holds. A sheet whose XML omits a row entirely would be numbered from
|
||
the converted table, and nothing here can see that.
|
||
|
||
---
|
||
|
||
## 5. After: the rebuild
|
||
|
||
`okf build` over the same 43-document folder, same arguments
|
||
(`--ingested-at 2026-09-03T00:00:00Z --bundle-id k2-trinn1-20260903
|
||
--okf-version 0.2`), run from a frozen copy of `src/` on `PYTHONPATH` so no
|
||
later edit could reach a measurement in flight — the log's first line names
|
||
that path.
|
||
|
||
| | before | after |
|
||
|---|---|---|
|
||
| `merged + coded rejections == N` | `39 + 4 = 43` | `39 + 4 = 43`, exit 0 |
|
||
| concepts | 629 | **629** |
|
||
| files in the bundle | 1 108 | **1 108**, identical name set |
|
||
| concept ids that moved | — | **0** |
|
||
| files byte-identical | — | **479 of 1 108** (every `index.md`, plus `log.md`) |
|
||
| files changed | — | **629 — every concept, and nothing else** |
|
||
| lines REMOVED across all 629 | — | **0** |
|
||
|
||
The diff is additive by measurement, not by intention: every changed file gained
|
||
`sources` plus exactly one locator shape, and no file lost a line or changed a
|
||
body.
|
||
|
||
| added keys | concepts | matches |
|
||
|---|---|---|
|
||
| `sources`, `source_pages` | 586 | every `.pdf` concept |
|
||
| `sources`, `source_lines` | 41 | every `.docx` concept |
|
||
| `sources`, `source_sheet`, `source_rows` | 2 | every `.xlsx` concept |
|
||
|
||
**The three questions, after:**
|
||
|
||
| format | (a) find | (b) verify | (c) open the right place |
|
||
|---|---|---|---|
|
||
| `.pdf` | **yes** | **yes** | **yes** — `source_pages` |
|
||
| `.docx` | **yes** | **yes** | **partly** — `source_lines` locates the extraction, not the document |
|
||
| `.xlsx` | **yes** | **yes** | **yes** — `source_sheet` + `source_rows` |
|
||
|
||
**629 of 629 carry an address and a locator.** For `.pdf` and `.xlsx`, all
|
||
three columns are yes; for `.docx` the third is honest rather than yes.
|
||
|
||
**The known-positive, walked again.** The same fire-safety concept now reads
|
||
`source_pages: [12, 13]` — the two pages the six-step hand-walk in § 1 arrived
|
||
at, computed by a script that shares no code with the builder. The walk is now
|
||
zero steps and needs nothing the bundle does not carry.
|
||
|
||
### The new content identity
|
||
|
||
**`sha256-tree:665563a2f74423fcbcc8e4f0b0954ee73b73985ac0418de4f6987bd162a1f7c8`.**
|
||
|
||
The previous ref, `sha256-tree:2f82fcfe…`, is superseded and stale. It was
|
||
recomputed from the previous bundle in the same run, as the known-positive, and
|
||
it matched — an instrument that has not reproduced a known figure has not been
|
||
shown to count.
|
||
|
||
The bundle on disk grows **97 011 B, 1 943 406 → 2 040 417 (+4.99 %)**, which is
|
||
**154 B per concept** — one address line plus one locator line.
|
||
|
||
### The payload regression
|
||
|
||
The order asks whether the pre-pass payload's token count moves now that the
|
||
frontmatter is bigger. Measured, same question, same `k`, both bundles through
|
||
the same tool:
|
||
|
||
| | before | after |
|
||
|---|---|---|
|
||
| payload bytes | 209 092 | **209 092** |
|
||
| delivered excerpts | 8 | 8 |
|
||
| budget spent (`BUDGET_UNIT`) | 118 409 | 118 409 |
|
||
| concept ids delivered | — | identical |
|
||
| excerpt texts, `text_sha256`, `withheld`, `denominators` | — | identical |
|
||
| changed lines in the whole payload | — | **18** |
|
||
|
||
The 18 are the bundle `ref` and the eight per-concept `sha256` values. **The
|
||
payload does not grow by one byte**, and the reason is § 6's finding stated from
|
||
the other side: an excerpt carries the concept's BODY, so frontmatter growth is
|
||
invisible to it. The frontmatter got bigger and the thing a consumer is billed
|
||
for did not move.
|
||
|
||
There is no o200k figure here. `tiktoken` is not installed on this machine, so
|
||
the byte count and the payload digest are the measurement; repeating a published
|
||
token number would be quoting, not measuring.
|
||
|
||
---
|
||
|
||
## 6. The consumer side: measured, not fixed
|
||
|
||
`tools/okf_consume.py` builds one excerpt per delivered concept, and
|
||
`excerpt_for` names its members explicitly. Measured on a real payload:
|
||
|
||
```
|
||
python3 -c "import json; d=json.load(open('…/k2-new-default.json')); \
|
||
print(sorted({k for x in d['excerpts'] for k in x}))"
|
||
# -> ['adjudication', 'bundle_id', 'bundle_id_inherited', 'concept_id',
|
||
# 'rank', 'sha256', 'text', 'text_sha256', 'trust_tier']
|
||
```
|
||
|
||
**No. Nine members, and not one of them is a provenance field** — not
|
||
`source_file`, not `sources`, not a locator, not even the concept's `title`.
|
||
An agent holding this payload cannot say "file X, page 12". It can say
|
||
`concept_id`, which is a path INTO THE BUNDLE, so the provenance is one file
|
||
read away — but only for a consumer that also has the bundle.
|
||
|
||
What it would take: `excerpt_for` carries `sources` and the locator keys
|
||
through from the concept's frontmatter. That is a payload-shape change, the
|
||
consumption contract § 8 enumerates an excerpt's members, and the shape is
|
||
`portfolio-optimiser`'s to decide. **Not done here**, deliberately, and this
|
||
paragraph is the whole of the fix's cost.
|
||
|
||
---
|
||
|
||
## 7. The viewer anchor, for `vegnormal-okf`
|
||
|
||
`https://viewers.vegnorm.vegvesen.no/product/859990/nb` answers **200** with an
|
||
**863-byte SPA shell** — no content, so the page itself decides nothing. The
|
||
route table and the query keys come from the app's own bundle
|
||
(`/assets/index-D1Z6iWLz.js`, 3 407 504 B):
|
||
|
||
- The route is **`/product/:id/:languageCode?/:parameter?`**. Read in the app,
|
||
`:parameter` is compared against exactly two literals — `"selection"` and
|
||
`"preview"`. **It is not a per-requirement anchor.**
|
||
- The app parses a QUERY STRING into `filePath`, `organization`,
|
||
`publicationUrn`, `tenantId`, **`reference`**, `isPrint`, `theme`, and passes
|
||
them to its content API call. **`?reference=` is the only candidate** for a
|
||
per-requirement address.
|
||
- **0 occurrences of `scrollIntoView`** in that bundle, so there is no
|
||
fragment-anchor scroll in it. `location.hash` appears 3 times, all inside the
|
||
router.
|
||
|
||
**Denominator, stated:** ONE bundle file, fetched once. The app lazy-loads
|
||
further chunks (`_mapDeps([20, 21])`) that this measurement did not fetch, and
|
||
what `reference` does to the rendered document needs a browser. So: a
|
||
per-requirement URL is **plausible via `?reference=<value>`** and **not
|
||
demonstrated**. Either way the value is the producer's to choose, and
|
||
`sources[].resource` is where it belongs — `vegnormal-okf` owns `sources` for
|
||
those bundles.
|
||
|
||
---
|
||
|
||
## 8. Honesty limits
|
||
|
||
1. **`source_lines` is not a paragraph number**, and for `.docx`, `.pptx`,
|
||
`.odt`, `.rtf`, `.csv`, `.json` and `.html` it indexes our extraction rather
|
||
than the document. § 4 has the numbers that make that the honest answer
|
||
rather than a shortcut.
|
||
2. **A row number holds only while one converted line means one `<row>`.**
|
||
Verified exact on both K2 spreadsheets and all three fixtures, all
|
||
contiguous from row 1. A sheet with a genuinely missing row would number
|
||
from the converted table.
|
||
3. **The address is inbox-relative, not absolute.** It resolves against the
|
||
folder that was dropped, and the bundle still does not say where that folder
|
||
is. Naming an absolute path would leak an operator's filesystem into a
|
||
published bundle; naming a URL is available whenever the source HAS one, and
|
||
for a dropped file it does not.
|
||
4. **The original is still not in the bundle.** § 6.3's `references/`
|
||
convention would put it there; copying binaries into a bundle is a decision
|
||
about size and licensing that nobody has taken, and it is not this order's.
|
||
5. **The rebuild is one corpus.** Every number above is 43 documents of one
|
||
public tender.
|
||
6. **`sources` carries no `id`.** § 5.1 says it SHOULD be present when the body
|
||
cites the source, and no body here cites with a `[^id]` footnote — measured
|
||
0 across the corpus in `docs/okf-nokkelinventar.md`. When one does, the key
|
||
is available and unwritten.
|
||
|
||
---
|
||
|
||
## 9. Verification log
|
||
|
||
| claim | command |
|
||
|---|---|
|
||
| 629 concepts, per-format key counts, before and after | `python3 /tmp/o3/measure1.py <bundle>` |
|
||
| the file set and the byte diff | a `rglob` set comparison plus `read_bytes()` equality over 1 108 files |
|
||
| every diff is additive | `difflib.unified_diff` per concept; 0 removed lines over 629 files |
|
||
| the guard's verdict on seven `sources` forms | `okf.parse_frontmatter` on each, known-positive first |
|
||
| docx paragraphs ≠ markdown lines | `<w:p>` count from `word/document.xml` against `extract_text(...).split("\n")` |
|
||
| spreadsheet rows against the workbook | `<row>` elements from `xl/worksheets/sheet1.xml` against `source_units(...).covering(0, len(text))` |
|
||
| the known-positive page walk | an independent script re-extracting and bisecting page offsets: pages 12–13, matching the emitted `source_pages` |
|
||
| the viewer's route table | `curl` the SPA shell and its one JS asset; `path:"…"` and query-key extraction |
|
||
| the payload members | `json.load` over a shipped K2 payload |
|
||
| suite, lint, types | `pytest -q`, `ruff check src tests tools`, `ruff format --check`, `mypy --strict src` |
|