open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(k3): round 19, a document's own identity, per-run frontmatter, and a description from the source

Browse source 
The report for K3-19, measured from a frozen export of 77e7cab: R761 from the
base 56b00e7 against 77e7cab -- directory `14ce59dc-...` -> `r761-prosesskoden`,
2 761 concepts and 710 shells unchanged, 110 = 110, 1 807 descriptions, the
flagged edition and address on 2 761 of 2 761, hit@1/8/50 over S1-S6 6/6 at
both k, `okf check` 4 of 4 with 0 findings, and --ingested-at alone
confirmed on 2 761 of 2 761.

Reported first rather than repaired: the known-positive falls from rank 1 to
not delivered at the default k (13 at k = 50), decomposed to the directory
name alone; and the order's own `sources` value is refused by PyYAML on every
concept while `okf check` accepts it.

CHANGELOG under [Unreleased], the README and CLAUDE.md paragraphs carry the
final numbers, and the README test count moves to 1659 collected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 03:59:34 +02:00
commit 7eeafb35c8
4 changed files with 357 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

59
CHANGELOG.md
View file

@ -7,6 +7,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- **A NISO-STS document's own identity names its directory and titles its
`sources` entry (K3-19).** `okf build` put every concept of an STS delivery
under a directory named for the delivery file -- measured, a UUID occurring
0 times in the document -- while the document's one `<std-ident>` carried a
`<doc-number>`. `extract.declared_identity` reads exactly one `<std-ident>`
(`<doc-number>`, `<year>`) and exactly one `<title-wrap>`; a value stated
more than once is not read. The directory is the `<doc-number>` through the
id grammar, replacing only the file's stem, and `sources[0].title` is
`<doc-number>` + `<year>`, then the `<title-wrap>` title, then the file name
-- the first that can be written into the flow mapping verbatim (the
measured `<full>` carries commas, so it never is). A declared name two
documents in one run claim is used by neither, and stderr says so.
- **Consumer cost: a re-run, and an STS document's concept ids move**
(`<uuid>/...` -> `<doc-number>/...`). Every other file type is untouched:
the five-document folder rebuilds identical except the `log.md` line that
records the venv's converter path.
- **Measured side effect on ranking, reported rather than repaired:** hit@1
/ 8 / 50 over S1-S6 stays 6/6 at both `k`, but the known-positive falls
from rank 1 to not delivered at the default `k` (13 at `k` = 50).
`consume`'s first signal reads a concept id's segments, and on a
one-document bundle every id now carries the document's own name; renaming
only the directory back restores rank 1. `--rarity-weight` delivers it at
rank 4 with S1-S6 unmoved, and stays off.
### Added
- **`okf build --frontmatter KEY=VALUE`**, repeatable, stamps a key on every
concept of a run -- for what an operator knows and a document does not say,
such as an edition or a publisher's address. SPEC SS 4.1 "Extensions" lets
a producer add any key; SS 11 forbids a consumer to reject one. Split on the
FIRST `=`, and the value is written verbatim on ONE line, because this
package's readers are line-oriented and blind to a block-form `sources`.
Precedence: a stated value beats what the document declares, which beats the
file name. A run may add any key and REPLACE only `sources` and
`description`; every key the door writes itself -- including Door A's
`ingest_manifest`, which would make that door claim the file -- is refused
before a proposal is written (`run_frontmatter_invalid`), as is a value that
would not read back as stated. Reachable as `build(frontmatter=...)` and as a
keyword-only `concept_frontmatter_values` on `measure`, `process_inbox` and
`render_inbox_concept`. Without the flag nothing moves. **Note:** a
`sources` value carrying a URL or `X:Y` in the flow mapping passes `okf
check` and is refused by PyYAML's `safe_load` -- measured on 2 761 of 2 761
concepts with such a value -- and it is written verbatim as stated.
- **`description` for an STS section, from its own first spec point.** The
first `<p>` of the first direct-child `<sec sec-type="spec">`, whole,
carried by the plan entry, screened by the gate, and written only where a
YAML reader reads it verbatim (`inbox._yaml_plain`; over 2 024 measured
values the rule and PyYAML agree on every one). The spec sets no length, so
the one-paragraph limit is ours. On the measured document: 2 026 of 2 761
titled sections carry a spec point, 1 807 descriptions are written (2 points
have no `<p>`, 217 carry `": "`), none is invented, and none is derived from
a title.
`--ingested-at` alone was confirmed to stamp every concept, on the segmented
route too, and to date `log.md`: 2 761 of 2 761. Report:
[`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`](docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md).
## [0.8.2] — 2026-09-11
### Added

16
CLAUDE.md
View file

@ -132,9 +132,19 @@ one boundary rule:
R761's `<full>` carries a COMMA, a flow terminator, so it is never written
and never cleaned up. A titled section's `description` is its own FIRST
spec point (first `<p>` of the first DIRECT-child `sec-type="spec"`, whole),
carried by the plan entry and screened by the gate: 2 026 of 2 761 on R761,
none invented for the rest. SS 4.1 sets no length, so the one-paragraph
limit is ours. The registries are COUPLED: a row in
carried by the plan entry, screened by the gate, and written only where a
YAML reader reads it verbatim (`inbox._yaml_plain`): 2 026 of 2 761 titled
sections on R761 carry a point, **1 807** are written (2 have no `<p>`, 217
carry `: ` and PyYAML refused exactly those frontmatters), none invented.
SS 4.1 sets no length, so the one-paragraph limit is ours. **The directory
name reaches the RANKING:** `consume`'s signal 1 reads the concept id's
segments, so on a one-document bundle every concept now carries the
document's own name, and a question naming the document matches all of
them -- measured, the known-positive went rank 1 -> not delivered at the
default `k` (13 at k=50) with S1-S6 unmoved, and renaming only the
directory back restores rank 1. Operator question, not a patch. Report:
`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`. The
registries are COUPLED: a row in
`_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
every proposal for the type, two layers away from the extractor.
`pdf`/`docx`/`xlsx` only via

16
README.md
View file

@ -643,7 +643,10 @@ bundle:
declared name is not used and the file name stays. Each titled section's
`description` is its own first spec point — the first `<p>` of its first
direct-child `<sec sec-type="spec">`, whole — and a section with none gets
no `description` at all; nothing is derived from the title.
no `description` at all; nothing is derived from the title. A point a YAML
reader could not read verbatim (a `: ` inside it) is left out rather than
quoted or cleaned up. Measurements:
[`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`](docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md).
The drop directory is walked **recursively**, in sorted relative-path order:
a file at any depth is ingested and records its path relative to the inbox
root as its `source_file`, while dot-directories and a bundle directory
@ -855,15 +858,16 @@ From a checkout, the test suite runs with:
.venv/bin/python -m pytest
```
The suite is the verification surface for everything above: **1603 tests
collected, 1602 passed and 1 skipped**, run on 2026-09-11 with the `[extract]`
The suite is the verification surface for everything above: **1659 tests
collected, 1658 passed and 1 skipped**, run on 2026-09-11 with the `[extract]`
extra installed. Both numbers are given because they are two measurements: the
figure published before this round was the PASSED count, and `pytest
figure published before the `v0.8.2` release was the PASSED count, and `pytest
--collect-only -q` reported one more.
(The figure stood at 596 until 2026-09-09 — measured 2026-08-21 and never
updated as the suite grew — at 1515 until the `v0.8.0` release, at 1575
through it, at 1582 through the `v0.8.1` release, and at 1602 through the
`v0.8.2` release: a count is a measurement with a date on it.)
through it, at 1582 through the `v0.8.1` release, at 1602 through the
`v0.8.2` release, and at 1658 after K3-19: a count is a measurement with a
date on it.)
Without the extra the same suite skips the tests covering the parser path;
that split was last counted on 2026-08-21 as 589 passed and 7 skipped and has
**not** been re-measured since. The tests holding the fail-fast rejection for

275
docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md Normal file
View file

@ -0,0 +1,275 @@
# K3 round 19: a document's own identity, per-run frontmatter, and a description from the source
**Date:** 2026-09-11 · **Base:** `56b00e7` · **Commits:** `be169ee` (red),
`ee8d5b5`, `72617ac` (red), `912b850`, `0dbc331` (red), `de7849e`, `31ffc2c`
(red), `77e7cab` · **Measured from:** a frozen `git archive` export of
`77e7cab`, never the working tree and never the `okf` on PATH.
A NISO-STS document built with `okf build` landed every concept under a
directory named for its delivery file, and every `sources` entry named that
file twice. A consumer's own build of the same kind of document carried an
edition key, a publisher's address and a `description`, and `okf build` could
carry none of them. This round closes those four points in general code: no
name of a document, a publisher or a corpus appears in `src/`.
## 0. Where the measurement differs from the order, first
- **HEAD was `56b00e7`, not the `e15de71` the order recorded.** The R4 release
landed after the order was written. No commit since `e15de71` touched
`inbox.py`, `propose.py`, `cli.py`, `extract.py`, `profiles.py` or
`tests/test_xml_sts.py` (empty `git log`, rc 0), and `uv.lock` was clean, so
nothing had to be checked out.
- **The known-positive falls, and the directory name is the whole cause.**
S1–S6 hold at 6/6 · 6/6 · 6/6 at both `k`, as the order required. The
known-positive (KP) row went from **rank 1 at both `k`** on the base to **not
delivered at the default `k` and rank 13 at `k` = 50**. Decomposed in § 7:
the same bundle with only the directory renamed back to the file name puts KP
at rank 1 again, and the same bundle with every `description` removed does
not move it.
- **The order's own `--frontmatter 'sources=...'` value is not YAML that
PyYAML reads.** With it, **2 761 of 2 761** concepts' frontmatter fail
`yaml.safe_load` (a `:` inside a flow-mapping plain scalar, in the URL and in
`R761:2025`). The consumer bundle the order modelled it on has the same
property: **446 of 446** of its `sources` lines fail the same way. `okf
check` accepts both, with 0 findings. The flag writes the operator's value
verbatim, as ordered; this is reported, not repaired.
- **1 807 descriptions are written, not 2 026.** 2 026 titled sections carry a
direct-child spec point. 2 of those points have no `<p>`, and **217 carry
`": "`**, which a YAML reader refuses in `description: a: b`. The first build
wrote them and made 217 frontmatters unreadable to PyYAML. That was repaired
before delivery (`31ffc2c` red, `77e7cab`); § 5.
- **The 43-document reference corpus was not rebuilt.** It holds **0 `.xml` of
86 files** (known-positive in the same query: 66 `.pdf`), and every change
here is reached only through `.xml` or through a flag the build did not
receive. The pinned bundle's test runs as before (§ 8).
## 1. The spec, and what it does not say
Checked against the canonical text first (`docs/okf-nokkelinventar.md § 0.1`):
`curl` of upstream `okf/SPEC.md` exit 0, 37 748 bytes, `diff` against the local
copy **0 lines**, 1 006 lines, "Version 0.2".
- § 4.1, l. 205: "**Extensions:** Producers MAY include any additional keys.
Consumers SHOULD preserve unknown keys when round-tripping and MUST NOT
reject documents with unrecognized fields."
- § 11, l. 760: "Unknown additional frontmatter keys." among what a consumer
MUST NOT reject a bundle for.
- § 4.1 on `description`: "A single sentence summarizing the concept." **No
length limit exists** in § 4.1, § 8 or § 11 (`grep -n description` over the
spec is the denominator: 17 lines, none a limit). The one-paragraph limit in
§ 5 is this package's own.
Conformance is the floor and never the proof: every number below that says
"`okf check` 0 findings" is a statement about the contract checker, and § 0
already has one bundle it accepts that a YAML parser does not.
## 2. (a) The document's identity names its directory and titles its address
`extract.declared_identity(name, data)` reads what a NISO-STS document states
about itself: exactly one `<std-ident>` (`<doc-number>`, `<year>`) and exactly
one `<title-wrap>` (`<full>`, else `<main>`). It is `None` for every other file
type, for XML that is not STS, for an unparseable file and for a document
stating neither. A value stated more than once is not read: an adopted
standard carries one `<std-ident>` per issuing body, and taking the first is a
guess.
Measured on R761: **1** `<std-ident>`, **1** `<doc-number>` (`R761
Prosesskoden`), **1** `<title-wrap>`, `<year>` 2025. The delivery file's stem
occurs **0 times** in the document. `<doc-type>` reads `Innledning`, the name
of a chapter, and nothing uses it.
**The directory.** `okf build` replaces the file's stem with the
`<doc-number>` reduced to the id grammar and keeps the folders above it:
`r761-prosesskoden/`, where the base wrote the UUID.
**A declared name two documents claim is used by neither.** If a second
document in the run declares the same name, or its file name reduces to it,
both keep their file name, and stderr says which documents and which name. The
collision gate Door B already has would refuse both with "rename one", and a
name read from inside a document is not one a rename can change. Tested with
two documents sharing a `<doc-number>` and with a document whose file name
holds the declared name.
**The `sources` title.** The candidates are `<doc-number>` + `<year>`, then the
`<title-wrap>` title, then the file name. The first that passes the gate and
can be written into the flow mapping verbatim wins. The order left the choice
between the first two to measurement:
| `sources[0].title` | hit@k S1–S6, both `k` | KP (default / 50) | bytes spent over 8 questions (default / 50) | excerpts delivered at `k` = 50 |
|---|---|---|---|---|
| `R761 Prosesskoden 2025` (shipped) | 1,1,1,1,1,1 | – / 13 | 278 693 / 853 803 | 348 |
| `<full>` with its commas removed | 1,1,1,1,1,1 | – / 13 | 282 289 / 854 200 | 346 |
**They rank identically.** `consume` passes `sources` through to the excerpt
and ranks on neither. The full title costs 3 596 bytes more at the default `k`
and two excerpts at `k` = 50. The order's rule for a tie is the shorter one.
The deciding fact is structural anyway: R761's `<full>` carries **commas**,
which end a flow mapping, and the guard refuses the quoted scalar that could
carry them. It cannot be written verbatim, so the comma-free row exists only
in this measurement. A title is never cleaned up, because a title with its
commas removed is not the document's title.
`resource` stays the inbox-relative file. The address says where the bytes
are; the title says what they call themselves.
## 3. (b) `--frontmatter KEY=VALUE`
One form, repeatable, and a `--frontmatter-file` was not built. The consumer
bundle measured carries eight keys of its own, and eight flags are still
readable. A file would be a second parser for the same mapping.
- **Split on the FIRST `=`, and only there.** A publisher's address carries
`?languageCode=nb`.
- **Written verbatim on ONE line.** This package's readers are line-oriented,
and a block-form `sources` is invisible to `parse_frontmatter` (measured in
`vegnormal-okf` 2026-09-08 and cited by the order). The flow form is the only
one that survives our own readers. The test reads a stated `sources` back
through both `parse_frontmatter` and `consume.read_sources`.
- **Precedence: flag > what the document declares > file name.** A run may
ADD any key and REPLACE only `sources` and `description`, the two keys with a
layer below the flag. Tested on both layers. A stated `sources` gives exactly
one `sources` line, on the STS document and on a markdown one.
- **Refused before a proposal is written** (`run_frontmatter_invalid`): every
key the door writes itself (`inbox._door_keys`). That covers the hash, the
offsets, the locators, `type`, `ingested_at`, the bundle id, the ownership
stamp, the derived facets and `derived` itself. It also covers Door A's
`ingest_manifest`, which would make that door claim a Door B file. Also
refused: a key outside `[A-Za-z_][A-Za-z0-9_-]*`, and a value that is empty,
spans lines or has surrounding whitespace, since `parse_frontmatter` strips
it. A key given twice is refused rather than resolved.
- **Nothing moves without it.** A test holds the flagged tree to the plain
one minus exactly the stated line. `okf project` does not take the flag,
because it owns no flag that moves a bundle's bytes.
## 4. (c) and the YAML repair
The NISO-STS reader records, per titled `<sec>`, the **first `<p>` of the
first direct-child `<sec sec-type="spec">`**, whole. The plan entry carries it
beside `title`, because a plan is the record a rebuild replays. The door writes
it after the gate has seen it: it is document text persisted outside the body
the gate screens, so it is kept only on the non-blocking floor and only as the
sanitized text. A section with no such point gets no key. Nothing is derived
from the title.
Measured on R761: **2 026 of 2 761** titled sections carry a direct-child spec
point. 264 of those points hold more than one `<p>` and 2 hold none. The first
`<p>` runs **17 / 109 / 273 / 521 / 942** characters at min / median / p90 /
p99 / max. A character cap was not chosen, because a cut inside a paragraph
writes a sentence the source never wrote.
**The repair.** The first build from `de7849e` wrote 2 024 descriptions, and
PyYAML's `safe_load` refused **218 of 2 761** frontmatters: 217 on
`description`, all carrying `": "`, and 1 on a `title` the base already had.
`inbox._yaml_plain` is now checked where the value is written. It refuses a
leading YAML indicator, `": "`, `" #"`, a trailing `:`, a tab or a line break.
Over the 2 024 measured values the rule and PyYAML agree on **every one**: 217
refused, 0 refused that PyYAML reads, 0 kept that it does not. Omitted rather
than quoted, because a quoted value comes back from our line readers with its
quotes. After: **1 807** descriptions, and **1 of 2 761** frontmatters refused
by PyYAML, which is the base's own `title`.
That `title` class is older than this round and is not touched here: **41 of
436** concepts on the pinned 43-document bundle fail `safe_load` on `title`.
## 5. (d) `--ingested-at` alone: confirmed, nothing built
Built with `--ingested-at 2026-09-08T12:00:00Z` and no `--proposed-at`:
**2 761 of 2 761** concepts carry that stamp, on the segmented route where the
door reads the plan's `proposed_at`, and `log.md` is dated `## 2026-09-08`. The
base carried the same, so the `1970-01-01T00:00:00Z` in the consumer's bundle
came from a build that passed no stamp. The flag works as specified.
`tests/test_cli_build.py::test_ingested_at_alone_stamps_every_concept_the_same`
already held it.
## 6. V1 FERDIG — R761, base `56b00e7` against `77e7cab`
Built with `--bundle-id r761-prosesskoden-2025 --okf-version 0.2 --ingested-at
2026-09-08T12:00:00Z`. The flagged build adds `--frontmatter utgave=R761:2025`
and a `sources` value addressing the publisher's viewer.
| row | base | after | denominator |
|---|---|---|---|
| document directory | `14ce59dc-…` (the file's UUID) | **`r761-prosesskoden`** | 1 |
| concepts | 2 761 | **2 761** | — |
| conservation | 110 = 110 | **110 = 110** | N = 110 (109 images `extractor_unknown`, expected) |
| shells (body with no non-heading line) | 710 | **710** | 2 761 |
| `sources[0].title`, no flag | the file name | **`R761 Prosesskoden 2025`** | 2 761 |
| `sources`, flagged | — | the stated value, **one line** | 2 761 |
| `utgave`, flagged | — | **2 761** | 2 761 |
| `description` | 0 | **1 807** | 2 026 with a spec point |
| `ingested_at` = the stamp | 2 761 | **2 761** | 2 761 |
| frontmatter refused by PyYAML, no flag | 1 (`title`) | **1** (`title`) | 2 761 |
| hit@1 · hit@8 · hit@50, S1–S6, default `k` | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| hit@1 · hit@8 · hit@50, S1–S6, `k` = 50 | 6/6 · 6/6 · 6/6 | **6/6 · 6/6 · 6/6** | 6 |
| KP rank, default `k` / `k` = 50 | 1 / 1 | **not delivered / 13** | 1 |
| `okf check`, KP and S5 × both `k` | — | **4 of 4, 16 rules, 0 findings, exit 0** | 4 |
hit@k is vegnormal-okf's `experiments/hitk_sk2.py`, run read-only against the
scratch bundle with its five flags. Its payloads land at fixed
`/tmp/hitk-sk2-<id>.json` paths and were moved out between runs. The same
cells hold on the flagged and the unflagged build. On `de7849e`, `okf check`
was also run over all 8 default-`k` payloads: 8 of 8, 0 findings.
## 7. The known-positive, decomposed
The KP question contains the document's own number and name. `consume`'s
signal 1 is `title + concept_id.replace("/", " ")`, so once every concept id
starts with `r761-prosesskoden/`, every concept matches those two question
tokens, and the gold concept's distinguishing match on its own title
disappears into a 2 761-way tie. With the UUID directory, the id matched
nothing.
| variant (all from `de7849e`, unflagged) | KP default `k` | KP `k` = 50 | S1–S6 |
|---|---|---|---|
| as built | not delivered | 13 | all rank 1 |
| V1: every `description` line removed | not delivered | 13 | all rank 1 |
| V2: only the directory renamed back to the UUID (root index link rewritten to match) | **1** | **1** | all rank 1 |
| as built, `okf consume --rarity-weight` | **4** | **4** | all rank 1 |
| base `56b00e7`, `--rarity-weight` | 6 | 6 | all rank 1 |
`--rarity-weight` weighs a token by `log(N/df)`, and a token in every concept
weighs 0. It delivers KP at the default `k` on this bundle with S1–S6 unmoved.
It stays off: it is a ranking decision this round was fenced out of, and its
wider measurement (`docs/2026-09-08-sjeldenhetsvekt.md`) was mixed. **Which way
to go is the operator's.** The options are to keep the directory and repair
the id-segment signal for a segment every concept shares, or to accept the row.
## 8. Nothing else moved
- **Five-document folder.** 5 = 5, 52 md. `diff -r` against the base build
differs in **one line of one file**: the `Converter` line in `log.md`, which
records the absolute path of the venv that ran it, and the two exports sit
in different directories. With that line ignored (`diff -r -I`), rc 0. The
folder holds 0 `.xml`, so it is a real known-negative.
- **The four goldens.** `tests/test_segmented_golden.py` is green in the
working tree. In the export its fourth test is the one known red: it runs
`git status`, and an export has no repository.
- **The pinned 43-document bundle.** `tests/test_default_bundle_pin.py`,
**7 passed** in the export with `GOLD_SET` copied in.
- **Suite.** 1 602 → **1 658 passed**, 1 skipped (56 new). `ruff check src
tests tools`, `ruff format --check .` and `mypy --strict src` are clean, each
with its rc captured directly, after `git add` and after commit.
- Nothing was written in `vegnormal-okf`. Its `git status` is empty, and the
two `hitk_sk2` bytecode files in its ignored `__pycache__` date from
2026-09-10.
## Honesty limits
- **N = 1 STS document.** "General" is a claim about the code: no document,
publisher or corpus is named in `src/`, and every rule is keyed to a schema
element. It is not a measurement over several documents. The fixtures are
hand-written from the measured document's shapes.
- **The title choice rests on hit@k and bytes, not on what reads well**, and
hit@k could not separate the two candidates at all. The tie-break and the
flow-terminator fact decided it.
- **The one-paragraph `description` limit is ours.** The spec asks for "a
single sentence" and sets no length. 942 characters is not one sentence, and
it is what the source wrote.
- **The YAML rule is measured on 2 024 values from one document** against one
parser (PyYAML 6.0.3, YAML 1.1). A YAML 1.2 reader is not measured.
- **KP's fall is measured on one question.** Whether a question that names its
own single-document bundle is common is not measured.
- **`okf check` accepting a bundle is not a quality gate.** § 0 has a bundle
it accepts on 2 761 of 2 761 concepts that PyYAML refuses on the same count.
- **The 43-document corpus was reasoned out, not rebuilt:** 0 `.xml` of 86,
and the five-document folder is the measured known-negative in its place.