Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.
- Generated fixtures are regenerated from their generators. Their structural
counts are identical before and after: elements, images, rows, cells,
headings, bookmarks and the witness inventory's per-document totals. The
image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
each are replaced by a generic, repeatable --corpus PATH with no default.
Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
removed, along with its three twins and five tests. The mutation harness
loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
are removed, and 40 are neutralized. Dead links are removed, and no new
dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.
Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.
Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STRUCTURED_V1 derived `references` from every number-shaped token in a body.
A consumer measured 12 false references out of 12 on their corpus, ten of them
version numbers in titles read as document numbers. Measured again here before
changing anything, on two corpora, through `derive_document_structure` itself:
corpus A a consumer's normative bundles 2 561 docs -> 2 838 subjects
corpus B this repository's own docs/ 28 docs -> 559 subjects
The reported class reproduces, and two larger ones they did not report turn up:
702 of corpus A's subjects (24.7 %) are hexadecimal fragments of UUIDs read out
of `id="..."` attributes in embedded MathML, and corpus B lifts tokens out of
escape sequences in quoted source (`\n60` -> `N60`), licence identifiers
(`AGPL-3.0` -> `3.0`) and package pins.
A derived subject is now a bundle-local link target, or a number immediately
preceded by a cue from a closed set. Positive rather than a blacklist because
the data forces it: `V221` is a genuine document number in corpus A and
`V0.3.0` is a software version in corpus B, and they are the same token shape,
so only the words in front of them can tell them apart. The cue matches at a
word boundary (a Norwegian compound ending in `-klasse` otherwise satisfies the
cue `se`, which admitted 86 class designations) and the window is NFC-normalised
so a cue survives a decomposed filesystem. Fragment-only and brace-carrying link
targets go too: neither can name a concept, so neither is a pending pointer.
After: 2 838 -> 1 279 (A) and 559 -> 72 (B). Hand-classified against the
occurrence that actually passed the gate: 30 of 30 sampled genuine on A, 60 of
60 on B. Residual known falses: 9 of B's 72, all illustrative link targets in
prose about link syntax.
The prefix-resolution rule stays open, per the order's condition: of 2 589
documents, 2 562 carry a number and 0 of those are dotted, so unique-prefix
match has no data here to be defended against.
Cost stated rather than hidden: a corpus phrasing cross-references outside this
vocabulary derives nothing and must declare `references`. A missing reference is
visible to the reader; a false one is not.
Record: docs/plan/references-cue-rule.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATyA5Lx53N1rKzwMnuMoem
A bundle a consumer can only look things up in is a filing cabinet. This adds
the derivation half of what lets one REASON over it: per-document title,
number, parent and references, plus bundle-level resolution of every pointer.
Two rules carry the design. Every fact is marked DECLARED or DERIVED, because
an unmarked heuristic is worse than no heuristic -- a consumer cannot know when
to doubt it. And resolution is a PURE function of the whole document set rather
than a diff, which is what makes rebuild-from-scratch equal an incremental
update by construction: there is no diffing algorithm to prove correct.
An unresolved pointer is kept and reported, never dropped: while a bundle is
built up over several rounds, pointing at something not dropped yet is the
normal state, and the dangerous version of it is the one that leaves no trace.
Symmetrically, a parent our own grammar could never admit (4.2 -> 4, a bare
integer) is not emitted at all -- an unresolved list that never clears is one a
consumer learns to ignore.
45 new tests; suite 615 -> 660.
