K3-21 C. `resolve_structure` asks `_segment_lookup` first for a `parent`
edge: `(source_file, segment_id)` -> concept name, keyed off each concept's
own frontmatter (`DocumentStructure.declared`, no file read again), so a
pointer lands only inside the pointing concept's document -- `p1` exists in
every document of a multi-document bundle. A value no segment answers to is a
document number and is looked up exactly as before; a pointer naming nothing
keeps `UNRESOLVED_MARKER`. The rendering rule is untouched: a resolved
relation renders as its subject, so `parent: p1977?` becomes `parent: p1977`.
Moved on purpose, each named: both segmented goldens' index files
(`examples/ingest-golden-segmented{,-okf-v0-2}/expected-bundle/krav/1-{1,2}/
index.md`), whose declared parents s1 and s2 -> s0 rendered `parent: s0?`
while s0 stood in the bundle -- four lines, `?` removed. The four goldens
`test_the_four_existing_goldens_are_untouched` guards are not among them.
`skills/okf-consume/` regenerated, because the golden's index bytes -- and so
its ref -- moved. `test_shell_parent`'s byte test now expects the resolved
facet. README and CLAUDE.md no longer say the index renders it unresolved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-21 B. The second form of `parent`: `okf consume --follow-parent`
(`consume.attach_parent_text`) puts the enclosing concept's text inside an
excerpt's `parent`, with that concept's own `sha256` so a claim resting on it
is cited as that concept. It runs AFTER the cut, on the room the cut left, in
rank order, so the delivered set, its order, the withheld list and the
denominators are the same with the flag as without it -- inherited text
cannot displace an excerpt, the mechanism a consumer measured when copied-in
ancestor text pushed the right section to withheld place 504 and 1 069. A
text that does not fit is cut to the longest prefix that does and marked
`truncated`; a parent the payload already holds, or one a higher-ranked
excerpt already carried, travels once. OFF; the defaults are chosen on the
measurement that follows this commit.
`delivered_text` is the one normalisation an excerpt's `text` and a parent's
share. Contract SS 8 point 6 gains the MAY; the template tells the reader
what `text`, `sha256` and `truncated` mean. README and CLAUDE.md name the
flag.
Moved on purpose: the SS 7.4 known-positive again (14 455 / 14 083 / 372 ->
14 721 / 14 346 / 375), and `skills/okf-consume/` regenerated with it.
`tests/test_parent_text.py::test_no_room_means_no_text_and_no_lost_excerpt`
changed from its red form: it asked through `build_payload` at `limit ==
spent`, where the knapsack's 500 B buckets admit nothing at all
(`budget_admits_nothing`); it now holds the rule at `attach_parent_text`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-21 A. `okf consume` resolves a concept's `parent:` pointer -- a
`segment_id`, unique only inside one document's plan -- among the concepts
sharing its `source_file` (`consume.link_parents`, one pass, no file opened
again) and an excerpt carries `parent: { concept_id, title }`. Conditional
like `req_number`: a concept with no `parent` key moves no byte. A pointer
that lands nowhere is named `parent_unresolved: true`, never dropped.
The door writes ONE line into a heading-only body whose entry has a parent:
`Enclosing section: [<title>](/<bundle-relative path>)` (SPEC SS 5.1 lineage
through links, SS 6.1 the recommended absolute form and the kind in the
prose). Only such a body, so the segmented goldens' declared parents -- bodies
holding text -- are untouched. Appended AFTER structure derivation and
screened on its own (`_screened`, the `description` rule): read as body text
the link was derived into a second, unresolved `references` edge, measured on
the fixture. `segmentation.heading_only` is the one predicate the proposer and
the door share.
`okf check` gains its seventeenth rule, `parent_unfollowable`: a `parent`
that is not a concept_id and title, names its own excerpt, or names a concept
in neither `excerpts` nor `withheld` (together every considered concept).
Contract SS 8 point 6 added, the figure carries `parent`, and "additional
members are not read by the checker" now says the checker reads only the
members SS 8 names. The template tells the reader what `parent` is and that
SS 2.2 lets it read that one concept; `skill.CONDITIONAL_FIELDS` gains
`parent`. README and CLAUDE.md say what consume now reads.
Moved on purpose, each named: the SS 7.4 known-positive IS the contract
document, so `budget.known_positive` moves in every payload (13 238 / 12 893
/ 345 -> 14 455 / 14 083 / 372); `skills/okf-consume/` regenerated from the
segmented golden, whose plan declares s1 and s2 under s0 -- its example
payload now carries both parents; `test_bundle_identity` 16 -> 17 rules;
`test_shell_parent`'s byte test also accounts for the link line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-22's report, CHANGELOG under [Unreleased], and two corrections of a
sentence measured false.
README and CLAUDE.md both said of the flow form "both are valid YAML, and a
real YAML consumer recovers the same structure from either". An unquoted URL
with a query string inside a flow mapping is not valid YAML for PyYAML, and
the quoted form is refused by the pinned guard; the paragraph now states the
limit. CLAUDE.md also records the K3-22 invariant and the new refusal of a
`--frontmatter` flow value with such a leaf.
Measured, fix `ed0418f` against base `0308169`, each tree built twice from
frozen exports: five-document project 0 files moved; R761 XML and HTML 1 line
each; K2 default 42 `title` lines, after which 454 of 454 frontmatters parse
and read back the same (base 413 parse, 412 read back). hit@k on R761 base =
fix, S1-S6 6/6 at hit@1/8/50 at both k, KP rank 1; okf check 32 of 32 payloads
0 findings over 16 rules; K2 pin 7 passed in the export.
Found outside the order: the pinned guard refuses nearly every segmented
concept okf writes, on its scalar flow sequences (`source_offset`,
`references`, `derived`) -- identical before and after, so older than K3-22.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A plan entry whose span holds its heading alone gets `parent_id` naming the
nearest preceding entry at a smaller level whose own span holds text, passing
over an ancestor that is empty too; the door writes the existing `parent:`
key. Nothing is copied and no boundary moves. The rule reads the plan's level
and order, never the row. Off by default: `okf consume` reads no `parent`
key, so no payload ranks differently, while the flag moves the bytes of every
bundle holding a heading-only section.
Measured before building on one 2 761-concept process code: 710 concepts are
heading-only; the level route names the ancestor the document's own <sec>
nesting names on 708 of 710 (two sit at depth 7, clipped to 6, and point one
level too high), where reading section numbers gets 686 (`12` begins with
`1`); 35 have no ancestor holding text and get none.
The red test expected only concept files to move. The index is a projection
of the frontmatter and shows the key as a facet, so the test now holds both;
the facet renders a segment id as unresolved (`p1?`), because `structure`
reads `parent` as a document number -- named in README, CLAUDE.md and the
CHANGELOG, not repaired here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first fusion signal read a concept's title together with every segment
of its id. On a one-document bundle every id starts with the same directory,
and since K3-19 an STS document names that directory after its own number, so
a question naming the document matched every concept except the one whose
title already named it. Measured on a 2 761-concept bundle, the known-positive
fell from rank 1 to not delivered at the default k (13 at k = 50).
`shared_id_prefix` returns the leading directory segments EVERY id shares,
never the leaf, and the signal reads the id below them. Where the ids share no
prefix the signal reads the same string as before.
Measured on a frozen export before this commit, four forms: the chosen one
gives KP rank 1 at both k with S1-S6 6/6, and K2 (12 payloads), N100/N200/N500
(15) and the five-document folder (5) byte-identical. Dropping each concept's
own document directory instead took a K2 hit@8 row from rank 5 to not
delivered, and is not shipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report for K3-19, measured from a frozen export of 77e7cab: R761 from the
base 56b00e7 against 77e7cab -- directory `14ce59dc-...` -> `r761-prosesskoden`,
2 761 concepts and 710 shells unchanged, 110 = 110, 1 807 descriptions, the
flagged edition and address on 2 761 of 2 761, hit@1/8/50 over S1-S6 6/6 at
both k, `okf check` 4 of 4 with 0 findings, and --ingested-at alone
confirmed on 2 761 of 2 761.
Reported first rather than repaired: the known-positive falls from rank 1 to
not delivered at the default k (13 at k = 50), decomposed to the directory
name alone; and the order's own `sources` value is refused by PyYAML on every
concept while `okf check` accepts it.
CHANGELOG under [Unreleased], the README and CLAUDE.md paragraphs carry the
final numbers, and the README test count moves to 1659 collected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-19 c. The NISO-STS reader records, per titled <sec>, the FIRST <p> of its
FIRST direct-child <sec sec-type="spec"> as `OutlineMark.description`. The
plan entry carries it (`description`, only where the source has one, so every
other row's plan keeps its bytes), `parse_segmentation_plan` refuses an empty,
multi-line or non-string value, and the door writes it as the concept's
`description` after the gate has seen it: it is document text persisted
outside the body the gate screens, so it is kept only on the non-blocking
floor and as the sanitized text.
SPEC SS 4.1 makes `description` RECOMMENDED and sets no length, in SS 4.1,
SS 8 or SS 11. The limit is ours and structural -- one paragraph, whole --
because a cut inside it writes a sentence the source never wrote. Measured on
R761: 2 026 of 2 761 titled sections carry a direct-child spec point; the
first <p> runs 17 / 109 / 273 / 521 / 942 characters (min / median / p90 /
p99 / max). A section with none gets no key; nothing is derived from the
title. A stated `--frontmatter description=...` replaces it.
The extracted text does not move: the description is read beside it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-19 b. `okf build --frontmatter KEY=VALUE`, repeatable, split on the FIRST
'=' (`cli.frontmatter_from_flags`) because a publisher's address carries '='
itself. The value is written verbatim on ONE line: the block form of a
`sources` list is invisible to this package's line-oriented readers, so the
flow form is the only one that survives them. Also reachable as
`build(frontmatter=...)`, `measure(concept_frontmatter_values=...)`,
`process_inbox(concept_frontmatter_values=...)` and
`render_inbox_concept(concept_frontmatter_values=...)`, keyword-only with
defaults, so every existing call site is source-compatible.
Precedence: a stated value beats what the document declares, which beats the
file name. A run may ADD any key and REPLACE only `sources` and
`description` -- the two with a derived layer below the flag. Every other key
the door writes is refused by `inbox.validate_concept_frontmatter` before a
proposal is written (`run_frontmatter_invalid`): measured from the bytes, owned
by another argument, the ownership stamp a later run reads back (including
Door A's `ingest_manifest`, which would make that door claim a Door B file),
or a derived facet whose `derived` marker would go on naming a replaced value.
A value that would not read back as stated -- empty, multi-line, or padded,
since `parse_frontmatter` strips -- is refused too.
SPEC SS 4.1 "Extensions" lets a producer add any key and SS 11 forbids a
consumer to reject one. Without the flag nothing moves: a test holds the
flagged tree to the plain one minus exactly the stated line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-19 a. `extract.declared_identity` reads what a NISO-STS document states
about itself -- exactly one <std-ident> (<doc-number>, <year>) and exactly one
<title-wrap> (<full>, else <main>) -- and returns None for every other row,
for XML that is not STS, for an unparseable file, and for a document that
states neither. A value stated more than once is not read: an adopted
standard carries one <std-ident> per issuing body, and picking one is a guess.
`okf build` names a document's directory from its <doc-number> through the id
grammar, replacing only the file's stem. A declared name another document in
the run also claims falls back to the file name for both, said on stderr: the
existing collision gate would refuse both with "rename one", and a name read
from inside a document is not one a rename can change.
`sources[0].title` becomes <doc-number> + <year>, then the <title-wrap>
title, then the file name -- the first that survives the gate and can be
written into the flow mapping verbatim. Measured on R761, <full> carries a
comma, which ends a flow mapping, so it is never the title there; it is never
cleaned up either. `resource` stays the inbox-relative file.
Every other row, and every profile without an address, is untouched: the
identity is asked for only where `sources` is written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The hand-filled skill predated `okf skill`, declared no bundle identity,
and was refused by the checker it tells its reader to run, against the
payload shipped beside it:
NOT conformant: 16 rules over 3 excerpts and 0 withheld entries, 1 findings
bundle_mismatch: the skill declares no readable bundle identity, ...
It is now the generator's output for
examples/ingest-golden-segmented-okf-v0-2/expected-bundle, the bundle the
payload always came from:
conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings
The payload's bytes do not move. --force and --example-question are both
required: without the question the payload is a different one (spent
2125 -> 1420, delivered 3 -> 2, withheld 0 -> 1) and the byte test fails.
`okf skill` writes the bundle root and the skill path absolute when --out
is not under .claude/skills/, so the checkout prefix is stripped in one
mechanical step that a test holds; the generator is unchanged.
The frontmatter name becomes b-golden-segmented-okf-v0-2-consume. Claude
Code takes a project skill's command from its directory, which stays
okf-consume, and nothing in this repository named the skill by name.
references/README.md is rewritten by hand with the full regeneration
command; living prose in README, CLAUDE.md, the test comment and the
skill.py docstring is corrected, dated journals are left as written.
Report: docs/2026-09-11-k3-runde18-konsumskillen-regenerert.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`okf check` had fifteen rules and none asked whether the skill and the payload
were talking about the same bundle. Reproduced on this HEAD before any code
moved: three pairs reported `conformant: 15 rules over 8 excerpts and 438
withheld entries, 0 findings` -- a skill generated from one corpus against
another corpus's payload, the unfilled template against that payload, and a
payload sharing the skill's `bundle_id` at a foreign `ref`. All three now exit 1
with one `bundle_mismatch` finding over 16 rules.
BOTH halves are compared and the `ref` half is load-bearing: three distinct
builds on this machine carry one `bundle_id`, so an id comparison would pass a
stale skill. SS 3.3: "a version is the producer's assertion; a ref is a fact
about bytes". An identity the rule cannot read is a finding, never a silent
pass -- that is what refuses the unfilled template.
No new field: the identity was already in the generated skill's prose, now
factored into `skill.identity_line` and read back by
`contract_check.skill_identity`. Generated skill bytes unchanged, measured on
both tracked bundles on one interpreter.
The rule's first real find is this repository's own hand-made
`skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
identity a reader can act on: 1 of 1. Nine tests that asserted the old, false
conformance now pair a skill with its own bundle's payload.
Measured, nothing else moved: `~/okf-test/dokumenter` `diff -r` empty old
source vs new on one interpreter (52 files, 26 concepts), `okf project` still
byte-equal to `okf build`, K2 pin unmodified and green (453 concepts, ranks
1,1,1,1,1,5), known-negative `{}` unchanged at 9 findings.
Report: docs/2026-09-10-k3-runde15-bundle-mismatch.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report, with every measurement carrying its denominator and both
instrument forms side by side on every row -- including the proof that the two
forms cannot both reach 2 761, because 2 739 of 2 761 fasit titles carry a
numbering token and the two keys then demand mutually exclusive strings.
Also states two things round 13 left unsaid:
- XML that is not STS gives 0 plans and exit 2, and the `.txt` known-positive
shows it is general `okf build` behaviour for any structureless document,
not an `.xml` defect. Separating the two states would change the outcome on
0 of the 4 reference corpora, so the gate stays.
- Which two instruments were corrected in scratch and what the correction is.
With the shipped matcher the XML bundle scores 0/6 with `rang: null` on
every question, including the known-positive whose top excerpt IS the fasit.
Round 13's 14 two-file directories are re-classified on measurement: they were
false positives of the text route reading the document's own contents listing,
not the publisher reusing a title, and they are gone. Two remain, the same two
the PDF arm has.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GREEN on the twelve tests from the two commits before it. `.xml` joins
`_CORE_EXTRACTORS` beside `.html`, and 110 of 110 unreadable becomes a bundle.
A CORE type, not an `[extract]` one, because the parser is stdlib: putting it
behind the extra would make a pure-stdlib file type binary-dependent for no
gain. And never through the converter, which the existing fence test now says
in one more line: a file routed there is read by a second parser that never
sees this reader's DTD refusal, so the hardening would be true of the code and
false of the file.
TWO PATHS, NAMED RATHER THAN GUESSED. STS when the root is `<standard>` or the
document holds any `<sec>`; generic otherwise. Generic XML keeps its text in
document order and gets NO structure -- an element name is never promoted to a
heading, because an RSS feed is not a numbered standard.
THE GRAMMAR IS MARKDOWN, the same markdown the office rows and round 11's HTML
row reach the proposer through, and `propose.py` is untouched. `<sec>` with a
`<title>` becomes one ATX line carrying `<label>` + space + `<title>` at its
own nesting depth; `<sec>` with only a `<label>` becomes a body line with the
label in front, the way `li` is treated in HTML -- 4 954 of R761's 7 715 are
lettered points and one heading each would bury its own 2 761; `<table-wrap>`
becomes its label plus ONE markdown table through this package's own
`render_table`, separator line included, which is what makes it a block.
TWO CHOICES THAT ARE MEASUREMENTS, not preferences:
- Inline by ALLOWLIST, block by default -- the inverse of the HTML reader,
because block-versus-inline is a property of HTML and XML has no such
universal. Assuming block is safe (an extra break never removes text and a
boundary needs a line that matches a grammar); assuming inline is not. The
allowlist is load-bearing: that document carries 1 701 `<italic>` and
1 396 `<bold>` inside its prose.
- The ATX ceiling is 6 and STS nesting reaches 7. The depth is CLIPPED, not
dropped: 9 of the 2 761 titled sections sit at depth 7 and `#######` matches
nothing, so dropping loses the section while clipping keeps the boundary and
states the nesting one level too shallow.
A DTD IS REFUSED UNPARSED, and that is a guarantee about this code rather than
about the machine. Measured on this interpreter (3.14.0, pyexpat 2.7.3): an
external SYSTEM entity is refused by the stdlib and never fetched, but the
amplification limit that stops a billion-laughs comes from libexpat >= 2.4.0
and NOT from Python -- five levels still expand -- while `pyproject.toml`
requires only `>=3.10` and no lockfile pins an interpreter. `XMLParser` exposes
no `.parser` attribute on the C accelerator, so the handler route is not
portable either. NO new dependency: `defusedxml` and `lxml` both occur 0 times
in `uv.lock` and still do.
pytest -q: 1566 passed, 1 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`okf build` recovers a PDF's boundaries from the shape of its page text and
never opens the file's own `/Outlines` bookmark tree. On a 701-page process
code whose publisher also ships a NISO-STS structure for it, measured outside
this repository and reproduced here exactly: the shipped default finds 1967 of
2761 titled sections, 0 of its 28 chapters, and 794 of 794 misses have their
heading text PRESENT in the extracted text. The line was read; the boundary
was never opened. The same file's bookmark tree matches 2761 of 2761 of those
titles exactly after normalisation.
`--pdf-outline`, OFF, cuts a PDF at the boundaries its tree declares.
boundaries 1967 of 2761 -> 2759 of 2761 (gate was 2700)
depth 1 0 of 28 -> 28 of 28
titles identical to source -- -> 2761 of 2761
false positives 163 of 2182 -> 3 of 2762
directories with two files 132 of 2050 -> 2 of 2738
front-matter concepts 72 of 2182 -> 2 of 2762
consumption fasit present 4 of 7 -> 7 of 7
hit@1 / hit@8 / hit@50 1/6 2/6 4/6 -> 3/6 5/6 6/6
It is a SEGMENTATION arm, not a reader option: the extracted text is byte for
byte the same either way. A PDF with no tree builds byte-identically with the
flag on -- `diff -r` empty across the pre-change tree, the arm off and the arm
on. An unresolvable `/Dest` is dropped and COUNTED, never fabricated into a
boundary and never a refusal of the file.
The bridge from (page, y) to a line index is the whole risk, so both routes
are measured. `extract_text_lines` splits lines identically to `extract_text`
on 701 of 701 pages, and is CHECKED per page rather than assumed. The y route
and the title route disagree on 0 of 2762 nodes, flat from a 0pt tolerance to
8pt and collapsing at 12pt, so the rule ships with no tolerance constant. The
naive "nearest line" rule was wrong on 1840 of 2762, one line early every time.
The orphan check is not applied to a bookmark mark: it asks whether anything
stands under a candidate's first line, which is the right question for a
heuristic's guess and the wrong one for a publisher's declaration. 683 of 2762
marks are container sections; applying it scores 2079 instead of 2759.
No new dependency and no second parse of the pages: `pdfminer.six` already
ships under `pdfplumber` in `[extract]`. 119.22s -> 183.31s wall, peak RSS
3252 -> 3251 MiB. The default does not move; 1 of the 8 reference PDFs carries
a usable tree at all.
`.pdf` also gains its `_EVIDENCE` row, as `measured` -- it was the row with the
most measurement behind it and no entry in the table.
Report: docs/2026-09-10-k3-runde12-pdf-outlines.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured outside this repository on a 3206-concept bundle of a published
handbook: the code's own process overview contributes 28 of 3206 concepts
(0.87 %) and 117 488 of 1 469 225 source characters (8.0 %), and took 8 of 8
delivered places on one question and 7 of 8 on the known-positive -- which was
not delivered at all. Identical at 343 and 1651 concepts, so the cause is the
corpus's COMPOSITION, that it holds its own table of contents, and NOT its size.
Splitting the corpus would move the defect, not remove it: any corpus with a
contents list, a project overview or a summary document has the same property.
`--source-quota N` caps how many DELIVERED places one source document may take.
It cuts where the shortlist is cut -- before the pack, never inside the DP,
which maximises a sum over a set it is handed -- so the freed place goes to the
next candidate and `k` is still delivered in full.
DEFAULT 2 SINCE TODAY, and it is the third change here that alters a payload
with NO bundle changing (after `--tie-shared-rank` and `--stem-prefix`).
Opt-out `--no-source-quota` reproduces the previous excerpt order.
Swept over {2, 3, 4, off} on three bundles, with the fasit prefixes validated
against the bundle FIRST (that control caught a defect in the measuring query
itself -- it read the last id segment where the document is the first):
- K2, both bundles: at 2 and 3, hit@8 goes 5 of 6 to 6 of 6 with all five
standing rank-1 rows unmoved. The recovered row had missed on every bundle and
every configuration measured until now. At 4 and off it is 5 of 6.
- The handbook bundle: hit@8 2 of 6 -> 4 of 6, the known-positive from not
delivered to rank 4, and the dominant document's share of delivered places
8 of 8 -> 2 of 8 (7 of 8 -> 2 of 8 on the known-positive).
- 2 rather than 3 on rank alone: the recovered rows come in at 5 and 4 rather
than 7 and 5.
WHAT THE GAIN IS NOT. hit@8 asks whether the gold DOCUMENT appears among the
delivered excerpts, and a document quota directly raises how many distinct
documents a payload holds, so that metric is not neutral with respect to this
rule. The five rows that were already rank 1 are neutral, and they did not move.
THE ADVERSE CASE IS NAMED, not left to a consumer. A bundle built from ONE
document carries the same `source_file` on every concept, so a quota applied
literally would deliver 2 excerpts where `k` were asked for -- a rule against
dominance turned into a rule against small bundles. The shortlist is topped back
up from the best-ranked over-quota candidates, which makes such a bundle
byte-identical to the quota being off, and a test holds it.
`--rarity-weight` was measured against the same defect and does NOT repair it:
it leaves the dominant document at 8 of 8 places on the question it floods,
delivers neither that answer nor the known-positive, and holds 5 of 6 on both K2
bundles. Combined with the quota it is worse than the quota alone (the
known-positive falls back out). It stays off.
The vocabulary stays CLOSED and the new code is published in all three places a
consumer can read it: `WITHHOLDING_RULES` (six -> seven),
`docs/consumption-contract.md` 5.3, and the generated SKILL.md -- verified by
reading the generated file, not the code that writes it. `source_quota_exceeded`
is a DIVERSITY drop and not a relevance one, so folding it into
`no_lexical_match` would tell a consumer the question reached nothing in a
concept the question in fact reached. `okf check --skill --payload` stays
conformant, 0 findings over 15 rules.
Editing the contract moved the 7.4 known-positive, which is the coupling
working as intended: 12 563 -> 13 238 encoded, 12 227 -> 12 893 raw, delta
336 -> 345, updated in the constant, the instantiated skill and the shipped
example payload.
Also adds the O6 guard on the reading side: `build_payload`'s signature defaults
are asserted equal to `okf consume`'s argparse defaults for every same-named
parameter. `okf project` shipped that exact disagreement for two rounds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three of round 9's four measured holes, each closed with a rule chosen on a
measurement rather than named as a limit.
`rtf` GIVES 0 SEGMENTS -> 6 of 6 AUTHORED TITLES over N = 4. The container has
no heading style, so the author's title is bold text. The grammar is markdown,
not `rtf`: the converter already writes that title as `**...**` in the same
output every office row produces, so no `rtf`-only heading form exists. Three
parameters were swept over 47 readable documents and ONE carried -- refusing a
line that ends in terminal punctuation takes false-positive lines from 9-12 to
1-2. A maximum title length (unlimited/40/60/80/120) and a
must-stand-between-blank-lines clause are both FLAT, so neither is in the rule.
The last false positive is closed by G1, the principle `_gate_outline` already
carries: recovery yields to declaration. False positives are then 0 of the 31
declaring documents by construction, and 0 of 27 on the corpus. Reach: 2 of 39
corpus documents, both `docx`, 0 of 33 `pdf` and 0 of 2 `xlsx`. Behind
`--bold-title`, default OFF pending the hit@8 measurement; the default bundle
is byte-identical without it.
BOTH ALTERNATIVES THE ORDER NAMED WERE MEASURED AND FELLED. A fourth hand-laid
fixture DECLARES heading styles in a stylesheet and the converter discards
them, emitting the same bold line -- so "read the declared headings out of the
markdown" has nothing to read. `rtf` -> `docx` -> markdown yields 0 ATX
headings on that same document, because the loss is in the `rtf` READER before
any writer sees the style. Fixtures are hand-laid in `make_k2_office.py` with
the fasit written first; they live in their own directory because Door B walks
a drop directory recursively and `k2-office/` reads its N off the listing.
THE PREFIX OVER-MATCH: THREE CANDIDATES MEASURED, ALL THREE FAILED ON ONE ROW.
Re-measured on the pinned 453-concept bundle with the control run first:
`under` occurs 79 times by equality and matches 172 by prefix, `undersjoisk` 0
and 172, `bilateral` 0 and 400 of 453, `standhaftig` 0 and 219. The two extra
known-negatives were FOUND, not chosen -- every 4-character prefix ranked by
document frequency, then a real word taken from the widest. A longer floor
(5-8), a coverage share (0.5-0.8) and a long-words-only floor (>= 8) each cost
row 1 its rank on the default bundle and the whole row on Arm B. Decomposed:
row 1's token `prisene` reaches its gold document through
`pris|sammenstilling` on four characters -- 0.57 of one word and 0.22 of the
other -- so the over-match and the wanted match are one mechanism.
THE FOURTH CANDIDATE IS THE ANSWER: the shared prefix must be a WORD the bundle
uses. `pris` is; `bila` and `stan` are not. `bilateral` 400 -> 0 and 512 -> 0,
`standhaftig` 219 -> 56 and 235 -> 33, every hit@8 row keeping rank 1 on BOTH
bundles. `undersjoisk` stops at 162 because `under` IS a word here -- a genuine
Norwegian morpheme, so that residual is a different answer, not a ceiling. ON
by default (`--no-stem-prefix`), pinned with its own known-negative on the
shipped bytes.
THE SHIM: a path importer holds the object `module_from_spec` made, and
`sys.modules[__name__] = _impl` never reaches it. Measured under both counting
methods -- 3 of 76 public names by `vars()`. One line copies the public names
into this file's globals; the dunder filter is load-bearing, because an
unfiltered copy overwrites `__name__` before the next line uses it as the alias
key. It restores attribute ACCESS and not patch-through, which is why the alias
stays. A CHANGELOG note under 0.7.0 and a shim docstring line say so, since
what the consumer asked for was the note.
Suite 1515 -> 1535.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 9: the four rests in STATE's NESTE that needed no operator decision.
CLAUSE 1 CLASSIFIED BY THE NUMBER, NOT THE TITLE. `_TRAILING_PAGE_NUMBER`
admitted a candidate into a contents run by asking whether the title ended in
an integer -- a question about the number. A drawing's dimension chain, a
schematic's labels, a door schedule, a coordinate column and a soil-layer
table all end in integers and name nothing. Measured over the 43-document
corpus: 68 candidates discarded over 11 of 39 readable documents, of which
19 over 5 documents are data rows.
That corrects round 8's own decomposition. Its "four misclassified numeric
tables and seven real contents listings" needs each document on one side, and
two of the eleven are both. Read across all 68 titles rather than the
three-title sample: 5 documents carry a data row, 8 carry a real entry.
`--contents-name` requires a NAME to survive stripping the page number. The
threshold is SWEPT, not chosen, and collapses at both ends: at an alphabetic
run of 1 a door schedule keeps a stray `V` and 13 of 19 are rescued; at 3 the
two-letter section name `VA` stops being a name, falls out of run membership,
and takes `RIB`, `MMI` and `Tittelfelt` below `CONTENTS_RUN` with it -- one
acronym costing four REAL entries. At 2: 16 of 19 rescued, 0 of 49 regressed.
The three not rescued carry a real word and are named rather than rounded off.
THE CONVERTER'S ANCHOR WAS IN THE CONCEPT ID. Pandoc writes a sheet as
`## <name> {#sheet-N}` and a titled slide as `## <title> {#slide-N}`. Because
a filename is reduced FROM the title, the anchor reached both. Operator
authorised the strip 2026-09-09 after the exposure was counted: 2 of 810
concepts on the previous default bundle, 2 of 1108 on Arm B, 1 of 26 on the
operator's folder. Two ids renamed, one of which `portfolio-optimiser` has
cited in writing; both are in the report so that message can be sent.
One rule in one function, read by BOTH title-forming sites -- a rule in only
one would leave the id and the title naming the same concept differently. The
known-negative is the point: `Mal for {kundenavn}` is a title an author wrote.
odt/rtf/pptx MEASURED END TO END FOR THE FIRST TIME, on hand-built documents,
because the corpus denominator is genuinely zero (86 files: 66 pdf, 10 docx,
4 xlsx, 2 zip, 2 smc, 2 doc). `_EVIDENCE` gains a third class rather than
stretching an existing one: `constructed` means the row has met a document,
but not one anyone wrote for their own purposes. odt 1 of 1 declared headings;
pptx 2 of 2 on a deck that declares slide titles and 0 of 2 on one that does
not -- round 7's reading of pptx was a fixture property, not the format; rtf
0 segments, because the container has no heading style and the author's title
is bold text. rtf is the one open finding.
ACCEPTANCE, all four. The 12-position reference is label-identical in BOTH
readings (pdf 7/8, docx 3/3, xlsx 0/1 or 1/1, sheet 10/12 or 11/12). One K2
bundle carrying both changes: 453 concepts / 865 md, hit@8 [1,1,1,1,1,None]
on it AND on Arm B, with the known-negative still reproducing on the new
bytes. `okf project` byte-equal to `okf build`, `diff -r` empty. Consumer
cost is a re-run: 436/832 -> 453/865, digest 21af4a1aa98315cf.
Three published numbers corrected: README's 596 tests (1515), README's "15
concepts out" for `okf project` (that was the O6 defect; it is 26), and O6's
print-mode method, which does not reproduce without --allowedTools.
Report: docs/2026-09-09-k3-runde9-restene.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`okf project` built a bundle two rules behind `okf build`. `cli.build`'s
signature defaulted `keep_table_heading` and `sheet_section_rows` to `False`
while argparse defaulted both to `True`, and `project.create` calls `build()`
as a function. Measured on a five-document folder: 15 concepts / 30 files
where `okf build` wrote 26 / 52, the whole difference in the priced sheet --
the document a question about price has to reach. The invariant test could not
see it: it compared `project.create` against the same function, and its two
fixture documents had neither a table nor a sheet. Both gaps are tests now,
and the two paths are byte-equal on that folder (`diff -rq`, 0 differences).
README opens with what / one install line / two commands / the three shapes of
request; the phase-status paragraph moved down, nothing deleted. One tag is
pinned everywhere: README pinned v0.4.0 on its install lines and v0.6.0 below,
llms.txt pinned v0.4.0, so an agent reading from the top installed a tag
without `okf project`.
The skill states three modes -- question, hypothesis (per premise, `confirmed`
/ `refuted` / `undecidable-from-bundle`), and a task producing a document
(source per claim in the artefact, an ungrounded paragraph written and marked
rather than dropped, the cut declared inside the document). The five markings
are untouched.
Generated skills state relative paths in the project layout: `okf consume
.okf/<id>` and `okf check --skill .claude/skills/<id>-consume/SKILL.md`,
runnable from where `okf project` tells the reader to start `claude`. Two
absolute paths to zero, measured with a query shown capable of finding first --
O5's published "4 -> 0" used `grep -c "^/"` against paths indented by two
spaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 7 named two open items: a table-block candidate displacing a declared
heading (26 041 characters between entries, `md` at 3 of 4 declared headings)
and 17 590 characters after the last entry, never examined. Measured on
`a364ef4`, the first premise does not reproduce and the second is not a
separate mechanism.
`md` recovers 4 of 4, not 3 of 4. D1 (`--keep-table-heading`) is the repair
for exactly the mechanism round 7 § 5 describes, and round 7 moved it into
the default in the same commit -- so § 5 is a pre-move measurement presented
as the post-move state. That is round 7's own trap, stated in its own report:
a number about a rule is a measurement of a configuration. Nothing held the
cell, which is why it could be wrong and stay wrong.
The remaining gap has ONE cause. Every rule closes a span against the NEXT
MARK; three steps then remove a mark after its neighbour's `end` was fixed
against it. The orphan check leaks 18 527 characters over 15 of 39 documents,
`fold_units` clause 1 leaks 7 514 between entries, and the same clause on the
last run leaks all 17 590 tail characters -- with `unit_fold=False` the corpus
tail gap is 0. Round 6 already established the principle (filter at admission,
let the mark above carry the text) and it was never applied post-filter.
`--close-span-gaps` states it once, after the fold, where every removal has
happened: a span runs to the next SURVIVOR, the last to the end of the text.
It adds no boundary. Measured: 43 631 characters (2.51 %) -> 0, entries
429 = 429, K2 concepts 436 = 436, the operator's folder 52 md = 52.
All four acceptance conditions hold at once. The 12-position reference is
label-identical -- 11 of 12 under |F|[3]=12 and 10 of 12 under |F|[3]=11,
both readings reported, not one position moved. hit@8 holds [1,1,1,1,1,-] on
the round 8 bundle (436), the round 7 default (436) and Arm B (629), and the
known-negative reproduces on the new bytes: `--no-tie-shared-rank` gives
[2,1,1,1,1,-].
The pin is rebuilt against the default that actually ships
(`K2-bundle-default-20260911`, 832 files, digest 8c93e5e3...); two independent
builds of it differ in nothing, `log.md` included.
Two measurement failures are recorded rather than hidden. The corpus identity
check first used `xargs` without `-print0`, every filename split on its
spaces, `shasum` read nothing, and both sides agreed on a digest of the
failure; a control against the digest of empty input separated them. And the
smoke build cannot prove the flag arrives -- the operator's folder has a
coverage gap of zero already, so `diff -rq` is 0 either way; the plumbing has
its own test on a document that has the defect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 6 left two things behind: a measured coverage hole with the repair built
but not the number to move it, and a retrieval regression that kept two
spreadsheet rules off the default. Both are measured; four defaults move.
THE HOLE IS BIGGER THAN THE RULE BUILT FOR IT. Measured over the 39-document
corpus, the pre-move default left 207 435 characters -- 11.92 % -- in no
segment: 163 804 above the first entry, 26 041 between entries, 17 590 after
the last. `--first-span-from-zero` closes the first part entirely, 79 % of the
whole, leaving 43 631 (2.51 %) over 8 of 32 documents. It adds no boundary and
the K2 concept count is identical with and without it (425 = 425); hit@8 holds
[1,1,1,1,1,-] under both tie-breaks and the 12-position reference does not move
one cell.
THE RANKER WAS PUNISHING FINE-GRAINING FOR BEING FINE-GRAINED. Round 6 held
`--sheet-section-rows --keep-table-heading` back because on a bundle built with
them row 1 fell rank 1 -> 2, and ruled out the document prior because its rank
over 39 DOCUMENTS was 1 on both bundles. RRF reads a rank over CONCEPTS:
splitting the gold document 1 -> 12 puts its own twelve concepts in that
signal's whole top tie group, so the one leading the body signal takes position
11, contributing 1/71 where the undivided concept contributed 1/61. A signal
that scored them all EQUALLY still emitted twelve ranks, ordered by concept_id,
and the fusion read alphabetical order as a measurement.
AND THE REPAIR WAS ALREADY IN THE TREE, OFF FOR A COST THAT HAD BEEN REMOVED.
`--tie-shared-rank` shipped off 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6.
Swept over 2 prior exponents x 3 bundles x 6 rows: that fall exists only at
DOCUMENT_PRIOR_EXPONENT 1.0. Round 6 moved the exponent to 0.5 for an unrelated
reason and correctly reported it moved no hit@8 row -- nobody measured the
pair. A flag's "off by measurement" is a measurement of a CONFIGURATION, not a
property of the flag.
Shipped, each with an explicit opt-out and both directions measured rather than
asserted: --no-first-span-from-zero --no-sheet-section-rows
--no-keep-table-heading reproduces the previous bundle byte for byte (diff -rq,
0 differences), and the no-flag build equals the explicit-flag build.
--no-tie-shared-rank reproduces the previous excerpt order -- the one change
here that alters a payload with no bundle changing.
K2 default: 436 concepts / 832 files, digest 8dff8a8e6c15d2f7..., hit@8 5 of 6
ranks [1,1,1,1,1,-]. The pin now holds its own known-negative on those bytes:
read with --no-tie-shared-rank the same bundle gives [2,1,1,1,1,-], so the
green assertion names its cause. Consumer cost is a re-run: 425/810 -> 436/832
on the reference corpus, 15/30 -> 26/52 on a five-document folder.
pptx and md measured end to end for the first time, on two hand-built
documents: md recovers 3 of 4 declared headings, pptx segments per slide only
where the converter recognised a title placeholder. A converter attribute leaks
into titles ({#slide-N}, {#sheet-1}) on 2 of 810 K2 files and 1 of 30 on a
five-document folder -- not fixed, because a filename is reduced from its title
and the fix renames concept ids a consumer has cited.
Suite 1486 (1478 before), ruff and mypy --strict clean over 21 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One rule explains every remaining `pdf` miss on the twelve-position reference:
where a document DECLARES headings, Arm D's RECOVERED headings are the whole of
the excess, and every declared one is a unit the reference wants. `--outline-gate`
admits recovery only where the document declares none of its own, plus any one
recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units`
clause 2's own principle moved from voting to admission, and it filters at
ADMISSION so the text a removed mark opened is carried by the mark above it --
the post-filter form scores identically on all twelve positions and loses that
text, which is why only one of them shipped.
`--outline-gate` and `--drop-wrapped-outline` become the package default, one
decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8
alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of
3. Each keeps an explicit opt-out. The bar the move had to clear was not the
reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-,
no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of
12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2.
Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810.
DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum
measures size and a density is diluted by every unit carrying none of the
question, so a document split 1 -> 12 lost its prior by 12. Swept over five
values on 18 rows it is at least as good as the delivered density everywhere and
strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row
on any of four bundles, so it did not solve the knot it was adopted for -- what
did is that the `pdf` gain never needed `--sheet-section-rows`.
`--first-span-from-zero` is off and repairs a measured loss found while chasing
one position's 940 characters: 32 of the 32 documents that get a plan leave the
text above their first concept in no segment -- 159 704 characters, 9.18 % of
the corpus, 45 841 from one document. It changes nothing on the reference. Off
because it moves the first span of essentially every bundle with no hit@8 number
behind it yet.
vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every
concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads`
carries it for the SEGMENTED profiles, read-only, after the emitted pattern
misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2).
vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1,
`bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`,
keyword-only, rendered before any disk mutation, written only when the index is
created -- Door B's mechanism and ordering.
Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md.
Suite 1478 passed (1449 before), ruff and mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3 round 5. Three questions, three answers, and two of them correct a figure
this repository published.
RETRIEVAL FIRST, because it could have reversed a default. hit@8 over the six
questions on BOTH K2 bundles -- Arm B at 629 concepts and the shipped default
at 492 -- is 5 of 6 with ranks 1,1,1,1,1,- on each, so 0 of 6 rows lost. The
order's rule reverses `--unit-fold` at >= 2 of 6; it does not fire, and the
default stands. The gold sets shrink (49->26, 20->17, 43->36, 11->18) while
every rank holds at 1, which is the fold merging concepts rather than removing
a document from the top.
TWO PUBLISHED NUMBERS CORRECTED, both ours. The S7 candidate ranks 96 of 629
and 159 of 492 were measured with the cost vocabulary passed to
`concept_scores` and NOT to `document_scores`, while `build_payload` passes it
to both; scored the way the shipped payload scores it, the same concept is 10
of 629 and 19 of 492. And round 4 attributed its non-delivery to the default
move -- measured here, it is not delivered on the Arm B bundle either, for a
different reason (knapsack eviction at 68 046 bytes of a 120 000 budget, versus
`below_k`). That column had been inherited from round 3's own build, never
re-measured.
`--pdf-headings font-reserve`, OFF, and the hypothesis behind it is falsified
by its own condition rather than by a score: position 7, the one position the
flag exists for, has THREE outline runs, so the reserve is silent there at
every minimum. It changes 0 of 12 cells on the reference and reaches 4 of 39
corpus documents, none of them rated. Built anyway because it was authorised
and because the condition is now measured rather than assumed. The predicate
lives in one place (`propose.heading_reserve_applies`) and the door receives it
as a callable, like `gate`: a plan indexes the exact string it was proposed
against, so a reserve firing on one side only would make every document it
touches a coded rejection.
The `xlsx` re-reading is confirmed on the artifact -- 11 `rule:sheet-section`
units plus 1 `rule:table-block` ingress -- but the number alone makes the cell
worse (distance 1 -> 2), because the criterion counts that ingress as a table
that should have been merged. A hit needs both halves ratified, and the
reference is the operator's.
`--sheet-section-rows` as a default: three cells better and none worse on the
twelve positions, but the K2 control moves -- row 1's gold document splits 1 ->
12 concepts and its best concept ranks 2 instead of 1. Condition not met,
default not moved.
Default build byte-identical before and after (`diff -r`, 30 md files).
Suite 1441 -> 1449; three of the eight were red first.
Report: docs/2026-09-08-k3-runde5-hitat8-og-skriftakse.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A PDF carries no notion of a heading -- a heading in a PDF is a typographic
fact -- so the text stream `pdfplumber` hands the segment proposer has already
thrown away the only evidence there was. The `docx` path never had that problem:
the converter emits ATX headings and `_ATX` cuts on them. Two readers close the
gap, and both are OFF.
`--pdf-headings font` infers a heading from the conjunction this repository
already measured (size above the document's character-weighted body median AND
a bold font name, recall 1.000 / precision 0.846) and emits it as ATX in the
SAME markdown the office path produces, so `_ATX` applies unchanged and no
PDF-only heading grammar exists.
It stays off BY MEASUREMENT, and the measurement is the point of the round:
against the operator's unit worksheet it takes `pdf` from 2 of 8 to 0 of 8,
losing two exact matches. The mechanism of the loss is stated rather than
guessed -- on those documents the outline rule already recovers the document's
own numbered chapters, so a second heading source can only add. Whole-corpus
screen: 25 of 32 `pdf` change, 0 of 5 `docx`, 0 of 2 `xlsx`. The default bundle
is byte-identical before and after this commit (`diff -r`, exit 0).
`--ocr` reads a page as an image when its own text never arrived: empty, or
`(cid:N)` placeholder codes at or above a threshold READ OFF a measured
distribution -- 834 pages over 32 files, 818 at exactly 0.0 and 16 at 0.93 or
above, nothing in between. On the one corpus document with the failure: 95.07 %
cid to 0 %, 44 to 2561 words of four or more letters, 17 to 18 pages with text.
Its engine is an optional dependency group and never a runtime dependency; a
packaging test pins both halves, and without the group every affected file is a
coded rejection (`extractor_ocr_group_missing`) rather than a crash.
Also corrects two stale published facts found while measuring: the README still
said two segmentation rules were on by default after `f6fea13` made it three,
and CLAUDE.md's K2 digest named the round-3 default. The current default is
492 concepts / 944 files, `bdefa679...`.
Report: docs/2026-09-08-k3-runde4-pdf-skrift-og-ocr.md
Co-Authored-By: Claude <claude-opus-5>
The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.
The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.
The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.
The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.
Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.
Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).
Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.
Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md
Co-Authored-By: Claude <claude-opus-5>
The default move landed in the previous commit; this is the number it costs a
consumer, measured rather than predicted. `okf build` with no flag over the
43-document reference corpus now yields 517 concepts in 969 files, content
digest `42c8646d...`, `merged + coded rejections = 43 = N`, 817.6 s. The
delivered 2026-09-03 bundle (629 concepts, 1108 files, `9cd74519...`) is
unchanged on disk and reproducing it needs `--outline-run 0 --no-unit-fold`
AND the stamp it was built with -- "the opt-out flags" alone is true and
incomplete, so the report says both.
README and CLAUDE.md now state which two rules are on, what each opt-out is,
and that the proposer's own defaults did not move. One CLAUDE.md sentence was
false as of the move ("a no-flag build stays byte-identical") and is corrected
in place rather than deleted: adding a flag still leaves the default
byte-identical; moving it does not, which is why it took an operator decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3 round 3, order 20260908T170037Z-3622420612-from-.claude, plus the PM
injection of 17:55Z carrying the operator's D3 answer.
TWO NEW RULES, both OFF by default.
`--sheet-section-rows` cuts an open table block at the rows that label its
sections: a RUN of at least three rows whose first cell is a bare numeric
label. It is the first rule here that opens a span INSIDE a table, and the
opposite direction from Arm E -- that arm decides how far a block extends,
this decides where it is cut inside. The spreadsheet is the one file type no
arm had ever moved: 1 concept -> 12 on the priced sheet, of which 11 are the
reference's 11 cost groups. The run guard buys exactly three candidates on the
corpus (the rows stating a computation basis), and each section carries its
own `source_sheet` + `source_rows`, verified on the artifact.
`--drop-wrapped-outline` declines an outline candidate whose line continues
onto the next one. Round 2 falsified the LENGTH axis on this case; this one is
not about size. Over the 12-document sample it separates 8 of 34 candidates --
the quoted regulation paragraphs and the risk-table rows -- and none of the 26
the operator kept. Position 4: 4 concepts -> 1, the reference. The cell is
`treffer` by count and lossy by bytes, and the report says so.
Whole corpus, one cached extraction: 1 of 39 documents changes under the first
rule (0 of 32 pdf, 0 of 5 docx), 5 of 39 under the second (all pdf). Arm B and
F2 are byte-identical against a frozen c3b645b, both halves by `diff -r`.
THE `okf build` DEFAULT MOVED, and this is the operator's decision executed,
not a side effect: no flag now means `--outline-run 3 --unit-fold`. Each arm
keeps an explicit opt-out (`--outline-run 0`, `--no-unit-fold`) and the pair
reproduces the pre-move bytes exactly. The proposer's own defaults do NOT
move: they are what the goldens and every published reproduction block are
pinned to, so the two layers now disagree on purpose and `cli.py` says where.
Two shipped expectations moved with it and both are stated in place. MEASURED
AND REPORTED BACK: the configuration made default scores 2 of 12 on the unit
worksheet, `docx` 0 of 3 -- Arm F's published 5 of 12 was measured with
`--table-grid` ON, and without it the fold has no joined table to fold.
Position 1 is declined for the second round on the axis the order named: 3 of
3 recovered chapters have body under them, so "heading-on-heading" separates 0
of 3.
The K2 ranking control round 2 could not measure: two bundles from frozen
c3b645b differing only in `--keep-table-heading`, 2 of 1108 files apart. The
priced concept ranks 96 of 629 in BOTH and is delivered at rank 10 in both, so
the flag buys 35 payload bytes and zero rank positions. Round 2's prediction
is falsified: the concept's NAME was already restored by the orphan-name
inheritance; the flag adds the heading LINE.
The stale S7 literal is restated where it stood (`c759a657...`, 171 614 B,
re-measured on this HEAD), so the next order cannot inherit the superseded one.
1397 -> 1414 tests. Report: docs/2026-09-08-k3-runde3-per-filtype.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The consumer's question about `vann- og frostsikring` in a subsea tunnel
delivered 0 of the 16 concepts covering it, best of them at fused rank 14.
Reproduced with the denominator, then decomposed per signal before anything
was built.
It is not a matcher miss. `normalise("vann- og frostsikring")` already returns
`('vann', 'frostsikring')` on HEAD, the prefix rule already bridges the
inflections, and the best covering concept already answers 7 of 7 question
tokens -- more than any delivered one. A tokeniser rule had nothing to widen.
It is the fusion, but not a weight. RRF ranks every concept in every signal,
including a signal that scored them all the same, and the declared
`(-score, concept_id)` tie-break then orders that group by id. On N500 the
document prior has TWO distinct values over 270 concepts, so the third signal
contributed alphabetical UUID order spread from 1/61 to 1/329 -- enough to put
a concept leading the body signal behind concepts sharing only `tunnel` and
`vann`.
`--tie-shared-rank` lets concepts a signal scores equally share that group's
first rank. The miss closes: best covering 14 -> 3, 2 of 16 delivered. OFF BY
DEFAULT, by the order's own rule: the three requirement lookups hold at rank 1
and the K2 digest holds, but hit@8 over the six published questions falls 5 of
6 to 4 of 6. Decomposed rather than guessed -- K2's prior is coarse (6 values
over 39 documents) rather than degenerate, and one gold sat early in its tie
group. That benefit was never a measurement, but it is a published row.
`--withheld-titles` gives each withheld entry the concept's title, so a reader
can see WHAT was withheld without reading the bundle. 11 lines of code; the
bytes are why it is off. It grows an N500 payload 37.9 % and takes the
629-concept K2 bundle's BOOKKEEPING to 122 704 B -- past the 120 000-byte limit
itself -- which would falsify the breaking point published in the tracked
`skills/okf-consume/SKILL.md` on the day it shipped.
Defaults measured, not asserted: six payload digests built from a frozen
`ff79cfa` (`git archive`, `__file__` checked) and from this tree with both
flags omitted are 6 of 6 identical, and `okf_skill.py` output is identical
apart from the paths each copy writes about itself. Contract checker exit 0 on
eight payloads, both values.
One known-positive did not reproduce and is reported rather than matched: the
order's S7 literal `2ae46f68`/169 573 B is stale by three excerpt-form commits;
HEAD measures `c759a657`/171 614 B.
Suite 1388 -> 1397. Report: docs/2026-09-08-rangeringsbom-sammensatte-ord.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3 round 2, per file type. Order 20260908T143513Z-6327528123-from-.claude, carrying two operator decisions taken beforehand: D1 the orphan-gate variant goes behind a flag, D2 the arms become reachable from `okf build`. No default moved. Report: docs/2026-09-08-k3-runde2-per-filtype.md.
THE REPRODUCTION HELD, all three numbers, before any edit: `okf build` on the five-document tender folder gives 31 markdown files with both PDFs flat and 5/5 merged; the tender PDF gives no boundary without a flag and 9 with `--outline-run 3` (reference 9); the price sheet gives 1 on HEAD. Both proposer runs had to go through `bash -c` -- zsh does not word-split an unquoted `$flags`, so a sweep hands `--outline-run 3` to argparse as one token and every row comes back exit 2.
D2 -- `cli.py:_propose_plans` called the proposer with no arm argument, so the build path ran Arm B while `tools/okf_propose_segments.py` could run D, E and F. It now passes `--outline-run`, `--table-grid`, `--unit-fold` and `--keep-table-heading` through unchanged. THE DEFAULT DOES NOT MOVE and that is measured, not asserted: same folder, no flags, before and after the change, digest 3af10770...8fbbe2 both times and `diff -rq` clean. The "before" bundle was built before the first edit, because the editable install reads src/ live. Red test on the PLANS and on titles rather than a count, with the same fixture and no flags as its control. Per-document table for B/D/E/F/F2 is in the README and the report; the tender PDF is 1 under the default and 9 under every arm above it, and the reference is 9.
D1 -- a sheet heading with a table opening under it has an empty body, so the orphan check drops it: the NAME survives (carried onto the table block), the LINE does not. `--keep-table-heading` lets the heading survive and absorb the table instead. Price sheet 1 -> 1 concepts, `source_offset` [34, 11048] -> [0, 11048], body now starting at the heading. ELEVEN IS NOT REACHABLE THIS WAY and the number says why: the sheet is one heading and one continuous pipe-table block, and the eleven cost groups are eleven ROWS inside it (lines 10-20 of 103). What is missing is a section-row rule inside a sheet -- the opposite of `--table-grid`. Corpus: the flag changes 2 of 39 documents, both `.xlsx`, under arms B, E and F alike; known-negative 0 of 32 `pdf` and 0 of 5 `docx`. With it off, Arm E over all 43 is byte-identical to session 109's tree (33 plans, 43 `.err`, 4 FAILED, diff exit 0, counts asserted first).
THE PDF REMAINDER, one at a time. Position 9: clause 1 read the list AFTER the orphan check, and a contents list without dot leaders is a run of bodiless headings, so all but the last entry were already gone and the run was one. The run is now measured on the pre-orphan list, predicate written once and read in both places. 11 -> 10. Position 7: the same clause required siblings, and a numbered report's contents list interleaves 1.1/1.1.1/2.1 -- its 34 entries are one block that the level condition cut into runs of 9, 1, 1, 1, 5, 2, 10, 2 and 3, so the short runs survived. The level condition is dropped; the run LENGTH, which is what the CONTENTS_RUN sweep bought, is unchanged. Measured outward: the relaxation changes 1 document of 39 and removes exactly the leftover line. 34 -> 33.
TWO REMAINDERS ARE DECLINED WITH NUMBERS RATHER THAN FIXED. Position 1: the three level-1 candidates are 3 of 3 `rule:outline`, same level, same grammar, and the operator keeps one of them by prose alone -- there is no property to read. Position 4: a title-length rule was measured on paper and falsified -- a real chapter is 56 characters and a real heading in a document the arms already score correct is 88, sitting between position 4's 86 and 91, so no threshold separates the classes. Position 0 stays an extraction failure.
ONE SHIPPED EXPECTATION MOVED and is stated rather than quietly updated: `Innhold 1` is now discarded with the contents list it heads. Its body is in no segment afterwards, which is a real cost on a fixture where that heading has one.
Nine new tests: five red before the implementation, four green by construction and named as such. Three mutations, three red, unmutated control green each time -- restoring the level condition, computing the run post-orphan, absorbing a table unconditionally. 1379 -> 1388 tests. ruff clean, mypy --strict clean on 17 files. K2 bundle untouched (1108 files, 9cd74519...). The K2 ranking control is NOT measured: no bundle was rebuilt with the flag, so the rank is a prediction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Order 20260908T133512Z-139864689-from-.claude. First iteration of the
per-file-type directive (operator 2026-09-08 13:05Z), not the last. No
threshold is set: ratifying a bar is the operator's, and setting one inside the
work that produces the measurement would be fitting the bar to the number.
[skip-docs] covers README.md only, and it follows a precedent re-measured this
round rather than quoted: `grep -c` for outline-run, table-grid, Arm C, Arm D
and Arm E returns 0 in README.md and CHANGELOG.md, while --path-prefix, a real
interface change, has a CHANGELOG entry. The rule is "interface and behaviour
changes yes, arm flags no", and --unit-fold is an arm flag that defaults off.
CLAUDE.md IS updated, because its `okf build` bullet enumerates which arms are
off there and would otherwise become false.
FUNN 1, and step 1 asked for it: the reproduction broke. Arm E on HEAD is
byte-identical to the archive on 31 of 33 plans; the two that differ are 2 of 2
spreadsheets in the corpus. The cause is EXTRACTION, not segmentation --
56ae274 writes a workbook as pipe tables, and the sample's price sheet extracts
to 11 048 characters where the worksheet records 100 694, which is the figure
that commit's own message predicts. The consequence is a segmentation
regression against the reference: K3 position 3 went 3 concepts -> 1 under both
Arm D and Arm E, where the operator wants eleven. The mechanism is the orphan
check dropping the sheet heading once a table opens below it (propose.py:461),
already reported there as a ranking regression. Doors unchanged: 43 .err, 4
FAILED, extractable 39/43.
THE MATCH CRITERION WAS WRITTEN DOWN BEFORE ANY CELL WAS SCORED, and it stalls
at 7/12 on the literal calibration gate after three rounds, each revision
recorded. The five failures are not the criterion's: at every one it agrees
with the operator's own (a), (b) or free text and disagrees only with (c).
Column (c) is a RELATIVE judgement ("closest today"); the four K3 categories
are absolute. The only way to reach 12/12 is to define "correct" as "the
closest arm", which reads (c) back out of itself. The dominance gate, declared
in advance as the second reading, holds at 11/12.
ARM F is one rule with three clauses derived from the operator's three, not
twelve special cases, and it only MERGES or DISCARDS: a run of at least
CONTENTS_RUN same-level page-numbered headings is a contents list and goes; a
heading deeper than the unit level folds into its parent, extending the
parent's span; a table folds back into the shorter heading that introduces it,
keeping the HEADING's name. K3 first rater, n=12: 2 coarse / 5 fine / 0
duplicate / 5 correct -- best of four arms, ceiling was 4, two moved, nothing
regressed anywhere.
THE PAPER MEASUREMENT CAME FIRST AND FALSIFIED THE FIRST VERSION. Clause 2 was
letting rule:outline -- Arm D's RECOVERY of an integer numbering run -- vote on
the unit level, which took K3 positions 1, 7 and 9 to 3, 4 and 7 concepts
instead of 17, 34 and 11. A recovered numbering is a heuristic, not a level a
document declares, and the unit worksheet showed the operator ATX and dotted
headings only. Fixed with its own red test; 11 of 12 predictions correct after.
PER FILE TYPE, which is the directive: docx 3 of 3 (solved on this sample), pdf
2 of 8 (lags, unchanged by Arm F, and the remainder is decomposed per position
rather than left as one number), xlsx 0 of 1 (regressed, see FUNN 1). Outside
the corpus, n=1 each: pptx and odt byte-identical, rtf proposes nothing either
way, txt differs and exposes clause 2's fallback.
CONTENTS_RUN swept 1..5 and off. Distance prefers 1; three ships anyway,
because at 1 the body chapter "... i henhold til TEK 17" is deleted for ending
in a number, and no K3 cell differs between 1 and 4 -- the metric prefers a
value that provably deletes a chapter and cannot see the cost.
Whole corpus, all 43 through arm_run in ascending foreground chunks: 32 plans,
491 entries against Arm E's 679, 14 documents changed, 1 plan disappeared
entirely (three drawing-schedule numbers that clause 1 correctly reads as a
contents run) and that is reported rather than special-cased.
THE okf build MECHANISM IS REPRODUCED AND IT IS NOT DOOR B: cli.py calls the
proposer with no arm flag at all, so the shipped build path is Arm B. On a
tender PDF that means no boundary where Arm D finds nine and the reference says
nine. Largest per-file-type gap this round found; it is a default change and
therefore the operator's.
5 tests red first, 1373 -> 1379. ruff clean, mypy --strict clean on 17 files.
K2 consumer bundle unchanged: 1108 files, digest 9cd74519... with the flag off.
No bundle built, no version bump, no tag, no push.
Report: docs/2026-09-08-k3-arm-f-mot-enhetsarket.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two changes, one theme: what a reader needs in order to cite is a property of
the PRODUCER, so neither the excerpt nor the skill may hard-code a list of the
producers someone thought of.
The pass-through rule is now the `source_` PREFIX, not the five keys this
library writes. Measured on the N500 bundle currently on disk: 269 of 274
concepts carry `source_element_id`, a locator that repository chose under this
chain's own rule ("the key says what it indexes") and that this library never
writes. The allowlist dropped it, and an excerpt that names a document without
naming the place in it is the defect this work exists to close. A prefix and
never a substring - `resource_owner` contains the literal and is not a locator,
and promoting it would be fabricated provenance produced by a matching bug. The
known-negative is tested: `bundle_id`, `type` and `ingested_at` do not travel.
Contract 8.5 states the rule as a prefix rather than a list.
K2 control, re-measured against the frozen tool at b6a8c8b, same question and
same k: the RANKING is untouched - same 8 ids in the same order, identical
`text_sha256`, identical `withheld`, denominators 629 = 621 + 8. The FIELD moved:
payload 108 877 -> 113 143 B (+3.92 %), spent 18 606 -> 22 210 (+450.5 B per
excerpt), excerpt members 9 -> 17, 99 changed lines. Known-positive follows the
contract document's bytes again: 12 049 -> 12 563 measured, 11 719 -> 12 227
raw, delta 330 -> 336.
`tools/okf_skill.py` instantiates the template for one bundle: id, ref, concept
count, the conditional-field table with a denominator per field (the `source_*`
rows DISCOVERED from the bundle, not listed), the whole-bundle cost by the gate's
own instrument, the share one measured answer spent, the concept count at which
the withheld bookkeeping alone reaches the limit, and the index-walk-against-
directory control - run once at generation time, never on the question path.
The form was chosen on a measurement that came out against the obvious gate:
the contract checker passes the UNFILLED template against a real payload, and
passes a skill built for a different bundle against this one's. It cannot tell
the two forms apart, so conformance could not decide it. What decides it is that
5's denominators, 6.4's conditional fields and 7.6's breaking point are
per-bundle numbers - a generic skill either leaves them as holes (the template's
own definition of unfinished) or states another corpus's numbers, which is worse
than a gap. Every gate the checker lacks is therefore a test here: no placeholder
survives, the skill names its own bundle's id and ref and not another's, its
commands are absolute and point at files that exist, and it refuses a directory
with no index (exit 1, `bundle_unreadable`), an index with no `bundle_id`
(`bundle_id_missing`), an empty bundle, and an occupied target without --force.
It lives in `tools/` for the reason `okf_consume.py` and `okf_contract_check.py`
state for themselves - outside `src/`, so no consumer's install surface changes -
and because a wheel-installed `okf skill` would emit a command pointing at
`tools/okf_consume.py`, which the wheel does not contain.
Suite 1372 (1347 before), ruff clean, mypy src clean.
Co-Authored-By: Claude <claude-opus-5>
The pre-pass delivered the right concept and the answer could not name it.
Measured by portfolio-optimiser 2026-09-08 over three paid arms: the gold
concept came back at rank 1 of 8 on 3 of 3 bundles, and the model answered
correctly on 1 of 3, because a delivered excerpt carried `concept_id`, body
text and nothing the document is known by. The previous session measured the
same gap from the other side: the provenance it had just written into every
concept did not reach the payload at all.
`excerpt_for` now carries `title` unconditionally, and `req_number`, the SPEC
5.1 address `sources` and each locator key (`source_pages`, `source_sheet`,
`source_rows`, `source_lines`, `source_offset`) when the concept has them. A key
the producer did not write stays absent: an empty value would assert that they
wrote an empty one, which is the contract's 6.4 failure.
`sources` is read in BOTH YAML forms, on a measurement rather than a taste. K2
writes the flow form on 629 of 629 concepts; the largest N-bundle writes the
block form on 270 of 270 and carries no locator key at all, so a flow-only
reader delivers that bundle with no address whatsoever. Reading the block form
is not a licence to write it - the emission rule is untouched, because the
line-oriented parser still cannot round-trip a block list. A `sources` value
this reader cannot decode is named (`sources_unreadable`), never dropped into
the same silence as an absent one.
Contract 8 gains the requirement and the checker gains its code
(`excerpt_unnamed`, 15 rules now, was 14): an excerpt a reader cannot name is
one an answer cannot cite, whatever its rank. `req_number`, `sources` and the
locators are SHOULD, not MUST - they are conditional on the producer, and a
bundle whose concepts carry no identifier cannot deliver one.
K2 controls, same question and same k, before against a frozen copy of the tool
at b6a8c8b: the RANKING does not move - the same 8 concept ids in the same
order, identical `text_sha256`, identical `withheld`, identical denominators
(629 = 621 + 8). The FIELD is what moved: payload 108 877 -> 111 744 B
(+2.63 %), budget spent 18 606 -> 20 907 (+287.6 B per excerpt), excerpt
members 9 -> 15, 83 changed lines. The contract document's own bytes moved with
8, so the budget instrument's known-positive moves with it: 10 349 -> 12 049
measured, 10 060 -> 11 719 raw, delta 289 -> 330.
New fixture `tests/fixtures/consume-provenance`: the two address forms and a
concept carrying neither address nor identifier. Purpose-built, because the two
real bundles are complementary and neither exercises both forms.
Suite 1347 (1339 before), ruff clean, mypy src clean.
Co-Authored-By: Claude <claude-opus-5>
A concept named its source file by basename and, when segmented, carried a
`source_offset` into the text THIS LIBRARY extracted. Following that pointer
needed the corpus directory, the extractor and its exact transitive version --
none of which the bundle carries. Hand-walked on a real K2 concept: six steps,
four of them requiring knowledge from outside the bundle, to learn that a
requirement sits on pages 12-13 of a 20-page document.
The address is spec's: `sources: [{ resource, title }]`, where `resource` is
the dropped file's inbox-relative path (SPEC v0.2 5.1:303-306 -- "an absolute
URL, a bundle-relative path, or a path into a `references/` subdirectory").
The locator is ours, and it has to be: 5.1 has no field for a place within a
resource, and the pinned guard (1.3.0) rejects every route to putting one
inside a `sources` entry -- a non-allowlisted key by name, a nested flow list
as "scalar leaves only", and quoting as an unsupported form. So the locator is
top-level keys shaped like `source_offset`, and a path carrying a flow
terminator is refused fail-fast rather than mangled.
The unit table is built AT EXTRACTION, where the extracted text and the
original's structure are known to agree: pdf -> `source_pages` from
pdfplumber's own page numbers (a page that yielded no text does not renumber
the ones after it), xlsx -> `source_sheet` + `source_rows`, everything else ->
`source_lines`. `source_offset` stays.
Two measurements changed the design before it shipped. A `paragraphs` key for
docx would name a number the document does not have: `<w:p>` counts of
108/27/65/176/57 against converted-markdown lines of 75/33/67/144/63, not one
pair agreeing -- so the key is `source_lines` and says what it indexes. And an
empty spreadsheet row renders exactly like a table separator: the content-based
rule ate 8 empty rows on the K2 price sheet and reported its last row as 92
against a workbook that says 100. The separator is now found by position, and
`tomrad.xlsx` keeps that red.
One profile moves. `provenance` is a policy object, `None` everywhere but
`SEGMENTED_OKF_V0_2`; the other five shipped profiles are byte-identical.
K2 rebuilt from a frozen src copy: 629 concepts, 1108 files, name set identical,
0 ids moved, 479 files byte-identical, 629 changed and 0 lines removed anywhere.
629/629 now carry an address and a locator. New ref
`sha256-tree:665563a2f74423fcbcc8e4f0b0954ee73b73985ac0418de4f6987bd162a1f7c8`;
`2f82fcfe...` is stale. The pre-pass payload does not grow by one byte
(209 092 B before and after, 18 changed lines: the ref and eight per-concept
digests) -- because an excerpt carries the body, not the frontmatter, which is
also why the consumer still cannot cite "file X page 12" from a payload alone.
Report: docs/2026-09-08-proveniens-k2.md. 1339 tests, ruff and mypy clean.
Co-Authored-By: Claude <claude-opus-5>
O2b asked whether a requirement number can be made worth more than a common
word by weighting each hit with the token's rarity in the bundle, with no
hand-set constant and no declared token class. It can, on one of the three
bundles, and the two it cannot are decomposed rather than guessed.
The rule is log(N/df) over the concepts' own tokens, counted with the same
four-character prefix rule a hit is scored with. It enters the RANKING and
never the GATE: `lexical` stays a count, because `krav` weighs exactly 0 on
all three bundles and a weighted gate would drop every concept matching only
that word -- which is the gate 54a0bc2 falsified for other reasons. One df
table per bundle reaches every stage that scores the question against text,
including the document prior. One pass, 0.241 s over 1 133 concepts.
Measured on four corpora, before and after, with every published figure
reproduced first: gold fused rank 96 -> 103, 9 -> 8 (withheld -> DELIVERED at
rank 8) and 35 -> 35; K2's priced sheet candidate rank 10 -> 2 with the cost
vocabulary and 251 -> 78 without; Q-good unmoved at rank 1; hit@8 5 of 6 with
every rank identical; the S7 control payload byte-identical on the default
command.
DEFAULT OFF, decided by the number and not by taste: it does not win on all
four, because N100's gold loses seven rank positions. Off means the bytes that
were already published, and that is measured -- 8 of 8 payload digests
identical against a frozen copy of 56c1205 built with git archive.
Two limits, both someone else's mechanism and both named: MIN_SHARED_PREFIX=4
makes a unique identifier read as 135-of-446 common on N100, so the weight
correctly ranks a common adjective above the exact requirement number; and RRF
consumes RANKS only, so on N500 -- where the gold already leads the one signal
that can see the identifier, and the other two cannot see it at all -- no
weighting inside a signal can move anything.
Consumption-side only, so no rebuild: the K2 bundle ref 2f82fcfe... stands.
Report: docs/2026-09-08-sjeldenhetsvekt.md. 13 new tests, red first; 8
mutations, 8 red, two of them only after the survivors were read as code -- one
exposed a fixture that put the identifier where the real corpus does not, and
the corrected fixture is what found the RRF limit. Suite 1295 -> 1308.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The prior measurement (docs/2026-09-08-blindsone-below-k-k2.md SS 3) found that
the budget, not the ranking, is the second lock on a mandate-shaped cost
question -- and that the same mechanism was a REGRESSION on the question that
works: raising `--k` to 16 evicted the gold concept, because the exact knapsack
maximises a SUM of fused scores and has no opinion about rank, so twenty small
excerpts out-value one that costs 56.5 % of the budget.
Measured here on the same 629-concept bundle, with the three known-positive
figures from `4c699fd` reproduced first:
- Corpus distribution, denominator 629: median excerpt 857 B, max 223 391 B,
3 concepts over the limit alone.
- Candidate rule (b), a corpus-derived budget, is FALSIFIED by two numbers: two
defensible derivations are 49x apart on the same corpus, the small one turns
the gold concept into `over_budget_alone` (13 refusals against 2), the large
one changes nothing at the default k. A budget is the consumer's constraint,
not a property of the corpus; `--limit` already belongs to the caller.
- Built instead, behind `--reserve-top-rank` (default OFF): the top-ranked
candidate gets its bytes before the pack runs, AFTER the `over_budget_alone`
pre-exclusion and never before, and the payload declares `budget.reserved`.
- It fixes the eviction: k=16 and k=24 deliver the gold concept at rank 1,
costing one and two excerpts, and 20.4 % / 27.3 % FEWER o200k tokens.
- It changes the delivered list in 2 of 24 measured combinations -- both of them
that eviction. In the other 22 the list, its order and `spent` are identical.
- It does NOT close the mandate-shaped blind spot: that concept ranks 10, not 1.
The one delivering command is `--cost-vocabulary --k 12 --limit 160000`
(62 149 tokens against 58 401), and that is a consumer's decision.
11 new tests (RED first), 7 mutations 7 red with an unmutated negative control
green before and after; two of the seven survived the first test set and the
tests were strengthened. Default payload byte-identical, both goldens unchanged.
Report: docs/2026-09-08-blindsone-laas2-budsjett-k2.md
Suite 1279 green, mypy --strict clean over 28 files, ruff clean.
Co-Authored-By: Claude <claude-opus-5>
The consumer report (portfolio-optimiser, S7 SS 2) found that a mandate-shaped
cost question withheld the corpus's one priced table under `below_k`. Measured
here, on a bundle proven byte-identical to a fresh HEAD rebuild:
- The mechanism is a VOCABULARY gap, not a `k` defect: two of three ranking
signals are exactly 0.0 and the concept is candidate 249 of 269.
- The k-sweep buys nothing: k in {8,12,16,24,32,64,128} all withhold it, at
+9.5 % tokens. It also found a regression -- for the question that WORKS,
k >= 16 EVICTS the gold concept, because one 67 838 B excerpt is 56.5 % of
the budget and the knapsack maximises a sum.
- Two proposed rules were falsified BEFORE any code: number/table density ranks
the priced table 178/165/46 of 269 (the form is unfilled, so it is
number-poor), and per-document spread puts its document 30th of 35.
Built instead, behind `--cost-vocabulary` (default OFF, DEFAULT byte-identical):
one declared cost/price/quantity vocabulary family that bridges a question and a
document naming money with different words. It moves the concept from candidate
rank 249 to 10 -- and does NOT close the blind spot: the budget still refuses
it, which is now a separately measured second lock.
Seven RED tests first; six mutations of the rule, six red (two survived the
first version of the tests and the tests were strengthened). Control: a question
with no cost term produces a byte-identical payload with the flag on, at every
k, on the real corpus. Known-positive: 164 987 B / 40 425 o200k tokens, equal to
the published pair.
Report: docs/2026-09-08-blindsone-below-k-k2.md
Suite 1268 green, mypy --strict clean over 28 files, both goldens unchanged.
Co-Authored-By: Claude <claude-opus-5>
The flag threads through `run` and `main` and takes no argument. Arm D's gate
is a run LENGTH where 0 means off; Arm E has no numeric parameter, so a boolean
is the honest shape and an integer would only manufacture a sweepable knob that
means nothing. `run` therefore adds no numeric validation, and the help says
why.
Both prose sites that enumerate the arms `okf build` does not expose are
updated: `src/llm_ingestion_okf/cli.py` and `CLAUDE.md`. The second was found
by review, not by grep of the first -- the same claim lives in two files and
only one of them is code.
The generalised attribution test earned itself in this commit. The first draft
of the Arm E help contained "byte-identical to Arm D -- Arm D rather than Arm
B", and argparse's rendering plus the test's ` --` chunk split meant the
attribution fell OUTSIDE the `table-grid` chunk. The test went red with the
truncated chunk printed, which is exactly the failure it exists to catch: a
whole-output grep would have been satisfied and the attribution would have been
unfindable in the option it belongs to. The clause is now parenthesised.
Arm C's marker check in `tests/test_cli_build.py` gains `rule:table-grid` and
is renamed to speak of all three arms, measured on the artifact rather than on
the flag: a flag `okf build` never passes is not evidence about what it emits.
[skip-docs] is the MEASURED precedent, not a convenience. `grep -c` for
"outline-run", "max-segment-chars", "Arm C" and "Arm D" returns 0 in both
README.md and CHANGELOG.md: an arm flag is documented in its constant's `#:`
comment, in `--help`, and in the round's measurement report, and it is off by
default so it makes no promise to a consumer. `--path-prefix`, which is a real
interface change, does have a CHANGELOG entry. The rule this follows is stated
at docs/2026-09-07-k3-arm-d.md: "interface and behaviour changes yes, arm flags
no." Arm E's report is docs/2026-09-07-k3-arm-e.md, later in this round.
Tests first: 3 red, then green (a fourth, the no-argument test, is honest in
its docstring that it is green before the flag exists too, because argparse
rejects an unknown option with the same code; it becomes evidence only once
the flag is real). 1248 -> 1251.
ruff check: exit 0. ruff format --check: exit 0. mypy --strict src/ tools/:
27 files, Success. pytest -q: exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Step 12's README section is brought forward to here because the docs gate is
right: a feat commit that ships a new command needs the command documented.
CLAUDE.md's Commands section gains the pre-pass beside `okf build`. Nothing
else moves.
Contract check against a real payload from the 629-concept bundle:
$ .venv/bin/python tools/okf_consume.py <K2-bundle> \
--question 'Hvordan skal prisene fylles ut?' --out /tmp/k2.json
$ .venv/bin/python tools/okf_contract_check.py \
--skill skills/okf-consume/SKILL.md --payload /tmp/k2.json
conformant: 14 rules over 8 excerpts and 621 withheld entries, 0 findings
exit=0
And the two negative controls, because a green checker proves little on its
own -- measured, it returns 0 findings on an empty payload paired with the
unfilled template:
broken denominator identity -> NOT conformant, 2 findings, exit=1
missing payload file -> exit=2
Placeholder scan, known-positive first: the DOTALL scan reports 20 occurrences
on the template and 0 on this copy. The shipped example payload is generated
from the in-repo golden bundle, not from the corpus, and a test regenerates it
byte for byte. No K2 concept path or document title reaches any tracked file
here, checked with a pattern shown able to find against the bundle's own index.
Suite run after git add: 1224 passed, mypy --strict clean on 26 files,
ruff clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now "run the door over a folder" was a shell loop over two scripts
under `tools/`, with nine flags between them and a `--path-prefix` rule
that lived in a code block in a measurement report. Neither script was
packaged (`pyproject.toml` ships `src/llm_ingestion_okf` only), so the
path the published K1/K2 numbers were measured on was reachable from a
clone and nowhere else.
`okf build <folder> --bundle <dir>` is that path, packaged, declared as a
console script and installed with the wheel. It is orchestration only:
the proposer and the corpus harness MOVED into the package
(`llm_ingestion_okf.propose`, `llm_ingestion_okf.corpus`) and the two
`tools/` scripts became thin entry points to them, so the published
reproduction blocks still run and there is exactly one implementation of
each rule. Neither move adds a dependency or a model call.
Two decisions belong to this layer and are stated where they are made.
A document's proposed paths are scoped by its RELATIVE PATH minus the
extension, not its basename: the door walks recursively now, and two
documents named alike in different folders would otherwise collide on a
path Door B is supposed to make impossible rather than merely detect.
And omitted timestamps do not come from the clock -- `--ingested-at` and
`--proposed-at` default to one shared epoch constant, because a
wall-clock default would put a changing byte in the artifact and take
rebuild-equals-incremental away from every caller who did not pass them.
Arm C and Arm D stay off and are not exposed here.
Measured on the 43-file K2 corpus, one invocation against the two-script
bundle of 2026-09-03: N = 43 computed, merged 39/43, coded rejections
4/43 (`extractor_unknown` 3, `extractor_empty_pdf` 1), K1b 39 + 4 = 43,
exit 0, 779.43 s. 1107 of 1108 files byte-identical. The one that
differs is the root `index.md`, by exactly the `log.md` link a commit
fifteen hours younger than the stored artifact adds -- appending that
line to the stored file reproduces the new one byte for byte. Against
the two scripts at THIS commit the trees agree in full, which is what
the byte-identity test holds.
Suite 1127 passed after `git add` (1113 before), mypy --strict clean,
ruff clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Door B listed `inbox.iterdir()` and kept only top-level files. A file in a
subdirectory was neither ingested nor refused: it appeared in none of the
result's buckets, so a nested drop produced a bundle that was silently short
of what was dropped and no count said so. That broke the K1b identity for any
inbox with folders in it. Operator decision 2026-09-06.
- `walk_inbox` is the ONE walk rule, shared with `tools/okf_corpus_run.py`:
the denominator N is now counted over exactly the set of files the door
ingests, rather than over a second listing that happened to agree.
- Sorted on the whole relative path, not the basename, so the order is a
function of the tree; that is what keeps rebuild-from-scratch byte-equal to
an incremental update.
- A concept's `source_file` is the path relative to the inbox root,
`/`-separated. The concept NAME still comes from the basename, so two
folders holding one basename hit the existing §3 collision refusal instead
of one silently claiming the other's concept.
- Dot-directories and a bundle directory inside the inbox are skipped with a
CODE, in a new `InboxResult.skipped`. Recursion makes the door's own output
reachable as its own input; a silent skip would be the same
absence-without-a-denominator defect one level down.
- `--path-prefix` reduces per component and rejoins with `/`, so the caller
driving a nested corpus can carry the relative directory. Reducing the whole
string folded the separator into a `-` and flattened `sub/sub2`.
`tests/test_inbox_flow.py::test_subdirectories_are_not_walked` asserted the
opposite and is superseded in place, with the reason written down.
Measured on the K2 corpus (flat, N=43): 39/43 merged, 4 coded, K1b holds. The
bundle digest is
`1472e98aec8643c5beee540f4c42b5e437bd26e7c61d69a91bcff799f06a6d13` over 1108
files -- byte-identical to a run of the same corpus at 190086f WITHOUT this
change (`diff -r` exit 0), so recursion costs a flat inbox nothing. It differs
from the stored 2026-09-03 artifact by one line in `index.md`
(`- [Corpus run history](log.md)`), which 95eb271 added 15 hours after that
bundle was built.
Suite 1113 passed, `ruff` clean, `mypy --strict src/ tools/` clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six decisions, each with the failure it prevents: confidence carried in the
format, certainty stated per field, supersession never inferred from one
document, the index as a projection rather than an append, facets as a new
profile, and Door B's ownership stamp left deliberately open.
Six findings reported as findings rather than as gaps, per the order: the cost
of closing the metadata gap is 3.3x-6.4x the flat index and whether that keeps
the OKF arm's cost advantage cannot be answered from this repo -- it needs
their bake-off re-run on their corpus; a reference to a parent-level number
does not resolve to a child, and the fix needs a real corpus before 'unique'
can be defended; OKF_V0_2's verified/status/stale_after are still expressible
and unwritten for the reason already recorded.
Every number carries its denominator, and the supersedes 0/55 carries a
known-positive control (50/55 on a corpus that has shared numbers) proving the
query can find what it reported missing.
The operator's condition for revisiting the pin was met (v1.2.0 contains
the flow-mapping frontmatter fix, commit 5870483) and the operator has now
approved the move itself, dispatched as its own order because it changes
consumer-visible Requires-Dist and shifts golden-fixture admission.
Floor 1.2: this library needs the flow-mapping support (`generated: { by:
x, at: y }`) that landed there — without it Door C fail-secures every
concept carrying that stamp. Ceiling <2.0, not narrower: the guard's own
1.0.0 release freezes its exported surface until a 2.0.0, and explicitly
keeps calibration (severities, dispositions) free to move within 1.x, so a
tighter ceiling here would claim a stability guarantee neither side needs.
Re-measured through Door C against the guard's own default
(allow_reserved=True, matching how the earlier recommendation measured
it), over the 9 concept documents across all four Door-A goldens:
4/9 admitted at 0.3.4 -> 8/9 admitted at 1.2.0, confirming the number
already reported. The ninth (a `sources` block-list carrying `resource`)
stays refused by design (G30) and is not expected to move.
Failing-test-first: test_guard_adapter.py::test_guard_version_is_inside_the_pin
and test_packaging.py::test_the_only_runtime_dependency_is_the_security_boundary
were updated to the new pin first and confirmed red against the
still-installed 0.3.4, then `uv sync --extra extract` installed 1.2.0 and
both went green. Full suite (615), ruff and mypy --strict clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxcxzNwpX1kDP53n1rLhM5
Order G2a. Populates the optional `[extract]` extra for the first time with
one parser, `pdfplumber>=0.11.10,<0.12` (MIT), and wires `pdf` through it.
The default install is untouched: exactly one runtime dependency, stdlib
otherwise, enforced by test_packaging.py.
The gate for `pdf` becomes an import probe rather than a frozenset membership
test, exactly as extract.py's docstring had promised. The rejection does not
change: without the extra, `pdf` still raises `extractor_extra_missing` with
the same message. That behaviour is asserted UNCONDITIONALLY via a sys.modules
monkeypatch, so it holds on machines where the parser is installed too — a
skip would have preserved nothing there. Verified in a clean venv without the
extra: 589 passed, 7 skipped; with it, 596 passed.
`docx`/`xlsx` are unchanged and still fail fast — the extra names exactly what
it ships.
The parser choice was forced by measurement, not preference (b73dd9d,
docs/2026-08-21-g2-pdf-extraction-measurement.md): on a real requirement table
pdfplumber keeps 4 of 4 rows with label and value on one line, where pypdf,
pdfminer.six and pymupdf each keep 0 of 4. pymupdf is additionally out on
licence (AGPL-3.0), which an MIT package must not push onto a consumer.
Three facts from that measurement are now carried in code rather than in a
report:
- Extracted text is pinned to an exact transitive parser version
(pdfplumber pins pdfminer.six==20260107; date-stamped, no stability
contract). tests/test_extract.py freezes the expected text of a committed
hand-written fixture so a parser upgrade breaks something visible instead of
drifting silently. Reasoning at the declaration site and in
tests/fixtures/README.md.
- Determinism within a version is now held by a test, not only measured once.
- Drawn content does not survive extraction. Every pdf extraction emits the
new `ExtractionWarning`: figures have no text to recover, so a bundle built
from drawn documents is incomplete by construction. Stated categorically
rather than detected — deciding "is there a figure here" is the layout
heuristic G2b declined.
Two new error codes, both mirroring existing patterns: `extractor_empty_pdf`
(a scanned/image-only PDF, refused rather than persisted as an empty concept)
and `extractor_pdf_error` (parser failure wrapped, never leaked).
Structured table recovery (G2b) is NOT implemented and is documented as out of
scope: two independent parsers return the same wrong shape, so the breakage is
document geometry, not a library choice. PDFs enter as prose.
Also corrects an install promise this change would otherwise have published:
the README no longer presents a bare `pip install 'llm-ingestion-okf[extract]'`
as working, because the package is not on an index.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtNhsdHnMGtMi7U2mvMU8z
Commons ratified V1 2026-08-02 and executed it at `54e0ec7`; verified
against their tree rather than taken on report. ingest-spec.md:217 now
defines `generated` as `{ by: process:okf-ingest, at: <ingested_at> }`,
unquoted, `at` repeating `ingested_at` verbatim. `generated: true` no
longer appears in the spec.
`DEFAULT` states commons' §5 layer, so its stamp is theirs to decide.
`DEFAULT.ownership` gains the actor; the four goldens this repo's plan
named in advance were regenerated by RUNNING the materializer, each on
its own case's `ingested-at.txt`. The v0.2 golden was untouched, as
predicted -- it has carried the O2 form since D5.
Not a migration onto OKF v0.2: `DEFAULT` stays v0.1 on every axis
upstream owns and still emits no `sources`. Commons' spec and the Google
version are independent axes, and comments that narrated them as one
were rewritten rather than left to mislead. README and CLAUDE.md said
the additive rule without that boundary, which would have told a
consumer their DEFAULT bytes can never move; both now state it.
V-A3 is amended, not dropped. `DEFAULT` must OWN the mapping it now
writes -- a profile refusing its own output fires the collision gate on
files its own previous run wrote -- while a mapping naming a foreign
actor, or §7's `human:` actor on curated content, stays unowned. That
half is what carried the safety and it is asserted directly.
§11's stamp-integrity condition moved with the value: the forgeable
stamp was `true` and is now the mapping naming the ingest actor. The
defence was never the value -- the §3 scan globs `ingest-*.md`, so a
Door C import is unreachable however well it forges. Second spoof test
added; both were hand-mutated (glob widened to `*.md`) to confirm they
can fail.
The characterization test derived its foreign-stamp fixture from the
literal `generated: true`, which V1 leaves without a referent -- a
silent no-op waiting to happen. It now derives the needle from the
profile and asserts the substitution occurred.
Door B is deliberately untouched: not the ingest-spec's, marker is
`generated` + `source_file`, disjoint from Door A's `ingest_manifest`,
and the divergence predates V1.
Nothing released or notified. The pilot set pins `v0.5.0a2`, not `main`,
so this is invisible to portfolio-optimiser's freeze and demo; the
consumer exposure report is owed at the release that carries this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VwcjUXbKySLbEG5WqTNkta
Operator, 2026-08-02: no MCP, no data lakes, no databases without a
stated need. We run on what we have -- defect fixes, improvements, and
features a consumer actually asked for.
That supersedes the 2026-07-27 line calling phase 5 committed, which
this file still carried. A roadmap that promises work nobody asked for
is a queue that generates its own deadlines, and the sql-is-sqlite-only
"gap" is the clearest case: closing it buys runtime dependency number
two for no asked-for use.
Phase 4 is not demoted the same way -- four named consumers have working
implementations to lift, so the need is real but untriggered. It starts
when one of them asks.
The upstream version policy is stated as the one exception and why: it
is a promise already made, so an upstream release is itself the need.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JxRjDS9U2szUGYDeuZB2M3
Measure first, widen after. The 19-fixture guard-surface suite was re-run
against v0.3.4 in a scratch venv before the range moved, and reproduced the
three deltas measured against v0.3.3 exactly, with none added. v0.3.4 is the
tag pinned rather than v0.3.3 because it shipped first and repairs a quadratic
regex (okf._MD_LINK_RE) that sits on Door C's own call path.
Door C now passes allow_reserved=False explicitly. The guard added the keyword
in the 0.3 line and defaults it True for received bundles, which would merge a
sender's index.md / log.md instead of rejecting them. The override keeps the
unconditional reserved-name refusal committed to before the keyword existed,
and the reason is structural rather than a second opinion on the guard's scan:
Door C generates the merged bundle's index.md from what it merged and writes
every merged concept verbatim, so a sender's index.md would be a second and
irreconcilable claim on one path.
This is not a behaviour change for anyone on the previous pin: under v0.2.0
the keyword did not exist and reserved names were refused by construction.
The floor is >=0.3 and not >=0.2 for a measured reason. allow_reserved is
absent in v0.2.0 and present from v0.3.0 onward, checked across all five tags:
a >=0.2 floor would admit a version that raises TypeError on every Door C
import. That measurement also corrects a recorded premise -- the plan said the
keyword "shipped in v0.3.3", which read the first version we ran the suite
against as the version it was introduced in. The conclusion held; the reason
did not, and the reason is what a future bump would have relied on.
test_door_c_pins_allow_reserved_false_against_the_guards_default locks both
halves: that the guard still defaults True, without which the override is a
no-op that would pass forever over nothing, and that Door C overrides it.
586 tests, mypy --strict clean, goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V2v1hrDhrff2H3y2TNJHkF
The code already normalizes filenames and titles to Unicode NFC
(materialize.reduce_to_id_grammar, inbox.process_inbox) because macOS/APFS
hands filenames over decomposed, which would otherwise split one visual
title into two generated filenames. The invariant was implemented but not
documented anywhere in the repo.
Upstream §10 adds a concept type carrying a sanctioned way to compute a
value. This lands its FORMAT: the five contract fields for emission and
round-trip, and §10.2's one requirement. No execution — upstream defers
the receipt and verdict wire formats, so there is nothing to build a
runtime against.
Two additions, both additive:
- The five fields (`runtime`, `parameters`, `computation`, `executor`,
`attester`) join `OKF_V0_2`'s emission order as one block, internally
in §10.2's own listing order. Without it they still emit — in `emit`'s
sorted tail, where `attester` precedes `runtime`, alphabetical order
standing in for the contract's own. No bundle that carries none of the
keys changes by a byte, and the v0.1 profiles gain nothing.
- `FrontmatterSchema.required_by_type` expresses "`runtime` is REQUIRED
for this type and no other" — the first rule here keyed off a
frontmatter VALUE rather than a key. It cannot be `required`, which
would demand `runtime` of every document. A type the mapping does not
name carries no extra requirement, which is what keeps it inside §14:
a consumer must not reject on an unknown `type`, so a conditional
keyed on an unknown type stays silent rather than guesses.
Also pinned, measured today: the line-oriented parser cannot read §10's
canonical BLOCK form. `executor` and `attester` both carry a `resource`,
and with no indentation model the second overwrites the first —
`executor.resource` is lost silently, no error. Characterized rather
than fixed: reading that form needs the structured reader (D1b), and a
half-reader that drops half a contract is worse than one that never
claimed to read it. CLAUDE.md gains the invariant that falls out of it:
we emit flow form, never block, or we write bundles we cannot read back.
578 tests, mypy --strict clean, goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KKKMwi7e7PVHoFW6dJK5XP
Placement settled by catalog's own reading of upstream at the pinned commit
3fcbb9f: SS8:509-510 and SS12:773-775 both put `okf_version` in a bundle-root
`index.md` frontmatter block, and SS12 calls it the only place frontmatter is
permitted in an index. Catalog's spec says the opposite about the same file;
that divergence is theirs against upstream, and we conform to upstream.
The value never touches a profile. `OKF_V0_2.index.root_frontmatter` names the
key; the caller supplies the value through a new keyword-only
`root_frontmatter_values` mapping. That keeps V4/V-A5 intact - `okf_version`'s
value tracks the upstream Google version and belongs to catalog (E1), so a
constant here would claim a decision we do not own and would have to be chased
on every upstream release. In the fixture the value is fixture DATA
(`okf-version.txt`), not a literal in our source.
Ordering comes from the policy, not the caller's mapping: a dict preserves
insertion order, so two callers passing the same keys would otherwise emit
different bytes. A key the policy does not name is refused fail-fast, before
any disk mutation. Omitting the argument emits no block at all - SS12 is a MAY
and none of upstream's four reference bundles declares the key.
The block is written only when the index is CREATED, so a re-run into an
existing bundle stays byte-identical (A-E5).
Raw-byte assertions rather than parsed ones, on the committed fixture as well
as on fresh runs: catalog measured that a quoted value fails their shape regex
with exit 1 and that a BOM hides the marker while still exiting 0.
`yaml.safe_load` returns "0.2" either way and strips a BOM first, so a parsed
assertion masks exactly those two defects. Asserting the frozen fixture catches
what a self-comparison cannot - regenerating from a broken emitter moves both
sides together.
A-E6 is now placement-explicit (promised catalog in 99cf987), and separates the
two byte properties: BOM-free is a property of the file, unquoted is a property
of CATALOG'S GATE and not of OKF v0.2 - upstream's own SS12 example is quoted,
so their gate rejects the spec's canonical form.
README gains the upstream-version section it was missing; CLAUDE.md gains the
mechanism behind "no profile hard-codes an upstream version": a profile names a
key, a caller owns its value.
550 -> 559 tests. test_profile_threading's `OKF_V0_2.index is DEFAULT.index`
assertion is replaced rather than deleted: object identity was a proxy for "the
shipped profiles differ in no NAME-bearing field", which is what makes the
synthetic test profile necessary, so the guard now asserts that directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dgkSPjkLpACjMayd9R5jx
Committed by the operator; the design is open and the scope is not. Recorded
in the roadmap with a scoping plan rather than in the v0.2 alignment doc, which
is about OKF v0.2 conformance and is the wrong home.
The plan settles nothing on purpose. It names the fork everything turns on —
whether we are the MCP server (an agent calls our doors, Door B's territory,
ours to design) or an MCP client (a manifest source type, which is normative in
commons' ingest-spec and therefore starts as a request to them, not as code
here). The two share a protocol and nothing else, including their owner.
Constraints written down because each has already refused something: the
one-runtime-dependency rule (an MCP SDK would be the second — stdlib JSON-RPC
or an optional extra), no model calls in the run path (the model may CALL us,
never the reverse), the guard persist gate (MCP is transport, not a bypass),
the network opt-in, and the required ingested_at.
