README, CHANGELOG (under the untagged 0.10.1), CLAUDE.md and the round's
report. The numbers are the census's, not a summary of them.
- 9 714 image rows over 18 403 files and 67 PDFs, two pinned trees: exactly
35 rows moved, 9 321 of 9 321 JPEG and PNG rows byte-identical.
- The 16 the round did not go looking for: JPEG 2000 objects out of PDF
streams, also unreadable by a model, with no stdlib route to convert them.
They become `asset_not_viewable` -- 16 pictures lost from a bundle, 16
statements gained that a picture stood there. Stated in the README, the
CHANGELOG, CLAUDE.md and the report rather than left in a total.
- The accounting gate: R761 0 -> 19 claimed-and-not-found when the conversion
landed, 0 again after the judge learned its second route; row 6 90 of 111
-> 110 of 111. The gate's verdict is the same `RED: rows 2, 3, 6` as before
the round, with row 3's numbers identical.
- `<!-- asset-viewable-media-types: ... -->` is pinned to
`assets.VIEWABLE_MEDIA_TYPES` by a test, compared as a whole set: a README
naming three of four would pass every containment check and tell a consumer
that a format is refused when it is carried.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chosen: a stdlib BMP reader, because `read_image` is on the CORE path and an
asset's name is its content digest. Measured first, as the order requires:
Pillow 12.3.0 IS in this tree (transitively under `pdfplumber`) and it DOES
decode RLE8 correctly -- a hand-written stdlib decoder and Pillow agree on
19 of 19 of R761's real files, RGB per pixel. So the choice does not rest on
capability. It rests on two properties of this package: `.html` and `.xml`
carry images with no `[extract]` extra installed, so a Pillow converter
either makes a core path depend on an optional binary wheel or buys the
second runtime dependency; and encoding through an installed library would
make a bundle's identity move with that library's version, which is the
property 0.10.0 felled page rasterisation over and `encode_png`'s docstring
already defends. Pillow keeps the job it is good for: the INDEPENDENT decoder
in the tests, on neither side of the conversion.
The defect, measured over the frozen R761 delivery's `assets/`, denominator
50: 29 JPEG, 2 PNG and 19 RLE8 BMP. The 19 are byte-correct files nothing
reads, so 19 figures were present and invisible while `images: N` reported
that they had arrived.
- `VIEWABLE_MEDIA_TYPES` is tested against every asset's SNIFFED type, so it
is a property and not a list of formats we met. WebP is on it and `sniff`
does not recognise one; the limit is stated, not implied.
- `bmp_to_png`: 8-bit uncompressed, 8-bit RLE8, 24-bit uncompressed. All five
RLE8 opcodes. 19 of 19 real files convert with RGB identical to Pillow's
decoding of the source, 2 366 365 pixels compared.
- `asset_not_viewable` and `asset_bmp_unsupported`, both published, both
leaving the concept's "not carried" line.
- Traceability on the pointer's second line, where the rest of the asset
metadata already lives: original media type, original sha256 in full, new
sha256 in full. A converted asset is ONE asset.
- The ceiling is paid on the DECLARATION before a row is allocated, and an
RLE run is one clipped slice -- painting pixel by pixel leaves the memory
bounded and the CPU unbounded.
Two repairs the change forced, each measured rather than assumed:
- `tests/test_assets.py`'s "dimensions absent is absent" used a TIFF, which
is now refused before `read_image` returns. The property still has a
reachable case -- a JPEG whose frame header never arrives -- and uses it.
- `asset_holds` in the accounting gate proved a carry by hashing the SOURCE
file, which a converted image's bundle cannot satisfy. It now also reads
the two digests the bundle states and HASHES THE ASSET ITSELF, so a bundle
claiming a conversion it did not perform still fails.
`tools/okf_asset_census.py` is the committed instrument for the
known-positive: one row per image, from two pinned trees. It was caught by
the rule it serves -- its first version handed `_pdf_images` the wrong page
object and reported 0 images over 67 PDFs with exit 0. The attribute is
asserted now and a known-positive runs before the sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `if False:` in this diff is a mutation DEFINITION -- a string in
`MUTANTS`, applied only to a throwaway copy of the tree inside the
harness and restored in a `finally`. No branch in this repository is
pinned by it.
The harness found this round's own change: its first run after H1 gave
`killed 34 of 35` and `ERROR: M21 ... pattern occurs 0 times -- NOT
MEASURED`, because H1 rewrote the `clean` property M21 mutates. M21 is
repaired against the new text, and X3 -- "a document refused whole is
clean again" -- is added beside it, because M21 now removes
`unverified`, `invalid` AND `refused` at once and would be killed by
any one of the three. Final run: killed 36 of 36, 0 survived, 0
errors, exit 0.
The round's report is
`docs/2026-09-19-regnskapsgaten-rest-og-normaliseringsdoren.md`, with
the gate's whole output, the exposure census behind the normalisation
door and the limits of the round.
CHANGELOG: this round's entries are folded into the UNTAGGED `[0.10.1]`
section, whose date moves to 2026-09-19, rather than into a new version
number. `v0.10.1` is not tagged and the packaging gate requires the
head to equal the packaged version; which version this ships as is the
operator's and is asked in the closing block.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README gains the partial-refusal paragraph and the soft-hyphen door
with its measured reach; CHANGELOG gets an Unreleased section over
this round; CLAUDE.md's open commitment "R761 is refused whole by
guard 1.4.0 ... that is the security repo's question" is replaced by
what was decided and built, with the alternatives that were felled and
the numbers behind each rule.
No version bump and no tag: `v0.10.1` is not tagged yet and releasing
is the operator's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 3 of the 0.10.1 review, and the finding is the pattern the three rounds
share: each bound an OUTPUT, and the bomb stepped one link along. The
declared size, then the first `FlateDecode`, then every `FlateDecode` -- and
then a link this package had documented as safe.
`ASCII85Decode` was classed as bounded "by its own input because it shrinks".
It quadruples: `z` is the shorthand for four zero bytes. And the output was
never the cost -- `base64.a85decode` appends one 4-byte object per group to a
list, about a hundred bytes of memory per byte of INPUT (101.4x at 1 MiB,
96.1x at 4 MiB, 94.5x at 16 MiB on CPython 3.14).
Paired subprocesses, idle machine, both sides from PINNED trees, the document
built once by a third process and read from a file because `ru_maxrss` never
falls and `b"z" * 64 MiB` alone costs 171 MB:
[/Fl /A85] z x 32 Mi 33 475 B CARRIED 3 261 599 744 -> too_large 42 070 016
[/Fl /A85] z x 64 Mi 66 090 B CARRIED 6 461 558 784 -> too_large 40 280 064
[/A85] z x 8 Mi 8.4 MB CARRIED 933 085 184 -> too_large 62 484 480
[/Fl /A85 /Fl] z x 32 Mi 33 488 B samples_invalid 3 519 180 800 -> too_large 43 438 080
The picture was CARRIED in three of the four: not a bound that fired late, no
bound at all. Doubling the `z` run trebles the old cost and leaves the new one
where it was.
WHY THIS FORM. `assets.MAX_FILTER_DECODE_BYTES` (512 MiB) is what decoding ONE
link may cost -- a separate number from `MAX_IMAGE_BYTES`, because that one
bounds the picture and this one bounds producing it. `FlateDecode` is measured
as it is paid; every other permitted filter carries a MEASURED cost ratio
(`assets.PDF_FILTER_COST_RATIO`) checked against its input BEFORE its decoder
is called, since those decoders take a whole string and return a whole string.
A filter with no ratio is refused unread. The budget TRAVELS: a deflate link
is inflated under the smaller of the picture's bound and what the next link's
decoder may be handed, or `[/Fl /A85]` pays 256 MiB for a refusal.
A chunked ASCII85 decoder written here was the alternative and was FELLED: it
would bound `_check_stream_cost` and not the run, because `stream.get_data()`
decodes the whole chain again with pdfminer's own decoder, and it would make
this package rather than pdfminer the authority on an image's bytes. The cap
is the only number that bounds that. `resource.setrlimit(RLIMIT_AS)` was
MEASURED before anything was built on it, as the order required, and is not
usable: Darwin 26.6.2 raises `ValueError: current limit exceeds maximum limit`
and does not enforce it. No child-process cap exists.
THE CAP IS READ OFF THE CORPORA, the posture `MAX_IMAGE_PIXELS` has: over the
9 668 image objects of the 77 PDFs on this machine, 16 decode through an
ASCII85 link and the largest input to one is 450 739 bytes, against a cap of
about 5.0 MB.
A PROPERTY TEST REPLACES THE LIST OF KNOWN SHAPES: every chain of length 1-3
over the ten filters pdfminer decodes, 1 110 of 1 110, both payload fills,
each delivered under the bound or refused with a published code and never paid
for on the way (`tracemalloc`, which counts allocations and is not disturbed
by load). Known-positive beside it: 258 of 258 chains over the permitted
filters still carry a small image.
MAJOR -- the backstop had no test. `check_payload` at the end of
`_check_stream_cost` could be deleted with the whole suite green, because the
second one after `get_data()` gives the same code one step later. The two
differ in whether the payment was made, so the test asserts `get_data` was
never called.
10 OF 10 MUTANTS KILLED, control green, each killer named in the report. Four
survived a first pass and two tests exist because of it.
NOT ONE PICTURE CHANGES HANDS, MEASURED BY NAME: `_pdf_images` over every PDF
on this machine from both pinned trees -- 9 306 -> 9 306 carried over 77
files, 50 -> 50 on R761, 0 of 78 files moving a count and 0 moving a code.
R761 also settles a question raised while this order was open: 50 objects, 29
[/DCTDecode], 21 [/FlateDecode], 0 ASCII85 links -- so round 2's count of 580
`[/FlateDecode /ASCII85Decode]` objects is reproducible from nothing on this
machine. It changes no decision; a bomb shape does not need a corpus.
Version stays 0.10.1, no tag. README, CHANGELOG, CLAUDE.md and errors.py
corrected TO what the code does; the round-2 report carries a correction block
rather than a rewrite. Report:
docs/2026-09-18-utgangsbudsjett-per-ledd.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two findings of the 18.09 PM checkpoint of `0f308c1`. Red tests landed
first in `3b587ea`; this is what turns them green.
BLOCKER -- `_check_inflated` read `filters[0]`, measured that one link and
returned, which is not a bound: a PDF decodes a stream through a LIST of
filters. Measured in paired subprocesses from two pinned trees, idle machine:
[/FlateDecode] 400 MB 408 516 B 59 232 256 -> 62 017 536 B
[/FlateDecode x2] 400 MB 1 636 B 886 554 624 -> 52 367 360 B
[/FlateDecode x3] 400 MB 1 070 B 889 393 152 -> 61 390 848 B
[/FlateDecode x2] 1,2 GB 2 927 B 2 567 204 864 -> 60 403 712 B
542 000x the file at two links, and the picture WAS refused at the end -- by
`check_payload` after `get_data()`, once the memory was spent. The single-link
row is the control and does not move. It also left the 16 corpus objects behind
an `[/ASCII85Decode /FlateDecode]` chain unmeasured, since `filters[0]` is not
`FlateDecode` there.
`_check_stream_cost` walks every link. THREE CLASSES and no fourth
(`extract.bounded_pdf_filters`, pinned by a test): `FlateDecode` MEASURED, a
link with another expanding link behind it inflated under the same bound and
handed on; `ASCII85Decode`/`ASCIIHexDecode` bounded by their own input because
they SHRINK; `DCTDecode`/`JPXDecode`/`JBIG2Decode` PASS THROUGH. Everything
else -- `LZWDecode`, `RunLengthDecode`, `CCITTFaxDecode`, `/Crypt`, anything
written later -- is refused UNREAD with a new code `asset_pdf_unbounded`,
decided before the FIRST link is decoded so a document cannot make the run pay
for the links in front of the one we cannot bound. An encrypted stream is
deciphered and then measured, where `stream.decipher is not None` used to
return unmeasured; 0 of 5 142 objects here are in an encrypted document, which
is why nothing caught it.
NOT ONE PICTURE CHANGES HANDS, AND IT IS MEASURED BY NAME. Every PDF on this
machine -- 78 documents, K2 in both trinn1 and trinn2, the shipped fixtures and
R761 -- run through `_pdf_images` page by page from both pinned trees:
images carried 9 356 -> 9 356
documents losing one 0 of 78
documents gaining one 0 of 78
asset_pdf_unsupported 322 -> 314
asset_pdf_unbounded 0 -> 8
The 8 are the 4 `CCITTFaxDecode` stencil masks (`/ImageMask true`,
`/BitsPerComponent 1`), counted twice because trinn1 and trinn2 hold the same
document. They were refused before and are refused now, one step earlier and
under a code that says why.
MAJOR -- `check_payload(len(data))` after `get_data()` is the counted refusal
four documentation surfaces point at, and deleting exactly that line passed all
2 132 tests. It is reachable through a stream pdfminer has ALREADY decoded
(`decode()` sets `rawdata` to `None`), which is now the ONLY case outside the
bound and has a test.
Eight mutations, one line each, every one DEAD, with the unmutated tree run
first as the control: first-link-only, loop dropped, inequality reversed,
encrypted skipped, backstop deleted, unknown filter passed through,
intermediate link not carried forward, whole check removed.
`tools/okf_accounting_gate.py` gains one line, the new code in
`REJECTION_CODES` -- what a rejection code requires and nothing more. Gate
unchanged: exit 1, GATE RED rows 2, 3, 6. Version stays 0.10.1, untagged.
Suite after `git add` against a clean tree: `uv run pytest -q` ->
2152 passed, 1 skipped (226 s). ruff, ruff format --check, mypy --strict clean.
Report: docs/2026-09-18-filterkjeden-og-backstoppen.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A second independent review read `230d1cb` -- the commit that closed the
`v0.10.0` review's two MAJOR findings -- and found one of them open. The
bound read `/Width` and `/Height`, which an untrusted document writes, while
`get_data()` pays for the stream beside them; `/Length` is the COMPRESSED
length and the two numbers are independent.
Re-measured here on `ed8d9d7` before anything changed, in its own
interpreter: a 408 516-byte PDF declaring 1x1 and carrying 400 MB of deflated
zeros was CARRIED, no rejection, 891 904 000 B peak RSS. After: 0 carried,
`asset_too_large`, 57 065 472 B. At 1,2 GB of zeros, 2 436 MB -> 64 569 344 B
-- the cost no longer scales with the bomb. End to end through the CLI with
the shipped defaults: 838 000 640 B and an asset written -> exit 0,
79 650 816 B, `0 carried of 1 found`, no `assets/`.
Three numbers are bounded now, not one: what a container DECLARES, what a
carried FILE measures (`read_image`, so a 49 MP PNG of 47 705 bytes is not
passed on to a consumer), and what a PDF stream DECOMPRESSES to
(`assets.inflated_size`, a chunk at a time, output discarded, before
`get_data()`). The limit is stated rather than implied: the stream
measurement runs where `FlateDecode` is the first filter and the document is
not encrypted; every other chain is a check on the decoded length AFTER the
decode, a counted refusal and not a bounded one.
A non-positive declared dimension is `asset_size_invalid`, its own code,
raised before the stream is read. `-1 x 40000000000` is a NEGATIVE pixel
count, under which every `>` bound read as satisfied, so the check returned
silently and the refusal arrived from `encode_png` as
`asset_samples_invalid`. Its own code because a publisher shipping a picture
bigger than this package carries and a dictionary written to be read wrong
are different facts about a document.
Two smaller findings in the line that says what is missing, both introduced
by the first fix: the address was written twice, once bare, and a linkifying
renderer autolinks a bare URL -- written once now, in one code span; and
`label` became a dead parameter, so the figure's caption was dropped, a
regression against 0.10.0. It is written again in the `-- <label>` form a
carried pointer uses.
Version bumped to 0.10.1 across all ten places. Nine were unbound and stale:
four README install lines naming the previous release, two prose lines, the
"current tag" entry, `uv.lock`, and a CHANGELOG whose 0.10.1 content sat
under `[Unreleased]`. Two new packaging tests bind them to `__version__`, and
the README's guard tag to `[tool.uv.sources]`.
Every test was red first. The fate of every image is identical with and
without the new bound on three K2 PDFs carrying 800 images (464/464, 311/311
with the same 12 rejections, 25/25), and the second inflate is below the
noise floor there. 0 shipped artifacts move: no bundle under `examples/`,
`skills/` or `tests/fixtures/` carries an image pointer at all, measured
against a known-positive control.
`asset_too_large` was undocumented in the error registry; both codes are
there now. `tools/okf_accounting_gate.py` gains the new code in its closed
list -- one string, no behaviour change, stated because that file belongs to
another order.
Suite 2141 passed / 1 skipped, ruff + format + mypy --strict clean.
Report: docs/2026-09-18-bildestien-holder-0-10-1.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The content accounting stays OPT-IN until the losses it reports on the
reference corpus are fixed; a default-on door would fail builds that pass
today. Of the three exceptions the gate proposed, only the PDF one
(heading, paragraph, table) is approved. Approving it moves no number,
because no witness counts a heading in a PDF: what it changes is that the
gap is a stated limit of the instrument rather than an open question
about the build. An image in a workbook, or in md/txt/csv/json/odt/rtf,
stays unaccounted and therefore stays red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two MAJOR findings of the independent v0.10.0 review, both with the
shipped defaults, both new in 0.10.0. Repros rebuilt as tests first.
- A remote <img src>/xlink:href became a LIVE markdown image link in the
persisted concept, with the address and query string chosen by whoever
wrote the document. Extraction opens no socket; a consumer rendering
the bundle does. Now inert text with the address in a code span,
pinned by a property over the readers rather than by one string. The
tier asymmetry (user-upload refuses, trusted-source persisted) went to
the guard repo with the repro.
- Nothing bounded a declared image size: 9.6 KB of PDF declaring
3000x3000 grayscale zeros took 83 MB peak RSS, linear in pixels.
MAX_IMAGE_PIXELS (40 000 000) and MAX_IMAGE_BYTES (256 MiB) are read
off the corpora (largest measured 18.6 MP on K2, 1.4 MP on R761) and
checked on what the container declares, before any decompression;
over them is asset_too_large, counted. The same bound closes the
inline data: URI, which the review flagged and did not measure.
Also fixed, added by PM to this order: an inline PDF image was named
from id() of a Python object, so two concept files of the reference
corpus differed between builds. It is now named from its position.
R761 unchanged: 50 carried of 50 found, assets diff -rq clean.
Report: docs/2026-09-17-bildestien-0-10-1.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
okf build --accounting PATH inventories every source document before
extraction, in the gate's per-format vocabulary, and after the run gives
each element one fate (carried / pointer / coded rejection), written as
JSON and summarised in log.md. "carried" is checked against the written
concept bodies, so a gate that drops a line is found (test). Exit 1 on
anything unaccounted or double-booked. Opt-in: +744 s (+19 %) on the
43-document reference corpus, and that corpus fails the check on 24 real
losses (22 images on text-less PDF pages, 2 docx Title paragraphs).
Changed without the flag:
- okf build exits 1 when it extracted documents and persisted none.
Door B and corpus.measure are unchanged. One test relied on exit 0.
- An image file carried through a persisted document is its own K1b
column, no longer also extractor_unknown. The set is what the resolver
actually carried (ExtractedDocument.files), never a byte match.
tools/okf_accounting_gate.py (checks untouched) is green on all six rows,
R761 110 of 110 under both gates.
Report: docs/2026-09-17-innholdsregnskapet-bygget.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now no reader in this package fetched, named, described or copied a
single image. `<img>`'s attributes were never read, a NISO-STS `<graphic>`
was walked past, a PDF was opened for its text alone, the converter's
markdown writer dropped every picture, and the only writer into a bundle
took `content: str`. The two lossiness warnings said so on every run, which
made the loss honest and did not make it smaller.
Measured on R761 Prosesskoden:2025, published as a 701-page PDF and as a
NISO-STS delivery: the process text is carried in full while 12 `Tabell N-N`
and 9 `Figur N-N` captions stand over nothing, because that publisher ships
those tables as raster pictures in both. Process 84's "toleranseklasse ...
er gitt i tabell 84-2" points at empty space.
THE GATE WAS WRITTEN FIRST AND RED. `tests/test_asset_gate.py` reads its
denominator out of the source (`page.images`, `word/media/`, `ppt/media/`,
`<img`, `<graphic`), never from a constant here. Measured at 332961a, built
from `git archive` and not from the editable tree: carried 0 of 8 local
images across 5 documents (9 declared), and no `assets/` at all. After: 8 of
8, with the ninth a remote source carried as a pointer without a file.
FIVE READERS PLACE, ONE MODULE DECIDES. `assets.py` owns what an image is
(sniffed from the bytes, never from the claimed extension), what it is
called (`<sha256[:12]>-<the source's own basename>`) and how it is pointed
at (one two-line block, one regex). `.xlsx` is deliberately not a row: a
block inside its pipe tables would break the `source_rows` locator, and 0 of
4 K2 workbooks hold media.
A PDF stream that is already a file is carried VERBATIM (29 of R761's 50
objects are DCTDecode); raw samples are encoded to PNG with stdlib zlib, so
no new dependency. Rendering the page region was the alternative and was
felled on determinism: a rasterised crop's bytes, and therefore the asset's
content-addressed name and the bundle's digest, would depend on the
installed rasteriser. What the encoder cannot express exactly is refused
with a code and counted, never approximated.
NO SIZE FLOOR, and that is a measurement: over the 4 828 image objects of
the K2 corpus the size distribution is a broad spread with no gap, unlike
OCR_CID_SHARE's bimodal one, so a threshold would be a number we chose.
ON BY DEFAULT, AND THE CONTROL IS TWO WHOLE BUILDS. The 43-document
reference corpus at 332961a versus rebuilt at HEAD with `--no-assets`:
865 files on both sides, `diff -rq` reports ONE difference, the added
`Images: NOT CARRIED` line in log.md. Every concept byte-identical.
Against the default: 453 -> 454 concepts, 865 -> 867 md, 0 -> 2 964 assets
(2 964 carried of 3 145 found, 4 622 pointers), 4.7 MB -> 115 MB, 2 414 s ->
3 088 s, peak RSS 6.26 -> 8.74 GB, 422 of 865 md files differ. The one new
concept has a measured cause: the pointers are body text, so a section
holding 146 of that document's images grew from 19.0 % to 30.6 % of the
extracted text and crossed `--outline-gate`'s 0.20 share clause.
THE IMAGE BYTES ARE NOT SCREENED. The guard is text-only, the pointer block
passes the gate as body text, the picture beside it passes nothing, and
log.md says so on every run.
Also fixed, both found by measuring rather than by reading:
- a markdown image is no longer read as a cross-reference. `structure._LINK`
never looked at the character in front of the bracket, so every pointer
would have arrived in the index as an edge to a concept that cannot exist.
- Door C carries the assets its merged concepts point at. Before this,
importing a bundle built with `--assets` merged 6 of 6 concepts and wrote
no `assets/` at all, so every pointer named a missing file.
Report: docs/2026-09-17-bilder-i-bundlen-trinn1.md
Spec proposal: docs/plan/okf-assets-section-6-4.md
Suite 1 955 passed / 1 skipped (from 1 896), ruff and mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects reported from outside by claude-code-llm-wiki (order
20260915T202332Z-228694739), both reproduced against this tree before
anything moved.
F1 -- the packaged CLI never ran the guard. corpus.measure wired an
unconditional approve-everything stub into process_inbox and 0 of 90
add_argument calls named a gate, so the one path most people use screened
nothing while pyproject.toml made the guard a mandatory runtime dependency
and the README recommended a composition the command line could not reach.
--gate takes guard-trusted-source (default), guard-user-upload or none.
corpus.resolve_gate is the one name->callable map, with the guard imported
lazily so importing the package still does not pull it in; an unknown name
RAISES rather than falling back, because a fallback reproduces the defect
with an extra step. The gate's NAME goes into the section 9 log.md -- a
stub is only dangerous when nothing downstream can see it -- and --gate
none renders NOTHING WAS SCREENED.
The default was chosen on a measurement: over the 453 concept bodies of
the pinned reference bundle, PRESET_TRUSTED_SOURCE persists 453 of 453 and
PRESET_USER_UPLOAD holds 1, costing that concept's whole source document.
Neither tier waves anything through -- an invisible carrier and a CRITICAL
finding fail secure at both. Door B's library default is UNCHANGED at
PRESET_USER_UPLOAD: an inbox drop is an untrusted upload, an operator
pointing this command at their own folder is not. The second tier ships as
guard_adapter.inbox_gate_trusted_source, the three-line adapter that
module's docstring already described, never a preset parameter.
process_inbox(segmentations=..., gate=inbox_gate) now has a test. Before
this, `grep -rl inbox_gate tests/` gave 1 file with 0 occurrences of
`segment` -- the recommended composition was untested, which is how the
defect survived.
F2 -- a fenced code block declared structure. `# Use the opus[1m] alias`
inside a ```bash fence became a level-1 ATX heading: the document was
refused entirely where the line carried [ or ] (5 of 191 pages of the
reporter's corpus), and the concept TITLE came from somebody's shell session
on 62 of 191 (32.5 %). The fix is in the proposer and never in Door B's
title rule -- that rule is right, and a heading that was never a heading is
what has to stop being proposed. propose.fenced_lines is computed once per
text and no rule reads a fenced line, including Arm D's outline RUN, which
selects from the whole line list. Backtick and tilde fences, three leading
spaces, a closing fence at least as long as its opener, and no backtick in a
backtick fence's info string -- that last one keeps a line holding only
`okf build` from silencing a document.
MEASURED ON THE BYTES, and this is the number that decides: the 43-document
reference corpus built at b6da09c (from git archive, never the editable
tree) and rebuilt at the shipped defaults differ in log.md alone, by the one
added bullet. 865 concept files on both sides, every concept byte-identical.
Found by that control and NOT caused by this work: the pinned artifact
K2-bundle-default-20260912 was written 2026-09-09, two days before ed0418f
changed title: quoting, so it differs from what HEAD produces on 42 concept
files. test_default_bundle_pin stays green because it pins the count and the
hit@8 ranks, not the bytes. Re-pinning is the operator's call.
Suite 1896 passed / 1 skipped (+27 from 1869). ruff, ruff format and mypy
--strict clean. No version bump, no tag, no push.
Report: docs/2026-09-15-f1-f2-gaten-og-kodefencen.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bundle-only gate returned UNMEASURED and exit 3 on the very arm it was
built for: no metric computable from a bundle alone reaches boundary recall.
`boundary_share` -- declared boundaries that became a concept, over declared
boundaries -- is the one metric measured that orders the arms correctly, and it
needs the publisher's own structure, so it arrives as an input.
Measurement first, threshold after, which is what the order asked for.
P1, the normalisation, derived rather than guessed: stripping all whitespace
and lowercasing reproduces the fasit's own `norm` from its own `title` on
2 761 of 2 761 rows (alphanumerics-only scores 58). P1's own bar is 99 % on the
known-good arm and the literal reading of it reaches 22 of 2 761 -- not because
the normalisation is wrong but because okf's default route moves the numbering
token a publisher glues into a heading over into the concept id. The pair form
(concept's own directory, residual title) reaches 2 737, either reaches 2 759
(99.9 %). Both forms ship and neither is a fallback: `r761-2025-d1` is the
control in the opposite direction at 2 727 literal, 0 paired.
P2, the single corpus, is in the OUTPUT and not only in the document: the bar
is declared `corpora = 1`, every boundary row prints `N = 1 corpus`, and the
line states that `--fasit` is the caller's ASSERTION that this bundle is a
build of the document the fasit describes -- the posture `okf consume --ref`
has. Measured: the K2 reference and `n100-2023` score 0 of 2 761 and read FAIL,
which is the assertion being wrong rather than the bundle being bad.
One bar, at the pinned artifact's own value, 2 759/2 761. It is tight and the
cost is published rather than tuned away: 2 of 4 R761 builds fall under it
(2 752 and 2 727), while any bar between 41.6 % and 98.8 % separates the
known-bad arm from every R761 build measured. The known-bad arm
(`860019-mdb-100`) is 1 148 of 2 761 -- FAIL and exit 1, where the bundle-only
gate gave exit 3.
A fasit is validated at the door: not a list, a row missing `title` or `norm`,
or anything that is not JSON exits 2 with the reason, never a quiet UNMEASURED.
A fasit under five rows is UNMEASURED -- the document floor in the fasit's own
unit.
Without `--fasit` the command is byte-for-byte what it was, held by a test.
`okf check` is untouched; no version bump and no tag. 17 tests red on
assertions before the implementation, the two new doc pins each driven red and
back. Suite 1 869 passed / 1 skipped / 1 870 collected (base 5e5d01c: 1 851).
docs/2026-09-12-g37-terskler.md SS 7 carries the premises re-measured, the
seven bundles, the interval any bar could sit in, and the honesty limits --
including the correction of SS 2's own grep claim, which went false in the
commit that wrote it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Version to 0.9.0 in pyproject.toml, __init__.py and uv.lock; CHANGELOG's
[Unreleased] becomes [0.9.0] - 2026-09-13, carrying G37's `okf quality` and
K3-26's README file-type table byte-identically. MINOR rather than patch: the
release adds a command.
README's four install lines and the two tag-pairing sentences move to v0.9.0,
and the tag history gains a `v0.9.0` entry stating what ships -- three
verdicts, two bars (.pdf 8/32, .docx 2/5), both regression bars against the
pinned reference and not quality claims, `okf check` untouched at seventeen
rules.
No behaviour change in src/ beyond the version string. The guard pin stays at
v1.4.0 and the dependency floor stays at >=1.2,<2.0. Suite 1 850 passed, 1
skipped, run after `git add`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
G37. `okf check` is a CONTRACT check and a green one is not a quality gate:
measured 2026-09-10 by `vegnormal-okf`, three arms over one corpus all
returned 0 findings and exit 0 while their hit@k ranged from 6 of 6 to 0 of 6.
`okf quality <bundle>` asks the other question, per file type, with the
denominator on every line. A separate command rather than `okf check
--quality`, because the two answer different questions and a caller must not
be able to read one as the other. `okf check` is untouched.
Three verdicts and no fourth -- PASS, FAIL, UNMEASURED -- and a type with no
measured threshold is never PASS. Exit 0 judged and clean, 1 at least one
FAIL, 2 did not run, 3 nothing could be judged: exit 0 over a table of
unmeasured rows would be the silent pass this command exists to stop.
Two bars today, both `structure_null_share` (documents of a type yielding
exactly one concept), read off the pinned 43-document reference bundle: .pdf
8/32, .docx 2/5. Plus one definitional bar for every type, taken from the
harness's own degenerate-merge definition: 0 concepts with an empty body,
measured 0 of 8 602 concepts over four bundles. A bar needs five documents on
BOTH sides -- its own and the judged bundle's -- so .xlsx (2), .xml (1) and
every type with no corpus class in `extract._EVIDENCE` are UNMEASURED and
print their numbers without a verdict.
The floor on the judged bundle was found by RUNNING the gate, not by reading
it: one PDF cut into 2 182 concepts scored 0 of 1 against the 32-document
reference and read as PASS.
The gate walks the index tree and never a directory (SS 9.2; controlled
against the listing on four bundles, 453 / 2 761 / 3 206 / 446 either way),
and prints the bundle's own run log beside its counts -- a document rejected
at extraction leaves no row in the bundle, so the pinned corpus's 33 PDFs
show up as 32 and the two denominators must never be read as one.
Three of the order's five premises moved when re-measured, and they are in the
document rather than glossed: the four evidence corpora carry `source_file` on
0 of 446, 0 of 1 133, 0 of 270 and 0 of 2 756 concepts, so they name no file
type and cannot PASS; "41,6 %" is `vegnormal-okf`'s number and not in this
repository; and the same 828-document bundle carries two published hit@k
figures from two question sets.
Three candidate metrics measured and NOT shipped: duplicate titles within a
document (0 of 3 206 on the known-bad arm against 349 of 2 761 on the
known-good one) and short concepts (5.6 % against 14.6 %) order the two arms
the wrong way round; duplicate titles across the whole bundle order all four
correctly (37.8 / 16.3 / 12.6 / 5.7 %) and still ship without a bar, because
any bar separating them is read off the two bundles it would judge.
19 new tests, each rule exercised in both directions; the three README pins
were each driven red before being kept. Suite 1 850 passed, 1 skipped, 1 851
collected, run after `git add` -- +19 against a base of 1 832 collected,
measured on the stashed tree (STATE's 1 831 is one short of that).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`### Binary extraction` carried its own six-row Format/Reader/Evidence table
over the same rows the pinned table now holds. It was true when written and
reachable by exactly the failure this module exists for: three evidence classes
copied into prose no test reads. That section now points at the pinned table
and keeps its prose about the extra.
A fifth assertion in `tests/test_docs_promises.py` holds the duplicate gone.
Known-positive: the same query finds 8 table lines in that section on the
previous commit, so it can go red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The extractor registry reads 13 extensions. The README's opening line named
five of them, and the full list existed only in a hidden
`<!-- extract-formats: ... -->` comment, which no reader reads -- so the README
undersold what the code does and stated no evidence class anywhere a consumer
would look.
A `## Supported file types` table now carries one row per extension: reader,
dependency (core or the `[extract]` extra), the evidence class `_EVIDENCE`
records for the row, and one honest note. The three `constructed` office rows
carry their denominators (N = 1, N = 2, N = 1) in the table itself, so a row
that has met no document anyone wrote cannot read as a supported one; `.htm`
does not borrow `.html`'s 828-file class, because the code records none for it.
A `Not read today` section states the absences (`.doc`, `.epub`, `.eml`/`.msg`,
image files, source files, `.one`/`.vsd`) as facts, not as a queue.
Test first, red before the table existed: four assertions in
`tests/test_docs_promises.py` pin the table's row set to
`_CORE_EXTRACTORS | _OPTIONAL_EXTRACTORS`, each evidence cell to `_EVIDENCE`
(and to a fixed `stdlib, no corpus class` where the code records none), the
core/extra split to the registries, and the opening to the table.
No change to `extract.py` and no version bump: nothing about what is read
moved, only what the README says about it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Version to 0.8.5 in pyproject.toml, __init__.py and uv.lock; CHANGELOG's
[Unreleased] becomes [0.8.5] — 2026-09-12, carrying K3-23's instrument,
K3-25's default and K3-24's block-sources fix byte-identically, with the
two `### Changed` headings merged into one and one post written here: what
`okf.parse_frontmatter` now returns for a block `sources:` is a READING
projection PyYAML reads back on 0 of 4 605 measured block files.
No behaviour change in src/ beyond the version string. The guard pin stays
at v1.4.0 and the dependency floor stays at >=1.2,<2.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`link_in_signal` defaults to `False` on all three entry points --
`searchable_text`, `concept_scores` and `build_payload` -- through one named
constant, `DEFAULT_LINK_IN_SIGNAL`. This carries out K3-23's recommendation
(alternative c), and its numbers are the whole warrant: of the newcomers the
door's `Enclosing section:` line ever added a question token to, 39 of 39
gained it from the bundle-absolute PATH and 0 of 39 from the link's title, and
every token the path contributed is a segment of the document's own directory
-- the saturation `shared_id_prefix` takes out of the id signal, back in
through the body.
A CONSTANT rather than three naked literals, chosen against the module: the
other eight defaults here are constants (`DEFAULT_PROFILE`, `DEFAULT_LIMIT`,
`DEFAULT_TIE_SHARED_RANK`, `DEFAULT_STEM_PREFIX`, `DEFAULT_TITLE_COVERED`,
`DEFAULT_SOURCE_QUOTA`, `DEFAULT_FOLLOW_PARENT`, `DEFAULT_K`) and this was the
one default spelled out three times. It also lets a test fall on the rule from
the module that owns it.
NO CLI flag is added and none is changed. `grep -n '"--.*link'` over `src/`
gives 0 hits before and after, and `consume.main` never passed the parameter,
so the CLI inherits the function default. A flag was considered and NOT added,
because no caller could be named that needs the older reading: the three call
sites outside `consume.py` (`skill.py:360`, `tools/okf_consume_measure.py:155`
and `:247`) all inherit it deliberately.
Measured before landing: 0 of 5 shipped bundles carry the door's line
(denominators 446, 1 133, 270, 2 756, 2 761 concept files) and 5 of 5 payloads
are byte-identical across the move. The consumer repository was read only and
its tree is unchanged, listing identical including mtimes.
The docstrings move with the code rather than after it. `concept_scores` said
this was "an INSTRUMENT, not a flag and not a default" and `build_payload` that
it "carries no CLI flag"; the first is now false and the second still true, and
each says so. `cli.DEFAULT_SHELL_PARENT` keeps its value and loses its reason:
the cost it cited -- delivered sets moving through that path -- is what this
commit removes, so the comment now states why it stays off NOW, which is N = 1
document and a consumption half nobody has judged. README and CLAUDE.md carry
the same sentence where they already describe the line, measured rather than
written on reflex: neither said anything about the ranking before.
One test changed with the code. The characterisation of what the LINE does
called `concept_scores` with no parameter, because the reading it characterises
used to be the default; it now names that reading. Same fixture, same number.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The measurement round 21 deferred. On R761, of the 39 newly delivered concepts
the door's link line ever added a question token to, 39 gained it from the
bundle-absolute PATH and 0 from the link's title; every token it ever
contributed is a segment of the document's own directory. So the cost is not
the link, it is the path inside it -- and only that statement points at a fix.
Rank movement and budget displacement are separated on one bundle by three
readings plus a scratch-only fourth: X vs Y is ranking, X vs W is bytes, and
Y equals Z on 16 of 16 rows, which is what makes the separation measured
rather than assumed. hit@1/8/50 stays 6/6 and the known-positive at rank 1
under every reading.
`--shell-parent` stays off at the current link form: two of three acceptance
conditions fail. The third exit is measured with the same numbers and
recommended -- reading the body without the line moves nothing on the 5 of 5
bundles anyone ships, and would make the flag free.
The link cost is 71 616 B / 4.45 %, not round 21's 72 265 B / 4.49 %; the rule
behind the number is stated, and round 21's file is left standing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`_render_sources` gave three measured reasons for refusing to emit the block
form. Two of them fell with `28f9a4b` and are struck rather than left
standing: a block list no longer round-trips to an empty value (4 609 of
4 609 concept files), and B6's acceptance test can now pass.
Reason 2 STANDS and now carries the emission rule alone, re-measured by
reading `portfolio-optimiser` at `6eb58e5`: `read_provenance` returns
`UnreadableProvenance(reason="block-sequence")` for a block value. It is not
the guard's objection -- guard 1.4.0 reads the block form on 4 609 of 4 609
of the same files -- so the answer to "should our own emitter write block" is
NO, with the number that carries it, and the measurement that would make it
decidable is named in STATE rather than deferred without one.
The README paragraph said this library "cannot read the block form" where one
reader could and one could not. It now separates the two by KEY: `sources` in
either form through every reader, every other block mapping still refused,
with the D1b pointer intact.
Report: 222 lines, S7 form. Five deviations first, including the acceptance
sentence naming THREE flat readers rather than one, a `title` denominator of
7 372 rather than 5 372, and the `okf check` rule answered NO with a
measurement (`contract_check` holds 0 occurrences of `sources`, and
`consume` already writes `sources_unreadable` for the state a rule would
restate; `len(RULES)` is 17). Honesty limits name the one-key-wide set, the
block `verified:` fixture still reading empty, and the rendering not being
byte-identical to the source.
Suite 1807 passed / 1 skipped, rc 0, 98 s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Version 0.8.3 -> 0.8.4 in pyproject.toml, __init__.py and uv.lock, and the
guard pin in [tool.uv.sources] moved from v1.3.0 to v1.4.0 (uv.lock follows
via `uv lock`: the guard's version, source and requires-dist lines, nothing
else). CHANGELOG [Unreleased] closed as [0.8.4] with K3-21's and K3-22's
entries moved as written, plus one entry for the guard pin: 1.4.0 parses a
flow sequence of plain scalars that 1.3.0 refused (26 of 28 frontmatter
blocks refused by 1.3.0, 0 of 28 by 1.4.0, on one five-document bundle).
Dependency range llm-ingestion-guard>=1.2,<2.0 unchanged.
README: the four install lines and llms.txt's one to @v0.8.4, the guard
pairing prose to v0.8.4 / v1.4.0, the pip fallback to @v1.4.0, a v0.8.4
row as the current tag (seventeen checker rules), and the suite count
(1783 collected, 1782 passed, 1 skipped) with its history extended.
No behaviour change in src/ beyond the version string.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The record for K3-21 A-D: what an excerpt, a heading-only body, the checker
and the index now carry, and every gate with its denominator -- the pinned K2
bundle (7 passed in each export), 32 regression payloads, R761 with and
without `--shell-parent` (hit@1/8/50 6/6 at both k, the known-positive at
rank 1, 675 of 710 shells with exactly one link and 0 broken, 710 of 710
pointing at the source's own ancestor after D), consumption 6 of 6 in both
forms, the byte cost against the consumer's 51.0 %, the ranking cost the
link's path carries, and the 35 shells with no ancestor holding text, listed.
Deviations from the order stated first; honesty limits last.
CHANGELOG under Unreleased; README and CLAUDE.md say 710 of 710, 17 rules,
and that the outline mark keeps its depth. The README's project measurement
was re-run (26 concepts, 0 checkout paths, `okf check` 17 rules, 0 findings).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K3-22's report, CHANGELOG under [Unreleased], and two corrections of a
sentence measured false.
README and CLAUDE.md both said of the flow form "both are valid YAML, and a
real YAML consumer recovers the same structure from either". An unquoted URL
with a query string inside a flow mapping is not valid YAML for PyYAML, and
the quoted form is refused by the pinned guard; the paragraph now states the
limit. CLAUDE.md also records the K3-22 invariant and the new refusal of a
`--frontmatter` flow value with such a leaf.
Measured, fix `ed0418f` against base `0308169`, each tree built twice from
frozen exports: five-document project 0 files moved; R761 XML and HTML 1 line
each; K2 default 42 `title` lines, after which 454 of 454 frontmatters parse
and read back the same (base 413 parse, 412 read back). hit@k on R761 base =
fix, S1-S6 6/6 at hit@1/8/50 at both k, KP rank 1; okf check 32 of 32 payloads
0 findings over 16 rules; K2 pin 7 passed in the export.
Found outside the order: the pinned guard refuses nearly every segmented
concept okf writes, on its scalar flow sequences (`source_offset`,
`references`, `derived`) -- identical before and after, so older than K3-22.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tag K3-19 and K3-20 together: a NISO-STS document's own identity names
its directory and titles its sources entry, --frontmatter KEY=VALUE and
description from a section's first spec point, and the consume fix that
keeps a leading directory every concept id shares out of the first
fusion signal. K3-19 alone dropped the known-positive from rank 1 to not
delivered; K3-20 A is what makes the identity safe to ship.
Version 0.8.2 -> 0.8.3 in pyproject.toml, __init__.py and uv.lock (uv
lock changed that one line only). CHANGELOG [Unreleased] closed as
[0.8.3] with its five entries moved unchanged. Five install lines move to
@v0.8.3, the tag history gains a v0.8.3 row, and the suite count is
re-measured: 1668 collected, 1667 passed, 1 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A plan entry whose span holds its heading alone gets `parent_id` naming the
nearest preceding entry at a smaller level whose own span holds text, passing
over an ancestor that is empty too; the door writes the existing `parent:`
key. Nothing is copied and no boundary moves. The rule reads the plan's level
and order, never the row. Off by default: `okf consume` reads no `parent`
key, so no payload ranks differently, while the flag moves the bytes of every
bundle holding a heading-only section.
Measured before building on one 2 761-concept process code: 710 concepts are
heading-only; the level route names the ancestor the document's own <sec>
nesting names on 708 of 710 (two sit at depth 7, clipped to 6, and point one
level too high), where reading section numbers gets 686 (`12` begins with
`1`); 35 have no ancestor holding text and get none.
The red test expected only concept files to move. The index is a projection
of the frontmatter and shows the key as a facet, so the test now holds both;
the facet renders a segment id as unresolved (`p1?`), because `structure`
reads `parent` as a document number -- named in README, CLAUDE.md and the
CHANGELOG, not repaired here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first fusion signal read a concept's title together with every segment
of its id. On a one-document bundle every id starts with the same directory,
and since K3-19 an STS document names that directory after its own number, so
a question naming the document matched every concept except the one whose
title already named it. Measured on a 2 761-concept bundle, the known-positive
fell from rank 1 to not delivered at the default k (13 at k = 50).
`shared_id_prefix` returns the leading directory segments EVERY id shares,
never the leaf, and the signal reads the id below them. Where the ids share no
prefix the signal reads the same string as before.
Measured on a frozen export before this commit, four forms: the chosen one
gives KP rank 1 at both k with S1-S6 6/6, and K2 (12 payloads), N100/N200/N500
(15) and the five-document folder (5) byte-identical. Dropping each concept's
own document directory instead took a K2 hit@8 row from rank 5 to not
delivered, and is not shipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report for K3-19, measured from a frozen export of 77e7cab: R761 from the
base 56b00e7 against 77e7cab -- directory `14ce59dc-...` -> `r761-prosesskoden`,
2 761 concepts and 710 shells unchanged, 110 = 110, 1 807 descriptions, the
flagged edition and address on 2 761 of 2 761, hit@1/8/50 over S1-S6 6/6 at
both k, `okf check` 4 of 4 with 0 findings, and --ingested-at alone
confirmed on 2 761 of 2 761.
Reported first rather than repaired: the known-positive falls from rank 1 to
not delivered at the default k (13 at k = 50), decomposed to the directory
name alone; and the order's own `sources` value is refused by PyYAML on every
concept while `okf check` accepts it.
CHANGELOG under [Unreleased], the README and CLAUDE.md paragraphs carry the
final numbers, and the README test count moves to 1659 collected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The version moves to 0.8.2 in pyproject.toml, __init__.__version__ and
uv.lock (one line; `uv lock` touched no other package). CHANGELOG's
[Unreleased] becomes [0.8.2] -- 2026-09-11 with the entries K3-15, K3-17
and K3-18 wrote, unedited. The five install lines (README x4, llms.txt)
point at @v0.8.2; the guard pairing prose names v0.8.2, whose own
[tool.uv.sources] still pins guard v1.3.0; the tag history gains a v0.8.2
row and v0.8.1 stops being "the current tag". Test count re-measured on
2026-09-11: 1603 collected, 1602 passed, 1 skipped.
What the release delivers to a consumer on a tag: `okf check`'s sixteenth
rule (bundle_mismatch), which v0.8.1 does not carry, the --title-covered
stop beneath a title that answers more, and the regenerated
skills/okf-consume/. No behaviour change in src/ beyond the version string.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The hand-filled skill predated `okf skill`, declared no bundle identity,
and was refused by the checker it tells its reader to run, against the
payload shipped beside it:
NOT conformant: 16 rules over 3 excerpts and 0 withheld entries, 1 findings
bundle_mismatch: the skill declares no readable bundle identity, ...
It is now the generator's output for
examples/ingest-golden-segmented-okf-v0-2/expected-bundle, the bundle the
payload always came from:
conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings
The payload's bytes do not move. --force and --example-question are both
required: without the question the payload is a different one (spent
2125 -> 1420, delivered 3 -> 2, withheld 0 -> 1) and the byte test fails.
`okf skill` writes the bundle root and the skill path absolute when --out
is not under .claude/skills/, so the checkout prefix is stripped in one
mechanical step that a test holds; the generator is unchanged.
The frontmatter name becomes b-golden-segmented-okf-v0-2-consume. Claude
Code takes a project skill's command from its directory, which stays
okf-consume, and nothing in this repository named the skill by name.
references/README.md is rewritten by hand with the full regeneration
command; living prose in README, CLAUDE.md, the test comment and the
skill.py docstring is corrected, dated journals are left as written.
Report: docs/2026-09-11-k3-runde18-konsumskillen-regenerert.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 16's partition read every concept whose WHOLE title the question
accounts for before everything the fusion ranked above it. That is a claim
about the covered title's PRECISION, and it overrode the fusion even against
a title answering MORE of the question. Measured on a 26-concept bundle of
five documents: the question names a section by three title tokens and holds
a neighbour's whole one-token title (1 of 9 question tokens); the fusion put
the named section at rank 1, the partition moved the neighbour over it.
A covered concept now RISES through the fusion's order and stops beneath the
first concept whose title answers more question tokens, by equality, than it
holds, or beneath a covered concept the fusion put above it. With nothing
above it answering more it reaches the top exactly as before. Same flag
(--title-covered / --no-title-covered), no new parameter, no new constant.
Measured before this commit, delivered ranks from build_payload:
- known-negative: rank 2 -> 1; the payload equals --no-title-covered's
- three own probes on that bundle: 1, 1, 1 (unchanged from round 16)
- R761 XML, 2 761 concepts: hit@1/8/50 6/6 - 6/6 - 6/6 at default k and at
--k 50, KP rank 1; 8 of 8 payloads byte-identical to 7cca9e0 at BOTH k
- payloads byte-identical to 7cca9e0: K2 pinned 6/6, Arm B 6/6, N100/N200/
N500 15/15; tests/test_default_bundle_pin.py 7 passed, file untouched
- candidates measured beside it: min title length (R761 hit@1 3/6), share of
the question (holds only for 1/9 < s <= 1/6), order inside the group
(group of one: no effect), stop list (no title involved is one)
Suite on the staged set: 1600 passed, 1 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`okf check` had fifteen rules and none asked whether the skill and the payload
were talking about the same bundle. Reproduced on this HEAD before any code
moved: three pairs reported `conformant: 15 rules over 8 excerpts and 438
withheld entries, 0 findings` -- a skill generated from one corpus against
another corpus's payload, the unfilled template against that payload, and a
payload sharing the skill's `bundle_id` at a foreign `ref`. All three now exit 1
with one `bundle_mismatch` finding over 16 rules.
BOTH halves are compared and the `ref` half is load-bearing: three distinct
builds on this machine carry one `bundle_id`, so an id comparison would pass a
stale skill. SS 3.3: "a version is the producer's assertion; a ref is a fact
about bytes". An identity the rule cannot read is a finding, never a silent
pass -- that is what refuses the unfilled template.
No new field: the identity was already in the generated skill's prose, now
factored into `skill.identity_line` and read back by
`contract_check.skill_identity`. Generated skill bytes unchanged, measured on
both tracked bundles on one interpreter.
The rule's first real find is this repository's own hand-made
`skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
identity a reader can act on: 1 of 1. Nine tests that asserted the old, false
conformance now pair a skill with its own bundle's payload.
Measured, nothing else moved: `~/okf-test/dokumenter` `diff -r` empty old
source vs new on one interpreter (52 files, 26 concepts), `okf project` still
byte-equal to `okf build`, K2 pin unmodified and green (453 concepts, ranks
1,1,1,1,1,5), known-negative `{}` unchanged at 9 findings.
Report: docs/2026-09-10-k3-runde15-bundle-mismatch.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A patch release carrying round 16's ranking rule and nothing else. No
behaviour changes in `src/`: the version string is the only line touched
there, and it is touched because it is the second half of the version --
`tests/test_packaging.py` holds `pyproject.toml`'s declaration and
`__init__.__version__` equal, and a tag whose package reports the previous
number lies to every consumer that installs it.
- `pyproject.toml` and `src/llm_ingestion_okf/__init__.py`: 0.8.0 -> 0.8.1.
- `CHANGELOG.md`: round 16's own entry closes as `[0.8.1]`, with a new empty
`[Unreleased]` above it and no comparison link, because the file has
published none since `[0.6.0]`. The entry gains one bullet it did not
carry: the rule's MEASURED DOWNSIDE, reproduced here on a five-document
folder -- a constructed known-negative question demotes the answering
section from delivered rank 1 to rank 2 because a neighbouring concept
titled with a single common process word has its WHOLE title accounted
for. Written as a known limitation of short generic titles, not as a fix.
- The five install lines (`README.md` 4, `llms.txt` 1) move to `@v0.8.1`.
Measured after: `okf.git@v0.8.0` 0 hits, `okf.git@v0.8.1` 5 hits as the
known-positive control, and the two remaining bare `v0.8.0` occurrences
are the tag-history row and the test-count history -- both are statements
about the past and both stay.
- The tag-history list gains `v0.8.1` as the current tag and demotes the
`v0.8.0` row to what it was; `v0.7.0` is untouched, and `docs/` is not
rewritten at all.
- The test-count history sentence named the `v0.8.0` release as the last
one and would have become false on this tag: it now records 1582 through
`v0.8.1` as well. The published figures 1583 collected / 1582 passed /
1 skipped were re-measured here and stand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report, the CHANGELOG entry under [Unreleased] and the README's ranking
section. No version bump and no tag: v0.8.0 was released this same day.
R761, delivered ranks, before -> after: hit@1/8/50 3/6 - 5/6 - 5/6 -> 6/6 -
6/6 - 6/6 at default k and 3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6 at --k 50, with
the known-positive at rank 1 on all four and the known-negative still not a
hit. --no-title-covered on the new binary reproduces the old row exactly.
Nothing else moves, measured on the bytes: the pinned K2 bundle and Arm B keep
(1,1,1,1,1,5) with payloads byte-identical on 6 of 6 questions each,
n100/n200/n500 payloads byte-identical on 5 each, the 828-file HTML corpus at
828 plans / 0 unreadable / 6 015 md with diff -rq empty, the five-document
folder diff -r-identical at 26/52, and okf project still byte-equal to okf
build.
README's test count corrected to what this round measures: 1583 collected,
1582 passed, 1 skipped. The published 1575 was the passed count.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 13 (the `.xml` core file type read as NISO-STS, and the PDF arm's
collision counter) and round 14 (a section the source DECLARES takes the
declared-structure route: `.xml` goes from 15 of 2 761 to 2 761 of 2 761
boundaries and from 23 to 2 761 concepts at the shipped defaults, hit@1/8/50
0/6 - 0/6 - 0/6 to 3/6 - 5/6 - 6/6) are both landed. This commit adds no
functionality: it sets the version, closes the CHANGELOG entry, and points
every install line at the new tag.
- `pyproject.toml` and `src/llm_ingestion_okf/__init__.py`: 0.7.0 -> 0.8.0.
The second is the only line in `src/` this release touches. It is not a
code change but the other half of the version, written without a `v`
prefix, so a search for `v0.7.0` cannot find it;
`test_packaging.py::test_the_declared_version_agrees_with_the_packaged_one`
is what did. Left alone, the tag would report the previous release to every
consumer that installs it.
- `CHANGELOG.md`: `[Unreleased]` becomes `[0.8.0] - 2026-09-10`, with a new
empty `[Unreleased]` above it. The entries are round 13's and round 14's own
words, unchanged. No compare link is added: this file has carried none since
`[0.6.0]`, and inventing one here would be a claim about a URL nobody checked.
- The five install lines and the two prose lines naming the current tag move to
`v0.8.0`. The tag history list gains a `v0.8.0` row as the current tag and
KEEPS the `v0.7.0` row: that list states it is not install lines, so a
rewrite would delete history rather than update it.
- README's test count was 1515, measured 2026-09-09; this tree measures 1575
passed / 1 skipped with ruff 0.16.6. The surrounding sentence about the
earlier figure is repaired too, because changing the date alone would have
made it false.
`v0.7.0` stays on 1260fac. No lock change, no history rewrite in `docs/`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report, with every measurement carrying its denominator and both
instrument forms side by side on every row -- including the proof that the two
forms cannot both reach 2 761, because 2 739 of 2 761 fasit titles carry a
numbering token and the two keys then demand mutually exclusive strings.
Also states two things round 13 left unsaid:
- XML that is not STS gives 0 plans and exit 2, and the `.txt` known-positive
shows it is general `okf build` behaviour for any structureless document,
not an `.xml` defect. Separating the two states would change the outcome on
0 of the 4 reference corpora, so the gate stays.
- Which two instruments were corrected in scratch and what the correction is.
With the shipped matcher the XML bundle scores 0/6 with `rang: null` on
every question, including the known-positive whose top excerpt IS the fasit.
Round 13's 14 two-file directories are re-classified on measurement: they were
false positives of the text route reading the document's own contents listing,
not the publisher reusing a title, and they are gone. Two remain, the same two
the PDF arm has.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The reader reaches the ceiling and the build does not, and the report separates
the two with denominators and a control for each: the orphan check removes 710
of 2 761 (710 of 710 removed headings are followed immediately by another
heading; 0 of 2 051 delivered ones are), and Arm F removes 2 066 more, swept
one flag at a time.
Both instrument forms on every row, because the pair key scores 28 of 2 761
here for a reason that has nothing to do with segmentation -- and the shipped
instrument turns out to carry the round-12 correction in NO block, not just in
the three the order named. That is proposed to the owning repository, never
applied there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`okf build` recovers a PDF's boundaries from the shape of its page text and
never opens the file's own `/Outlines` bookmark tree. On a 701-page process
code whose publisher also ships a NISO-STS structure for it, measured outside
this repository and reproduced here exactly: the shipped default finds 1967 of
2761 titled sections, 0 of its 28 chapters, and 794 of 794 misses have their
heading text PRESENT in the extracted text. The line was read; the boundary
was never opened. The same file's bookmark tree matches 2761 of 2761 of those
titles exactly after normalisation.
`--pdf-outline`, OFF, cuts a PDF at the boundaries its tree declares.
boundaries 1967 of 2761 -> 2759 of 2761 (gate was 2700)
depth 1 0 of 28 -> 28 of 28
titles identical to source -- -> 2761 of 2761
false positives 163 of 2182 -> 3 of 2762
directories with two files 132 of 2050 -> 2 of 2738
front-matter concepts 72 of 2182 -> 2 of 2762
consumption fasit present 4 of 7 -> 7 of 7
hit@1 / hit@8 / hit@50 1/6 2/6 4/6 -> 3/6 5/6 6/6
It is a SEGMENTATION arm, not a reader option: the extracted text is byte for
byte the same either way. A PDF with no tree builds byte-identically with the
flag on -- `diff -r` empty across the pre-change tree, the arm off and the arm
on. An unresolvable `/Dest` is dropped and COUNTED, never fabricated into a
boundary and never a refusal of the file.
The bridge from (page, y) to a line index is the whole risk, so both routes
are measured. `extract_text_lines` splits lines identically to `extract_text`
on 701 of 701 pages, and is CHECKED per page rather than assumed. The y route
and the title route disagree on 0 of 2762 nodes, flat from a 0pt tolerance to
8pt and collapsing at 12pt, so the rule ships with no tolerance constant. The
naive "nearest line" rule was wrong on 1840 of 2762, one line early every time.
The orphan check is not applied to a bookmark mark: it asks whether anything
stands under a candidate's first line, which is the right question for a
heuristic's guess and the wrong one for a publisher's declaration. 683 of 2762
marks are container sections; applying it scores 2079 instead of 2759.
No new dependency and no second parse of the pages: `pdfminer.six` already
ships under `pdfplumber` in `[extract]`. 119.22s -> 183.31s wall, peak RSS
3252 -> 3251 MiB. The default does not move; 1 of the 8 reference PDFs carries
a usable tree at all.
`.pdf` also gains its `_EVIDENCE` row, as `measured` -- it was the row with the
most measurement behind it and no entry in the table.
Report: docs/2026-09-10-k3-runde12-pdf-outlines.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three of round 9's four measured holes, each closed with a rule chosen on a
measurement rather than named as a limit.
`rtf` GIVES 0 SEGMENTS -> 6 of 6 AUTHORED TITLES over N = 4. The container has
no heading style, so the author's title is bold text. The grammar is markdown,
not `rtf`: the converter already writes that title as `**...**` in the same
output every office row produces, so no `rtf`-only heading form exists. Three
parameters were swept over 47 readable documents and ONE carried -- refusing a
line that ends in terminal punctuation takes false-positive lines from 9-12 to
1-2. A maximum title length (unlimited/40/60/80/120) and a
must-stand-between-blank-lines clause are both FLAT, so neither is in the rule.
The last false positive is closed by G1, the principle `_gate_outline` already
carries: recovery yields to declaration. False positives are then 0 of the 31
declaring documents by construction, and 0 of 27 on the corpus. Reach: 2 of 39
corpus documents, both `docx`, 0 of 33 `pdf` and 0 of 2 `xlsx`. Behind
`--bold-title`, default OFF pending the hit@8 measurement; the default bundle
is byte-identical without it.
BOTH ALTERNATIVES THE ORDER NAMED WERE MEASURED AND FELLED. A fourth hand-laid
fixture DECLARES heading styles in a stylesheet and the converter discards
them, emitting the same bold line -- so "read the declared headings out of the
markdown" has nothing to read. `rtf` -> `docx` -> markdown yields 0 ATX
headings on that same document, because the loss is in the `rtf` READER before
any writer sees the style. Fixtures are hand-laid in `make_k2_office.py` with
the fasit written first; they live in their own directory because Door B walks
a drop directory recursively and `k2-office/` reads its N off the listing.
THE PREFIX OVER-MATCH: THREE CANDIDATES MEASURED, ALL THREE FAILED ON ONE ROW.
Re-measured on the pinned 453-concept bundle with the control run first:
`under` occurs 79 times by equality and matches 172 by prefix, `undersjoisk` 0
and 172, `bilateral` 0 and 400 of 453, `standhaftig` 0 and 219. The two extra
known-negatives were FOUND, not chosen -- every 4-character prefix ranked by
document frequency, then a real word taken from the widest. A longer floor
(5-8), a coverage share (0.5-0.8) and a long-words-only floor (>= 8) each cost
row 1 its rank on the default bundle and the whole row on Arm B. Decomposed:
row 1's token `prisene` reaches its gold document through
`pris|sammenstilling` on four characters -- 0.57 of one word and 0.22 of the
other -- so the over-match and the wanted match are one mechanism.
THE FOURTH CANDIDATE IS THE ANSWER: the shared prefix must be a WORD the bundle
uses. `pris` is; `bila` and `stan` are not. `bilateral` 400 -> 0 and 512 -> 0,
`standhaftig` 219 -> 56 and 235 -> 33, every hit@8 row keeping rank 1 on BOTH
bundles. `undersjoisk` stops at 162 because `under` IS a word here -- a genuine
Norwegian morpheme, so that residual is a different answer, not a ceiling. ON
by default (`--no-stem-prefix`), pinned with its own known-negative on the
shipped bytes.
THE SHIM: a path importer holds the object `module_from_spec` made, and
`sys.modules[__name__] = _impl` never reaches it. Measured under both counting
methods -- 3 of 76 public names by `vars()`. One line copies the public names
into this file's globals; the dunder filter is load-bearing, because an
unfiltered copy overwrites `__name__` before the next line uses it as the alias
key. It restores attribute ACCESS and not patch-through, which is why the alias
stays. A CHANGELOG note under 0.7.0 and a shim docstring line say so, since
what the consumer asked for was the note.
Suite 1515 -> 1535.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`okf project` built a bundle two rules behind `okf build`. `cli.build`'s
signature defaulted `keep_table_heading` and `sheet_section_rows` to `False`
while argparse defaulted both to `True`, and `project.create` calls `build()`
as a function. Measured on a five-document folder: 15 concepts / 30 files
where `okf build` wrote 26 / 52, the whole difference in the priced sheet --
the document a question about price has to reach. The invariant test could not
see it: it compared `project.create` against the same function, and its two
fixture documents had neither a table nor a sheet. Both gaps are tests now,
and the two paths are byte-equal on that folder (`diff -rq`, 0 differences).
README opens with what / one install line / two commands / the three shapes of
request; the phase-status paragraph moved down, nothing deleted. One tag is
pinned everywhere: README pinned v0.4.0 on its install lines and v0.6.0 below,
llms.txt pinned v0.4.0, so an agent reading from the top installed a tag
without `okf project`.
The skill states three modes -- question, hypothesis (per premise, `confirmed`
/ `refuted` / `undecidable-from-bundle`), and a task producing a document
(source per claim in the artefact, an ungrounded paragraph written and marked
rather than dropped, the cut declared inside the document). The five markings
are untouched.
Generated skills state relative paths in the project layout: `okf consume
.okf/<id>` and `okf check --skill .claude/skills/<id>-consume/SKILL.md`,
runnable from where `okf project` tells the reader to start `claude`. Two
absolute paths to zero, measured with a query shown capable of finding first --
O5's published "4 -> 0" used `grep -c "^/"` against paths indented by two
spaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.
The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.
The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.
The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.
Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.
Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).
Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.
Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md
Co-Authored-By: Claude <claude-opus-5>
Two producer-side findings from the consumer's S7c acid test (ordre 20260908T063454Z-3648220855-from-.claude), both measured on K2 before and after, both with the corpus rebuilt from scratch.
FUNN 3 -- THE FORM. The converter's default markdown writer emits simple tables, which pad every cell out to the width of the widest cell in its column. Measured on the tender's price sheet: one 594-character prose cell produced a 67 244-character whitespace carpet with runs of up to 887 characters between a label and its amount, 19 integral amounts carrying a converter `.0`, and a header row naming one column. The bytes reached a live model in 2 of 11 prompts and 0 of 11 answers. The spreadsheet row now writes pipe tables with `--columns=1` (load-bearing: the pipe writer pads to a width computed from it, so at the default 72 a narrow table gains runs of up to 45). Same sheet after: 11 048 characters, longest run 2, one row per line, 0 artificial `.0`. Spreadsheet-only, and the scoping is pinned by three digests -- the same change moves the odt fixture 1366 -> 1105, so it can fail.
The `.0` rewrite is bounded twice: to a cell whose whole content is such a number, anchored between unescaped pipes, and skipped when the literal is in the workbook's shared string table -- the converter renders the number 92 and the TEXT "92.0" identically, so the output alone cannot tell them apart. Read with zipfile and xml.etree; no new dependency.
FUNN 2 -- THE LOG LINK. `link_log_in_root_index` (95eb271) is removed. Consumption contract SS 9.2 forbids a consumer from enumerating the bundle directory unless the profile says the index is derived, so the index tree is the entire map a consumer may use and everything it links is a document: their navigator returned 630 where our pre-pass counts 629, and a corpus run's own log was citable as content. The log is still written to the bundle root (SPEC section 9); `tools/okf_consume.py` keeps its exclusion for the bundles already built with the link.
K2 rebuilt twice. BEFORE reproduces the consumer's ref exactly (`sha256-tree:f14872a0...c8a92a`, 629 concepts) and their three consume figures to the token (57 289 / 62 149 / 58 401). AFTER: 629 concepts, `merged + coded rejections = 43 = N`, new ref `sha256-tree:c26eed6a...e3261f`, 627 of 629 concepts byte-identical, 1104 of 1108 files identical to the delivered bundle.
ONE REGRESSION, MEASURED AND NOT FIXED: on the mandate-shaped question with the vocabulary bridge the priced concept moves from candidate rank 10 to 19, so `--k 12` withholds it `below_k`; `--cost-vocabulary --k 20` delivers it at 65 912 o200k. The cause is measured rather than argued -- restoring only the concept's title on the new short body ranks it 10 again. The chain ends at the orphan check (`propose.py:461`), which drops the sheet heading once a table block opens two lines below it. That is the already-reported orphan gate, and changing it is a default-ON segmentation rule affecting every document type. The specific question is unaffected: rank 1 before and after. The priced excerpt's budget share falls from 56.5 % to 9.7 %.
11 new tests (RED first), 8 mutations, 8 red, with an unmutated control green each time. One mutation survived twice before the fixture could make it fire, and both survivals are written down. 1279 -> 1287 tests. mypy --strict clean on 28 files. ruff clean. Both proposer goldens byte-unchanged. One frozen literal moved with the fix and is reported rather than hidden.
Report: docs/2026-09-08-prisform-og-loggen-k2.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two producer-side defects from the S7 acid test (ordre 20260907T234741Z-9578626297-from-.claude), both reproduced on K2 before and after.
F1: `okf build --ingested-at` alone stamped only 11/629 concepts -- the unsegmented ones, which read the call's value directly. The 618 segmented concepts read `segment.ingested_at`, the plan's `proposed_at`, independently defaulted to `DEFAULT_STAMP`. `proposed_at` now falls back to `ingested_at` when omitted; neither flag passed still yields `DEFAULT_STAMP` for both.
F2: the consumption pre-pass's index walk counted a root-linked `log.md` (`corpus.link_log_in_root_index`, `95eb271`) as a concept, inflating a 629-concept K2 rebuild to 630 and letting the log rank and cut like real content. The link stays -- the contract is silent on `log.md` and `95eb271` already named it a LOCAL choice -- but the walk now treats `LOG_NAME` like the index itself: reachable, never a concept.
K2 rebuilt twice from the same corpus and diffed against the delivered `K2-bundle-20260903`: FOR (stashed fix, matching fbaac6d) reproduces po's numbers exactly -- 619/1108 files differ, 618 ingested_at-only, ref `sha256-tree:4ffd750c...`. ETTER (fix applied) leaves exactly 1 line differing (the deliberate log link, predating this fix) -- 0 files stamped 1970, 629/629 stamped 2026-09-03, ref `sha256-tree:f14872a0...`. The delivered bundle's ref is unchanged before and after (`sha256-tree:9a4e5561...a968b5`), since it carries no log link and the new branch never fires. Conservation identity holds both times: merged + coded rejections = 43 = N, 39/0/4.
1258 -> 1260 tests. mypy --strict clean on 28 files. ruff clean. Both goldens byte-unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
run() parses the plan before building anything and parses its own
verdict after. Only the first failure is the operator's file.
The second is reachable: an empty --adjudicator produces a verdict the
grammar refuses ("adjudication field 'adjudicated_by' must be a
non-empty string"). The plan parsed fine; the fault is in what this
command stamped onto it.
Catching SegmentationError at the top of main() -- the previous commit
-- caught both raise sites and printed "malformed plan" for each. On
this path that is a clean, confident, WRONG diagnosis: it sends the
operator to fix the one artifact that was fine. Worse than the traceback
it replaced, because a traceback at least does not claim to know.
The verdict parse now raises AdjudicationError, which is what "this
command failed" already means in this file and already returns 2. Exit
code unchanged either way, nothing written either way; only the message
changes.
Suite 1073 -> 1074 passed (pytest exit 0, measured without a pipe);
ruff and mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A SegmentationError raised by the plan grammar escaped main() as a
traceback and exit 1, while every other malformed-plan case in the same
file already returned 2. Exit codes are the interface a caller scripts
against, and exit 1 with a traceback is the code an unhandled bug
produces -- it says "this command broke" where the truth is "this file
is not a plan".
The refusal itself is unchanged: nothing was written before and nothing
is written now, and the grammar in src/ is untouched. What changes is
one line on stderr naming the error code, and the exit code.
Both branches that can raise are covered: the pre-write parse of a
non-empty plan, and the required-field check reached through the empty
branch.
The old behaviour was pinned by
test_an_entries_value_that_is_not_a_list_is_still_refused, which
asserted that a wrongly-typed `entries` reaches the caller as a raised
SegmentationError and recorded that as a finding rather than fixing it.
That test is rewritten here, in the same commit as the code, to assert
exit 2 plus the code on stderr. A second test pins the one-line stderr
shape on the non-empty branch.
Suite 1072 -> 1073 passed; ruff and mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The guard could not read back what this library WRITES. At 1.2.0,
`okf.parse_frontmatter` refused the OKF v0.2 golden outright --
`OKFFrontmatterError: value begins with a disallowed YAML indicator '['`
against `sources: [{ id: golden-v0-2-sales, resource: fixture }]`. Flow is
the only form this library can emit, because its own line-oriented parser
cannot round-trip the block form at all, so a gate that refuses flow
refuses everything Door A produces under `OKF_V0_2`.
The control was run BEFORE the bump, which is the only moment it exists:
the probe raised on 1.2.0, so the new test discriminates rather than
merely passes. `[project.dependencies]` already said `>=1.2,<2.0` and is
unchanged; only `[tool.uv.sources]` and `uv.lock` move.
TWO gate rows moved, not the one the work was scoped around, which is why
the whole documented probe was re-run instead of just the `sources` case:
the BLOCK form of `sources` now passes too, retiring G30. That changes
nothing about what we emit -- our own parser is still the binding
constraint on writing flow -- and `docs/okf-nokkelinventar.md` now carries
a `guard 1.3.0` column beside the 1.2.0 measurement rather than
overwriting it. A third row kept its verdict but changed its reason, so
the quoted message was corrected too.
The Door C boundary is unmoved, verified with a known-positive:
`resource` is allowlisted only inside a `sources` entry, so section
10.2's `executor.resource` and `attester.resource` are still rejected
("not on the OKF mapping allowlist under 'executor'") while top-level
`resource` passes.
`uv.lock` also gains `pypandoc-binary==1.17`. That is a stale lockfile
being corrected, not a new dependency: it was already declared in the
`[extract]` extra, and `uv lock --check` reports the lockfile out of date
on the untouched tree. Core keeps exactly one runtime dependency.
Not addressed, and recorded rather than built: the guard reports that
`sources[].resource` is scanned as text but never URL-validated, because
SPEC 5.1 permits bundle-relative paths and scope descriptions. No
consumer has asked for a gate there.
Guard 1.3.0 installed from 44e2b31, verified anonymously over https
against the remote tag. 1054 -> 1055 tests. `mypy --strict` clean, `ruff`
clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on the K2 artifact by a consumer: `log.md` was on disk and no
index named it, so a reader entering the bundle at `index.md` -- the walk
section 8 exists to support -- never reached the one file carrying `N`.
Stated as a LOCAL choice rather than conformance, because it is one.
Upstream's own bundles do not link their log: measured at `9a15b13`, 0 of
the 24 shipped `index.md` files name the single `log.md` in the set, with
the same grep form finding `tables/index.md` in 4 of them as the
known-positive control. That shows the link is not REQUIRED -- not that
it is disallowed. `docs/plan/okf-v0.2-alignment.md` P1-F6 already
recorded the upstream shape; a line there now separates the two claims,
since reserved names still stay out of an `entries_match_directory`
listing and this profile has that off.
It lives in the harness because the library cannot make it. The log's
content IS the run's outcome, so it cannot exist when the indexes are
projected, and an index that enumerated it off the directory would gain
the link only from the second run onward -- breaking
rebuild-equals-incremental, the property the segmented bundle is built
on.
The membership test is load-bearing and was measured, not assumed. The
two reprojections disagree about this line: the per-directory one drops
every managed entry before re-emitting its block, while the flat one
keeps a managed line whose target is not an owned concept, deliberately,
so that a regex cannot delete curated content. Appending unconditionally
therefore doubled the entry on the second unsegmented run, which is why
both run modes are pinned separately.
1052 -> 1054 tests. `mypy --strict` clean, `ruff` clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM decision B6 asked for a list-taking _render_sources so a concept can record
more than one source, and prescribed the block list as the emitted form. The
list is delivered; the block form is not.
Three measurements, not an argument. Our own parse_frontmatter skips indented
lines, so a block list round-trips to an empty value with every entry silently
gone -- and _is_ingest_owned reads through that same parser. The consumer B6
was written for accepts the multi-entry flow sequence and classifies a block
sequence as unreadable provenance, so block would hand it exactly the state it
cannot read. And B6's own acceptance test asks for a round trip through this
parser, which no block form can pass.
A single source renders byte-identically, so all six goldens are unmoved. The
unquotable-value gate now runs on every entry, not just the first. New code
sources_empty refuses an empty list.
1023 -> 1034 tests, including the negative control that pins the block form's
silent data loss.
The README told consumers that `docx` and `xlsx` ship no parser and always fail
fast. True when written; false the moment the converter seam landed -- and
false SILENTLY, because prose has no test. This repository has been bitten by
that exact shape before: a published guarantee is a test obligation.
So the correction comes with `tests/test_docs_promises.py`, which compares the
README's declared format list against the registries it describes and fails on
a format added without touching the README, on the old claim reappearing in any
wording, on an unmeasured row going unnamed, and on the exclusions being
dropped. Negative control: removing one format from the README's marker turns
it red.
The README now states which rows are measured and which are not. Three of the
five office rows have denominator ZERO in the corpus -- they work by
construction and have never met a document anyone wrote. They are not known to
be broken and not known to be right, and a reader should not have to open the
source to learn which.
The CHANGELOG's shipped entry is left as written, because a changelog records
what a release did; the correction is stated at that line instead so a reader
arriving there is not misled.
Suite 913 -> 917.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
