PM's checkpoint on 2c8296b ran eight cheating attacks at this gate and four
went through: a row came back GREEN without one label becoming true or one
concept ranking better. This is the first of them, written as a test that must
refuse it.
J2: `row_eight` iterates over whatever `--real` handed it and counts. One set
of three came back `6 of 6 GREEN`, and the realistic route is `--real r761`
alone -- the set PM's own registration puts at 7 of 7, with the two that miss
left out. The docstring already said "never green by leaving a set out"; the
code did not, and THIS REPOSITORY'S OWN TEST asserted the opposite at
`tests/test_retrieval_gate.py:486`, `(1, 1, GREEN)` for a single wiki set.
That assertion is corrected here rather than worked around: a test that pins
the defect is the defect.
J2b: the headline `k of N` is `quoted_hits + concept_hits` over
`quoted_units + concept_units`, written on the line directly above the detail
that says the two granularities are NOT summed into one number.
3 red on an assertion about behaviour (GREEN where NOT RUN is required, and a
headline that is a sum), 46 passed. The green direction -- all three named
sets present -- is in the same commit and passes already, so the fix cannot
close the rows by making the row unreachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner could only run one test file, which is why PM's three
survivors from `43331fc` could not be added: two are held by the gate's
row 3 and one by the soft-hyphen door's suite. A mutant now names its
suite; the catalogue goes 39 to 45.
X3 and X4 rewritten against the code as it now stands -- a mutant table
is a copy of the code it mutates, and this round moved the lines both of
them quoted. X6 is the defeated state exactly, X7 cuts the ledger off at
its source, X8 removes the cursor rule, P6/P11/P12 are PM's three.
Two survivors on the first run, both findings, both closed:
- X4 survived because every forgery arm now fails on the ledger check
before the binding is reached. An arm was added where the run DID book
the pair and the block stating it points at another picture.
- X5 survived the WHOLE suite -- 2134 passed with the disarming removed
-- because a document-supplied field can no longer reach the gate. The
property is about the BUNDLE and not about one judge, so it is kept and
measured in `tests/test_assets.py`, with a known-positive counting the
run's own two fields on the same expression.
killed 45 of 45, exit 0. Report, CHANGELOG and CLAUDE.md written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`test_the_four_existing_goldens_are_untouched` ran `git status` with
`check=True` in a `git archive` extract and raised `CalledProcessError`.
Measured 2026-09-19 it was the single failure of the whole suite run from
a clean extract -- twice reported as a round's one failure, twice by a
round that had not touched the file.
Both arms measure the CHECKOUT with `git`, so an extract has nothing to
ask. It skips with the reason stated, never silently: a checkout still
has to answer. Reproduced before and after on a non-repository copy --
`CalledProcessError` -> `SKIPPED [1] ... not a git checkout`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`assert sum(tbx.values()) == 568` sat behind a `skipif` on a file only
this machine has, so on a fresh clone the sentence five files publish was
unguarded again -- the state in which 574 survived in four docstrings
until PM counted it.
`N101_TBX_TAGS` is now the one place the number lives, the delivery test
asserts against it, and a second test reads the published sentence out of
all five files and holds them to it. It needs no corpus and no clock:
editing CLAUDE.md to 600 is red on a fresh clone.
It was red at birth for a reason worth keeping: the scan read this test
file's own known-positive string (`574`) as a sixth publisher. The
known-positive is now assembled from pieces, and that failure is the
demonstration that the scan reads what it is pointed at.
What it does NOT prove is stated in the docstring: five files agreeing is
agreement, not a count. The measurement stays where it was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chose the CURSOR over a pixel-coverage count because the corpus cannot
choose -- 25 of 25 of the R761 delivery's RLE8 BMPs paint every pixel,
25 of 25 reach the end of the frame, 0 of 25 use a delta -- and an
independent decoder can: a delta and an end-of-line escape state their
skip, so every decoder agrees on the index-0 pixels they pass over, while
a pixel count would refuse both constructions the format defines.
`_bmp_rle8_rows` now refuses (`asset_samples_invalid`) when the terminator
arrives with the cursor short of the last row. Pillow reads 5 of the 8
streams in the table and refuses the same 3, one of them short by a
single pixel.
Both docstrings the round was sent to correct are rewritten: the test no
longer claims every pixel is decoded (it is not -- a stated skip keeps
index 0), and `_bmp_rle8_rows` no longer frames the delta argument as
read off the corpus, which it never was.
R761 rebuilt: bundle `diff -r`-identical to the build before this commit,
50 assets (29 JPEG + 21 PNG), 19 of 19 conversions, SHY 71, u = 0, d = 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM measured it on `ae441ab`: a stream whose first two bytes are the
end-of-bitmap escape is carried with 32 of 32 pixels never decoded, while
Pillow refuses the same file. The round before closed a truncated stream
by requiring the terminator, and a stream can say it is finished anywhere.
Eight streams over one 8x4 frame, each labelled with whether the cursor
reaches the end of the frame at the terminator. Three do not and are
carried today. The table's values are the INDEPENDENT decoder's, measured
on those eight files, and a second test holds Pillow to them -- otherwise
the table is our own rule restated.
The line is the cursor and not the pixels, and the corpus cannot choose:
over the 25 real RLE8 sources of the R761 delivery, 25 of 25 paint every
pixel, 25 of 25 reach the end of the frame and 0 of 25 use a delta.
Red on behaviour: 1 of 2 (the decoder table already holds).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chose the side channel over neutralising pointer-shaped document text,
because the second fix changes what every document SAYS in order to
defend a tool outside the build: a source quoting a bundle listing would
come out altered and existing bundles would move bytes. This reads a
file the run already writes.
`assets.conversion` names the pair, `DocumentAssets.conversions` carries
it out of the run, `DocumentAccount.conversions` books it, and the
accounting JSON states it per document. `_declared_conversions` reads it;
`_conversions` now believes a pair only when the RUN booked it AND a
pointer block confirms it for the asset it names. The confirmation can be
forged and the ledger cannot, which is why the ledger decides.
Measured through the real `okf build`: the three arms PM reproduced
(two `<p>`, one `<p>` with `<br>`, a markdown note beside the carrier)
go forged -> refused, 3 of 3, with the known-positive True in all three.
The text-level regression guard goes 3 arms to 13, the two new ones being
a perfectly written pointer block the run never booked.
R761, rebuilt: 25 BMP sources, 19 held, 19 of 19 conversions confirmed
against 19 declared, 50 assets (29 JPEG + 21 PNG, 0 BMP), SHY 71,
u = 0, d = 0, exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM measured it on `ae441ab` and this reproduces it through the real `okf
build`: a BMP refused `asset_too_large`, absent from `assets/`, read as
CARRIED because the document wrote two paragraphs. `_POINTER` matches a
FORM, and a form is not a signature.
Three arms, each a whole build, each a way to put two markdown lines in a
bundle without the run writing them: two `<p>` elements in one HTML file,
one `<p>` with a `<br>`, and a markdown note beside the HTML file that
carries the real PNG. All three forge on this commit. The known-positive
is in the same builds -- a 2x2 BMP the run really does convert -- so an
arm cannot pass by the route going dark.
Red on an assertion about behaviour: 1 of 1 (the loop stops at the first
arm; the other two are reproduced in the same shape).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One command, one exit code, and no capability: nothing here changes the
ranking, the fusion, the tokenisation or the cut. For a frozen question set
against a bundle it asks how many of N measurement units the payload carries
the fasit for, gives every miss exactly ONE class, and asks the thing the
payload does not do at all -- say so when it does not know.
Today, on the synthetic corpus this repository generates and the five sets it
ships pinned by sha256: rows 1 (9 of 9), 2 (7 of 7) and 6 (9 of 9) green;
3 (2 of 5), 4 (3 of 6), 5 (0 of 1), 7 (11 of 13), 8 (not run) and 9 (0 of 6)
red. GATE RED, exit 1.
- Row 3 is the finding to read first. In a bundle built from ONE source
document, every concept past the first two carries that document's
`source_file`, so a concept the RANK had already lost is withheld as
`source_quota_exceeded`. The truth is decided by the SAME cut run without
the quota -- PM measured the same lie on 13 of 25 real misses 2026-09-17.
- Row 4 is the second. No payload key says "this bundle does not answer
that", so the only honest reading today is `delivered == 0`; three of six
controls come out right and the three that do not get eight excerpts and no
statement. A control the bundle DOES answer must come back unmarked, or the
marking says nothing, so both directions sit in the same row.
- A miss has exactly one class -- (a) not in the bundle, (b) below k, (c) cut
by quota or budget, (d) delivered without the citation, (e) other -- and
each is FORCED by its own fixture. An unplaced miss takes the row to 0 of N.
- The judge opens the bundle. Every concept is read off disk before a payload
is built, a delivered excerpt whose text is not the concept file's bytes is
not a hit, and the two title forms come from `quality` rather than being
written a second time.
- FELLED means a row got WORSE, never merely that something changed: removing
the quota makes row 3 green, and counting that as a kill would credit this
gate with a check it does not have. The two survivors are reported with what
they measured -- the document prior dead and RRF_K flattened each moved 0
ranks and 0 deliveries on these fixtures -- and with a mechanism for each.
- Rows 8 and 9 are never green by leaving something out. The three real sets
are read from path + sha256 and never committed: a gold set names a
consumer's documents and this repository is public. K2 is 0 of 6 because the
gold set exists nowhere while its bundles do.
Denominators re-measured against the sources 2026-09-19, and all four hold:
wiki 20 questions / 29 fasit entries (sha 972d0f57...), R761 8 (S1-S6 + KP +
KN), vegnormal 32 questions / 43 must_cite, K2 6. PM's 45 of 70 is itself a
mixed sum -- wiki at question level, vegnormal at citation level -- so this
gate carries it labelled as recorded and never sums the two granularities.
46 tests, each failing row driven both ways through inputs rather than by
editing the row. No network (the socket guard is fired before its silence
counts), no private corpus, no clock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CHANGELOG under the UNTAGGED `[0.10.1]` -- no version bump, no tag. Three
entries: the forged carry under Security, the truncated RLE8 under Fixed, and
the two small rests (the judge's fidelity limit, the lossless guard's core
arm) under "Documented, not changed".
The report carries the measurements with their denominators, including the two
checks this round introduced and then REMOVED after measuring that no mutant
can fell them -- with the numbers for each (the gate suite green without the
line anchor; 95 passed with and 95 without the length check, so the mutant the
order asked for survives).
CLAUDE.md records both defects and both rules where the viewable-asset
paragraph already stands.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
killed 39 of 39, exit 0 (36 before this round). Three new mutants, one per
check that closes PM's measured forgery, each felled by its own arm:
- X3 restores the pre-round `_conversions` verbatim -- the clause read from
anywhere in the bundle text. Felled by the body-text and table-cell arms.
- X4 drops the tie between the clause and the asset its block points at.
Felled by the arm where the clause stands in another asset's block.
- X5 drops `_inline`'s checksum disarming. Felled by the end-to-end arm where
an `<img alt>` carries the clause.
X5 IS WHY THE HARNESS CHANGED, and it is a finding about the harness rather
than about the code: it SURVIVED 112 green tests on the first run, and it had
never been applied. The copy is run with the venv's interpreter, which carries
an editable install pointing at the working tree, so `import llm_ingestion_okf`
in the copy resolved to the original `src/`. The subprocess now gets the copy's
own `src/` on `PYTHONPATH`, which wins over the editable finder, and X5 dies on
the first run under it -- the survivor/killed pair is the measurement. The gate
and the witness were never affected; the suite reaches those through the copy's
own `tools/`.
The docstring's "the twenty-six mutants" now says "twenty-six of them", since
the list has been 39 for a while and the sentence is about their provenance.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
160 passed over the three affected files. Two guards this round introduced are
removed, each because it was measured and found unkillable, and one real cut is
added in their place.
The end-of-line anchor on the judge's clause expression: with the pointer-block
route and the asset-name tie already in place, the only case it covered is a
label forging a clause inside a block that also carries a real one -- which
`assets._inline` now makes unreachable from a document. Removed with `$` gone
the whole suite stays green, which is the measurement: a guard nothing can fell
is not a guard, and it would also have made the mutant below unfaithful.
The length check on a short absolute run in `_bmp_rle8_rows`: unobservable by
construction. A run shorter than its declared count means the stream ran out,
so the loop cannot reach an end-of-bitmap afterwards and the picture is refused
either way -- the partial paint is discarded with the frame.
A FIFTH CUT REPLACES THEM, and it makes an existing bound load-bearing: a
stream cut in the middle of a delta escape, 17 of 32 pixels wrong by this
file's own decoder. Without the two-byte bound in that branch the cut raises
IndexError instead of a coded refusal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
19 passed in this file, 0 skipped. Green on arrival and stated as such: no
production code changes here, so there is nothing for a red commit to precede
-- the defect is that the property was never MEASURED on a core install.
Measured by PM 2026-09-19 on one: 4 of the 13 guards in this file were
SKIPPED, the lossless one among them, because they decode through Pillow --
which this package neither uses nor ships and which arrives only as a
transitive dependency of `pdfplumber` under `[extract]`. A plain
`pip install llm-ingestion-okf` therefore never measured the property this
round is named after.
The new arm decodes the carried PNG with `zlib` and the five filters of PNG
SS 9.2, and compares against pixels written out in this file: `rle8_indices`
for the RLE8 source and `RGB_ROWS` for the 24-bit one. Still independent in
the way that matters -- the package compresses rows it filtered, the test
reverses filters it decompressed. Two arms: one over `read_image`, one over
what the BUILD wrote.
The environment is MADE rather than assumed: a `sys.meta_path` finder raising
for every `PIL` name, with a known-positive that it fires before the guards
re-run under it. Control, run once and not committed: a Pillow-dependent line
placed inside the blocked section turns the guard red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Green: 112 passed in the gate's own file. The docstring now says what the two
routes do NOT reach -- neither decodes a pixel, so a converter writing a blank
PNG is accepted, and "claiming a conversion it did not perform" means claiming
one whose FILE is missing, never one whose pixels are wrong.
Teaching the judge pixels is a different job and is not this order. Leaving a
reader of the gate to infer the gap was the defect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 of 1 red, on an ASSERT about the published sentence. The behavioural half of
the same test is already green and that is the point: PM's M10 measured that a
mutated converter writing a BLANK PNG gives `asset_holds = True`, because the
bundle is internally consistent -- the digest it claims to have written really
is the asset's digest -- and the judge has no opinion about whether those bytes
hold the source's picture.
The limit is therefore asserted in both directions: it is REAL (the blank PNG
is accepted here) and it must be STATED (the docstring names what the route
cannot see). "A bundle claiming a conversion it did not perform still fails"
reads wider than the route reaches; the suite fells that mutant, this gate
cannot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both red guards green, and the whole suite is 2204 passed / 1 skipped (2199
before this round, +5 new tests, no golden moved).
TWO HALVES, AND NEITHER IS SUFFICIENT ALONE. The judge now reads the clause
only from inside a POINTER BLOCK -- the markdown image line plus the detail
line under it -- and only where the clause names the asset that block points
at, anchored to the end of the line because the build writes it last. That
closes ordinary body text and a table cell. It cannot close an image's own
alt text, because a label is document text that the build writes INSIDE a
pointer block, which is the second half: `assets._inline` disarms a checksum
field in anything that came from the document.
WHERE THE BOUNDARY RUNS, stated in both files. Everything `_inline` returns
came from the document -- an alt attribute, an STS caption, a publisher's file
name. Everything `render_block` appends after it came from the run: the size
it measured, the type it sniffed, the digests it computed. The second line
carries both, so document text may not emit the grammar the run writes there.
The digits are kept, because a reader is owed what the document said; the
colon that makes them a FIELD is not.
The judge's expression stays restated rather than imported, for the reason
`asset_holds` already gives about the naming rule: a judge sharing the
judged's own expression agrees with it by construction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 of 2 new guards RED, both on an ASSERT about behaviour (`assert True is
False`), and the second REPRODUCES PM's measured path end to end through the
real `okf build`: a BMP declaring 50 000 x 50 000, refused `asset_too_large`
and absent from `assets/`, gives `asset_holds = True` because an `<img alt>`
in the document states the conversion clause. The unit arms reproduce the
other two ways in -- ordinary body text and a table cell -- and a third the
report did not name: the clause inside a pointer block for a DIFFERENT asset.
The judge's first sentence is "THE FASIT NEVER COMES FROM THE READER IT
JUDGES". Before the conversion route landed, `asset_holds` hashed the source
file and looked in `assets/`, so no document could reach it; the route added
for converted images reads two digests out of the bundle text with a free
expression, and `claimed and not found` can now be silenced by a document
that asks for it.
Each arm carries a source that was never carried while the bundle holds one
unrelated REAL asset, so the digest the forgery names is genuinely in
`assets/` -- the property that made the measured forgeries work. The
known-positive on the same bytes keeps the arms from passing on a route that
has merely stopped working.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four cuts now raise `asset_samples_invalid`, the code the UNCOMPRESSED
path already raises for a body that stops short. No pixel is guessed: the loop
may end at an explicit end-of-bitmap escape and nowhere else, and running out
of bytes -- mid encoded run, mid absolute block, or one opcode before the
terminator -- is a refusal with a row instead of a partly blank PNG.
WHY THE TERMINATOR AND NOT `biSizeImage` OR A COVERAGE COUNT. `biSizeImage` is
a claim by the same untrusted header, and a coverage count would refuse the
delta escape the format defines. Read off the corpus instead: over the 19 real
RLE8 assets of the frozen R761 delivery, 19 of 19 end at an explicit
end-of-bitmap, on 19 of 19 it is the stream's LAST two bytes, and on 19 of 19
`biSizeImage` equals the available bytes. A whole stream that omits the
terminator is refused alongside a cut one, because from a reader's side they
are the same bytes and the measurement says no real writer here produces the
first.
An absolute run shorter than its own declared count also stops the loop rather
than painting what arrived, so the refusal names the cut and not the frame.
KNOWN-POSITIVE, re-measured on the frozen delivery with stdlib on BOTH sides
(an independent BMP reader and an independent `zlib` + filter-reversal PNG
decoder, no Pillow anywhere): 19 of 19 still convert losslessly, 2 366 365
pixels compared, 0 refused, and a one-byte control confirms the comparison can
fail. 92 passed over the three asset test files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 of 3 new guards RED, on an ASSERT about behaviour: `DID NOT RAISE
ExtractionError` on all four cuts. The other two are the section's own
known-positives and pass by describing what is already true -- where each
cut lands in the fixture's opcode stream, and that the UNCOMPRESSED BMP path
already refuses the same shape with `asset_samples_invalid`.
Measured by PM 2026-09-19 on a real R761 asset (352x548 = 192 896 pixels): a
stream cut to 90 % was carried with 13 923 pixels wrong, to 50 % with 95 890,
to 10 % with 166 525 -- no code, no row, a partly blank PNG standing under a
content address that says it holds the source's pixels.
The test brings its OWN permissive RLE8 decoder, so the pixel cost of each cut
is a number this file computes (8, 11, 0 and 0 of 32) rather than the package
agreeing with itself; two of the four cuts lose no pixel at all, which is the
whole reason the rule has to be the terminator and not a pixel count.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"0 across `tests/fixtures`, `examples`, `skills`, `docs`, `src`, README and
CHANGELOG" was published for all SIX characters and is false. Counted here
over `git ls-files` in those seven places, with a planted known-positive
proving the scanner can find one:
TRACKED in the seven places: 257 | readable: 230 | binary: 27
files carrying ANY of the six: 2
tests/fixtures/accounting/rejected-inventory.json {'200B': 1}
tests/fixtures/accounting/rejected/avvist.html {'200B': 1}
files carrying U+00AD: 0 []
KNOWN-POSITIVE planted file: {'00AD': 1, '2060': 1}
The two are this repo's own known-negative fixture for a carrier the door
refuses to remove. For U+00AD -- the character the door actually removes, and
the one the reach argument needs -- the figure is 0 of 230. (PM measured 229
on `43331fc`; the tree has gained one readable tracked file since.)
CHANGELOG under the untagged [0.10.1]: the three now-held sentences, the log's
stated provenance, and both corrected numbers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chose the stated provenance over a second, independent counter, because a
second counter would be a number built to disagree. The door acts on the
EXTRACTED text; a counter inside `accounting.py` would have to re-read the
SOURCE bytes, and for every type extraction does not carry verbatim (pdf,
xml, html, docx) those are counts over two different strings. The gate would
then have to decide which difference is a loss and which is extraction doing
its job -- inventing a discrepancy where there is none. `accounting.py` does
not hold the source bytes at that point either; being a second reader of a
document is what `tools/okf_witness.py` exists for, outside the package.
Where an independent denominator DOES exist the tests already pin it from
outside the door: the fixture writes its own five, and R761's 71 is PM's
count over the delivery.
The `**Normalisation**` bullet now ends: "The count is the door's own, read
off the run and not recounted from the source."
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`normalised_soft_hyphen` is read off the RUN (`result.normalised`) and never
recounted from the source. It is the one number the accounting publishes with
no independent denominator behind it, and nothing in the output said so.
RED on the claim:
assert "The count is the door's own, read off the run and not recounted" in log
AssertionError: ... '* **Normalisation**: 5 soft hyphen(s) (U+00AD) removed
from 1 of 1 document(s) before the persist gate. No other character is
touched.'
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
P11 and P12, PM 2026-09-19: the `refused={u.refused}` column on row 3's
detail line and the clause "N element(s) lost with R of D document(s) refused
whole" in its reason could each be deleted with this file green at 106
passed. The LOSS is held -- `Unit.refused` keeps the unit unclean and the
note names the source and its code -- but what the ROW says about it was
decoration nothing pinned, and the row is what a reader of the gate's output
sees first.
R, D and the element total are counted over the units the test builds, never
read back off the row. The known-negative is the same units with no refusal:
the sentence must move with them, or it is a constant that happens to read
true.
Red proven in a scratch copy of HEAD (`/tmp/shy-mut`), control green at 109
passed:
P11 detail line drops `refused={u.refused}` -> 1 failed (line 666)
P12 reason drops the whole "... refused whole" clause -> 1 failed (line 661)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`log.md` writes that sentence on every run and README and CLAUDE.md repeat
it. Until now nothing held it: PM's mutant P6 -- a door that ALSO removes
U+00A0 -- passed the entire suite green (2 171 passed, RC 0) while it would
have eaten all 6 633 NBSP in R761 with the log still claiming otherwise.
Ten characters, each a way for the door to reach past its own name: the four
real zero-width carriers, U+2060, U+00A0, the three hyphens a reader confuses
with a soft one (U+002D, U+2010, U+2011) and a combining accent (the door is
the one step that does NOT normalise to NFC). Each appears at least twice in
the fixture -- asserted, so the loop cannot pass over an empty set.
The invariant is the stronger of the two equivalent forms: rather than
reinserting the removed characters at their booked positions, the test builds
the expected string with its OWN filter over the source. That pins ORDER as
well as multiset, and needs no positions -- which the door does not return.
All three numbers (21 removed, the expected string, every surviving count)
come from the test, never from the door. The second test repeats it through
`extract_document`, the one place the door is applied, so a second remover
beside it is red too.
Red proven in a scratch copy of HEAD (`/tmp/shy-mut`, verified that the
scratch `src/` is the code that runs), control green at 7 passed:
P6 door also removes U+00A0 -> 2 failed
`assert text == expected` / `AssertionError: assert 'arbeider pa...'
== 'arbeider \xa...'`
P2010 door also removes U+2010 -> 2 failed
P6b door collapses U+00A0 to " " -> 2 failed
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Counted three ways over `N101-2025-860031.json`, none of them importing this
package: raw `tbx:` substring 568, regex over `"tag": "tbx:` 568, node
traversal 568 across 10 distinct tags (94 x 6 + 1 x 4), out of 7 374 tags in
the delivery. The number is corrected in all four places that published it --
CLAUDE.md, `tools/okf_witness.py`'s `_sts_role_json` docstring,
`tests/test_accounting_gate.py` and the round's report -- and the test from
the previous commit now counts it rather than repeating it.
The one remaining `574` in the tree is `docs/2026-09-11-k3-runde8...md:89`
("11 016 and 6 574"), a character count of two documents and a different
number.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The known-negative for stripping a namespace prefix -- "N101 ships 574 `tbx:`
tags, and not one of their local names is a role" -- was published in four
docstrings (CLAUDE.md, tools/okf_witness.py, this file, the round's report)
and asserted nowhere. A measurement presented as a fact, with nothing that
could turn red when it stopped being true.
The walk is written HERE rather than taken from `okf_witness`: the witness's
own reader is what this known-negative judges, so counting through it would
make the two agree by construction.
RED on the claim, not on an import:
assert sum(tbx.values()) == 574
AssertionError: assert 568 == 574
where 568 = sum(dict_values([94, 94, 94, 94, 94, 94, 1, 1, 1, 1]))
Three independent counts of the delivery agree on 568 -- raw `tbx:` substring,
regex over `"tag": "tbx:`, and node traversal -- against 574 in the tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README, CHANGELOG (under the untagged 0.10.1), CLAUDE.md and the round's
report. The numbers are the census's, not a summary of them.
- 9 714 image rows over 18 403 files and 67 PDFs, two pinned trees: exactly
35 rows moved, 9 321 of 9 321 JPEG and PNG rows byte-identical.
- The 16 the round did not go looking for: JPEG 2000 objects out of PDF
streams, also unreadable by a model, with no stdlib route to convert them.
They become `asset_not_viewable` -- 16 pictures lost from a bundle, 16
statements gained that a picture stood there. Stated in the README, the
CHANGELOG, CLAUDE.md and the report rather than left in a total.
- The accounting gate: R761 0 -> 19 claimed-and-not-found when the conversion
landed, 0 again after the judge learned its second route; row 6 90 of 111
-> 110 of 111. The gate's verdict is the same `RED: rows 2, 3, 6` as before
the round, with row 3's numbers identical.
- `<!-- asset-viewable-media-types: ... -->` is pinned to
`assets.VIEWABLE_MEDIA_TYPES` by a test, compared as a whole set: a README
naming three of four would pass every containment check and tell a consumer
that a format is refused when it is carried.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chosen: a stdlib BMP reader, because `read_image` is on the CORE path and an
asset's name is its content digest. Measured first, as the order requires:
Pillow 12.3.0 IS in this tree (transitively under `pdfplumber`) and it DOES
decode RLE8 correctly -- a hand-written stdlib decoder and Pillow agree on
19 of 19 of R761's real files, RGB per pixel. So the choice does not rest on
capability. It rests on two properties of this package: `.html` and `.xml`
carry images with no `[extract]` extra installed, so a Pillow converter
either makes a core path depend on an optional binary wheel or buys the
second runtime dependency; and encoding through an installed library would
make a bundle's identity move with that library's version, which is the
property 0.10.0 felled page rasterisation over and `encode_png`'s docstring
already defends. Pillow keeps the job it is good for: the INDEPENDENT decoder
in the tests, on neither side of the conversion.
The defect, measured over the frozen R761 delivery's `assets/`, denominator
50: 29 JPEG, 2 PNG and 19 RLE8 BMP. The 19 are byte-correct files nothing
reads, so 19 figures were present and invisible while `images: N` reported
that they had arrived.
- `VIEWABLE_MEDIA_TYPES` is tested against every asset's SNIFFED type, so it
is a property and not a list of formats we met. WebP is on it and `sniff`
does not recognise one; the limit is stated, not implied.
- `bmp_to_png`: 8-bit uncompressed, 8-bit RLE8, 24-bit uncompressed. All five
RLE8 opcodes. 19 of 19 real files convert with RGB identical to Pillow's
decoding of the source, 2 366 365 pixels compared.
- `asset_not_viewable` and `asset_bmp_unsupported`, both published, both
leaving the concept's "not carried" line.
- Traceability on the pointer's second line, where the rest of the asset
metadata already lives: original media type, original sha256 in full, new
sha256 in full. A converted asset is ONE asset.
- The ceiling is paid on the DECLARATION before a row is allocated, and an
RLE run is one clipped slice -- painting pixel by pixel leaves the memory
bounded and the CPU unbounded.
Two repairs the change forced, each measured rather than assumed:
- `tests/test_assets.py`'s "dimensions absent is absent" used a TIFF, which
is now refused before `read_image` returns. The property still has a
reachable case -- a JPEG whose frame header never arrives -- and uses it.
- `asset_holds` in the accounting gate proved a carry by hashing the SOURCE
file, which a converted image's bundle cannot satisfy. It now also reads
the two digests the bundle states and HASHES THE ASSET ITSELF, so a bundle
claiming a conversion it did not perform still fails.
`tools/okf_asset_census.py` is the committed instrument for the
known-positive: one row per image, from two pinned trees. It was caught by
the rule it serves -- its first version handed `_pdf_images` the wrong page
object and reported 0 images over 67 PDFs with exit 0. The attribute is
asserted now and a known-positive runs before the sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found by running the gate against the working tree: R761 goes from 0 to
**19 claimed and not found** the moment the conversion lands, and 19 is
exactly its RLE8 BMP count.
The cause is a rule that was right until this round. `asset_holds` proves a
carry by hashing the SOURCE file and looking for those bytes under their own
content address in `assets/`. A converted image's source bytes are not in the
bundle and never will be -- the run carried the picture under a new digest,
which the bundle states on the pointer line beside the old one.
Red on the claim: `asset_holds` returns False for a build whose bundle
declares the conversion and whose `assets/` holds the named file. Two
known-negatives are in the same test so the second route cannot become a way
of believing the report: a bundle that claims a conversion and ships no file,
and a bundle shipping a file under a digest it never tied to this source.
Both must stay False, because the judge has to hash the asset itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Written after walking the five mutants the order names against the eval as
committed. Two of them SURVIVED it, which makes them holes in the eval and
not in the code that does not exist yet.
1. "the format is read from the file extension instead of the bytes" survived,
because every image in the fixture is named after what it is: a `.bmp` maps
to image/bmp either way and a `.tiff` to image/tiff either way, so the
fixture could not tell a sniffed type from a claimed one. A BMP named
`graphics/figur.png` can. Red today: `image/bmp` != `image/png`.
2. "the ceiling is checked after decoding instead of before" survived because
the guard only asserted that the refusal happens, and `encode_png`'s own
`check_size` refuses too -- one frame later, after the memory is spent. The
guard already measured the peak; what it could not do was measure it
affordably, because a 50 000 x 50 000 frame is 2.5 GB. At 7 000 x 7 000 the
declaration is still over the 40 MP bound and the unbounded frame is 49 MB
-- measurable, and two orders of magnitude over the 4 MB the guard allows.
Nine of thirteen guards are now red on their claim; the four green ones state
properties that already hold.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The eval the round is judged on, written before the decoder exists. Eight of
twelve guards are RED on their claim, four are green because they state
properties that already hold.
Red, and each on an assertion rather than an import:
1. Visibility as a PROPERTY of every carried asset, read off the bytes: 3 of
3 carried assets in the fixture bundle are image/bmp or image/tiff, which
no model displays. The denominator is counted in the test from the inbox,
so a build that carried nothing could not pass over an empty set.
2. Lossless: 0 carried assets in the viewable set hold the source's pixels.
The holder must BE viewable -- a BMP carried verbatim trivially holds its
own pixels, so a guard without that clause would be green today.
3. Traceability: the bundle states neither source checksum and neither
original format.
4. One asset, not two: the assets directory holds 3 files for 2 convertible
images plus one that should be refused.
5. An RLE stream far longer than its declared 8x4 frame is carried, in a
format nothing reads.
6. A TIFF and a truncated BMP are both carried silently; neither raises.
7. The concept says "not carried" nowhere.
Green already: the fixture's own known-positive (Pillow decodes the
hand-built RLE8 opcode stream to the literal grid this file states), the two
byte-identity controls for PNG and JPEG, and the bound on a BMP declaring
50 000 x 50 000.
The fixture is built byte by byte in the test and exercises all five RLE8
opcodes -- encoded run, absolute run with its pad byte, delta, end-of-line,
end-of-bitmap. A fixture of runs alone would pass with a decoder that
implements none of the other four. The independent decoding is Pillow, which
this package neither uses nor ships, so "identical pixels" is not this
module agreeing with itself; the viewable set and the rejection codes are
written out in the test rather than imported, for the same reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `if False:` in this diff is a mutation DEFINITION -- a string in
`MUTANTS`, applied only to a throwaway copy of the tree inside the
harness and restored in a `finally`. No branch in this repository is
pinned by it.
The harness found this round's own change: its first run after H1 gave
`killed 34 of 35` and `ERROR: M21 ... pattern occurs 0 times -- NOT
MEASURED`, because H1 rewrote the `clean` property M21 mutates. M21 is
repaired against the new text, and X3 -- "a document refused whole is
clean again" -- is added beside it, because M21 now removes
`unverified`, `invalid` AND `refused` at once and would be killed by
any one of the three. Final run: killed 36 of 36, 0 survived, 0
errors, exit 0.
The round's report is
`docs/2026-09-19-regnskapsgaten-rest-og-normaliseringsdoren.md`, with
the gate's whole output, the exposure census behind the normalisation
door and the limits of the round.
CHANGELOG: this round's entries are folded into the UNTAGGED `[0.10.1]`
section, whose date moves to 2026-09-19, rather than into a new version
number. `v0.10.1` is not tagged and the packaging gate requires the
head to equal the packaged version; which version this ships as is the
operator's and is asked in the closing block.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README gains the partial-refusal paragraph and the soft-hyphen door
with its measured reach; CHANGELOG gets an Unreleased section over
this round; CLAUDE.md's open commitment "R761 is refused whole by
guard 1.4.0 ... that is the security repo's question" is replaced by
what was decided and built, with the alternatives that were felled and
the numbers behind each rule.
No version bump and no tag: `v0.10.1` is not tagged yet and releasing
is the operator's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operator decision 2026-09-18, and the whole of it: okf removes the soft
hyphen, the guard is untouched, and the number is BOOKED.
`normalise_extracted(text) -> (text, removed)` in `extract`, applied
once at the end of `extract_document`, so both entry points and both
sides of a plan's `text_sha256` see one string. `_pdf_units` measures
its page offsets through the same door, because that table is
CHARACTER offsets rebuilt from the pages while the text it indexes
comes back normalised -- two readings of one document, and a table
built against the other names the wrong page with full confidence.
Removing a character removes no newline, so every line-indexed rule
downstream is unmoved by construction.
`ExtractedDocument.soft_hyphens` carries the count out;
`InboxResult.normalised` carries it per document; `account_run` reads
it off the RUN rather than recounting the source, because a second
count would be a second reader. It reaches the accounting JSON as
`normalised_soft_hyphen` at both levels and `log.md` as a
`**Normalisation**` bullet naming the count, the documents touched and
that no other character is moved.
EXPOSURE, measured and not assumed: 0 of the 78 readable documents of
the K2 reference corpus carry U+00AD or any of the four real
zero-width characters (the 8 unreadable ones raise before extraction),
0 in the pinned K2 bundle's concept bodies, and 0 across
`tests/fixtures`, `examples`, `skills`, `docs`, `src`, README and
CHANGELOG. The door cannot have moved a byte anyone here has measured.
Suite 2179 passed, 1 skipped; ruff and `mypy --strict src/` clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operator decision 2026-09-18, the alternative chosen verbatim: okf gets
ONE documented normalisation door before the guard, removing U+00AD and
BOOKING the count in the content accounting. The guard is not touched.
The defect it answers, PM's measurement: R761 Prosesskoden:2025 is
refused whole by guard 1.4.0 as `output:zero-width-present` -- an
any-tier carrier, `fail_secure` at every trust level -- over 71 U+00AD
and 0 real zero-width characters. The 71 are Norwegian hyphenation
points inside words. Reproduced here in the small on this tree before
any code moved: three soft hyphens in one markdown file give `0 of 1
extracted document(s) persisted; rejection codes: fail_secure 1` and
exit 1.
Five tests, each red on its claim:
- the source passes the DEFAULT gate, the accounting carries
`normalised_soft_hyphen: 5` at both levels, and `log.md` says it;
- text preservation as an EXACT invariant and never a share: the
extracted text of the hyphenated source EQUALS the extracted text of
the same source written without them;
- the known-negative, U+200B, still `fail_secure` and still in the
text -- removing a real carrier would be the guard's decision and
would take a screen away from every consumer;
- a PDF's `source_pages` table is CHARACTER offsets rebuilt from the
pages while the text comes back through the door, so both must be
measured over the same rendering or every locator drifts;
- R761's own 71, on the delivery the decision was taken for.
U+00A0 NBSP is untouched: `_ZERO_WIDTH_CPS` is exactly {200B, 200C,
200D, FEFF, 00AD} on guard 1.4.0, read off the installed source, so
the guard has no rule about NBSP and nothing here needs one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`measures_no_class(units)` returns a sentence when EVERY document of a
corpus has no declared fate, and row 6 prints it beside that corpus's
tally. On N200 that is 16 549 elements of `u` which look like a finding
about the build and are a finding about the run: the build exits 2 on
0 proposed plans before the accounting door is reached, so no
accounting file exists to read a fate from.
The function's docstring carries what it would take, because that is a
capability and not a threshold: `.json` is read as generic JSON, and
the publisher's STS delivery in that form would have to reach the
markdown grammar `_extract_xml` writes for the XML one. Measured
2026-09-19, with the query validated against a known positive:
`standardContent` occurs 0 times in `src/` and 4 in the witness.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reproduced 2026-09-19: `okf build` on N200 proposes 0 plans, prints
`FAILED - no segmentation plans` and exits 2 BEFORE the accounting door
is reached -- no accounting file is written at all. Every one of the
16 549 elements then lands as `u` with `no declared fates`, which reads
like a finding about the build and is a finding about the run.
Two tests and a known-negative, so a rule that calls every corpus blank
fails too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`real_corpora(R761_DEFAULT, N200_DEFAULT)` instead of the parsed
arguments. Row 6 is SKIPPED exactly when the corpora the arguments
name are absent, so asking the arguments made the branch unreachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The guard at the end of `main` asked whether the corpora named by the
ARGUMENTS are available. Row 6 is SKIPPED precisely when none of them
is, so the branch could never fire; PM read it as dead code and no test
covered it. The question it meant to ask is about the machine: a corpus
that is HERE and was pointed away from is a row that did not run, and
with `CI=1` the gate then printed a qualified GREEN and exited 0.
Two tests, so a guard that fires on everything fails too: with the
default source present and the arguments pointing elsewhere the run
exits 1 and names the row; with no corpus on the machine at all it
exits 0, because nothing to measure is not a row that did not run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two edits, one line each. `count_sts_json` takes the tag through
`_local` the way `count_sts_xml` always has, and `_sts_role_json` gains
a `figcaption` clause beside the NISO-STS `fig/caption` one. The
`figcaption` clause is in the JSON map ALONE: the XML map has never
been measured carrying one, and the two maps are written twice on
purpose so row 5 can see a hole in either.
NOT ONE OTHER COUNT MOVES, and it is measured rather than argued.
Old rule against new over the eight JSON deliveries on this machine
and the three committed twins, role by role:
N100-2022 figure_caption 0 -> 123
N100-2023 figure_caption 0 -> 127
N101-2025 figure_caption 0 -> 62
N200-2024 figure_caption 0 -> 49, math 0 -> 74
N303-2026 figure_caption 0 -> 43, math 0 -> 3
N500-2024 figure_caption 0 -> 27, math 0 -> 88
N601-2022 no change
R761-2025 no change (row 5's pair, unmoved)
the two existing twins no change
N101 ships 574 `tbx:`-prefixed tags and not one of their local names
is a role, which is the known-negative the prefix rule is measured
against.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM measured it on N200 Vegbygging:2024: 74 `mml:math` carrying 2 811
characters and 49 `figcaption`, and the witness counted 0 and 0. Two
independent causes and both are in this red:
- `count_sts_xml` takes every tag through `_local`, which strips a
Clark-notation namespace AND a prefix; `count_sts_json` compared the
raw string, so `mml:math` never reached `tag == "math"`.
- the publisher's JSON puts a figure's caption in `figcaption` under
the `graphic`, not in the `fig/caption` NISO-STS writes. Measured
over the eight deliveries on this machine: 430 `figcaption`.
Three tests. The JSON half of `sts-rikt.xml`'s family carries both
shapes as a committed fixture; the real N200 carries PM's four numbers
and skips where the corpus is absent; and the known-negative for
stripping a prefix is N101's 574 `tbx:` tags, whose local names name
no role -- a generalising rule measured against what it must NOT take.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mirror of the `persisted` refusal, driven from both sides: a report
declaring `rejected` while a concept in the bundle names the document
is `invalid >= 1` with the note quoted, and the same declaration with
its elements booked as coded rejections and no concept naming it is
`invalid == 0` and `refused == 3`. Without the second arm the test
would pass on a gate that refused every rejected document.
Measured against the X2 mutant from the previous commit: rc 0 before
(98 passed), rc 1 after.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM wrote this mutant outside the tree 2026-09-18 and it survived 155
green tests. Reproduced here on the mutant as committed: `98 passed`,
rc 0, the pattern applied exactly once.
`_document_unit` refuses a report that declares a document `rejected`
while a concept in the bundle names it -- one of the six refusals B-1
introduced, and the only one no test drives from either side. The check
works; nothing stops the next refactor removing it. With H4 in place
the harness now exits 1 on this survivor, so the red is the harness's
own verdict and not a line in its output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`verdict(survived, errors)` is the one place the run's exit code is
decided: 2 when a mutant could not be applied (it was never measured,
and that outranks everything), 1 when one survived, 0 otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The harness returned `2 if errors else 0`, so a run printing `killed 0
of 1` and naming its survivor exited 0. PM measured it on a copy
carrying only the X2 mutant. Red on the claim: the module imports, and
offers no verdict to ask.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gate gains a FIFTH column, `Unit.refused`, and it is the only one
that is not a defect in the report: the elements of a document the
build read and persisted nothing of. Their fate is declared honestly,
so `unaccounted` and `double_booked` both stay 0 -- which is exactly
why nothing else could see the loss. `refused_whole` asks its question
only for a corpus that persisted NOTHING, so one refused source beside
an accepted one, the ordinary case on a heterogeneous corpus, reached
row 3 as clean.
Row 3's reason now carries `N element(s) lost with R of D document(s)
refused whole`, and each unclean unit's detail line carries
`refused=` beside u, d, unverified and invalid, with the document's
own rejection code in the note.
On the build side `Accounting.refused` is written into the JSON and
into the `**Accounting**` bullet of `log.md` as `R of D document(s)
refused whole`. The exit code is NOT moved: it belongs to the whole
run, and a corpus holding one unreadable file among many is ordinary,
so the order's other half -- state it in the accounting -- is the one
taken. `okf build` still exits 1 when it persisted nothing at all.
`test_a_corpus_refused_whole_under_the_default_gate_is_red` kept its
point and lost its premise: the numbers still balance, and that is now
asserted as u = 0 and d = 0 rather than as a clean unit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's checkpoint of ed8d9d7 measured it: an inbox holding one refused
document and one accepted one gave `clean = 4 of 4`, `refused_whole =
None` and `okf build` exit 0, while three elements of `avvist.html`
were gone unseen. `refused_whole` asks its question only when the
corpus persisted NOTHING, and a refused document's own unit balances
by construction -- every element booked as a coded rejection is u = 0
and d = 0.
Three tests, all red on the claim and not on an import:
- the unit of a document the build refused is not clean, and its note
names the code;
- the same beside a PERSISTED neighbour, which is the ordinary case on
a heterogeneous corpus, with row 3 RED and the source named in the
detail line;
- `okf build`'s own report says how many of M documents it refused
whole -- the exit code belongs to the whole run, so the count has to
stand in the accounting and in `log.md` or the loss is silent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 3 of the 0.10.1 review, and the finding is the pattern the three rounds
share: each bound an OUTPUT, and the bomb stepped one link along. The
declared size, then the first `FlateDecode`, then every `FlateDecode` -- and
then a link this package had documented as safe.
`ASCII85Decode` was classed as bounded "by its own input because it shrinks".
It quadruples: `z` is the shorthand for four zero bytes. And the output was
never the cost -- `base64.a85decode` appends one 4-byte object per group to a
list, about a hundred bytes of memory per byte of INPUT (101.4x at 1 MiB,
96.1x at 4 MiB, 94.5x at 16 MiB on CPython 3.14).
Paired subprocesses, idle machine, both sides from PINNED trees, the document
built once by a third process and read from a file because `ru_maxrss` never
falls and `b"z" * 64 MiB` alone costs 171 MB:
[/Fl /A85] z x 32 Mi 33 475 B CARRIED 3 261 599 744 -> too_large 42 070 016
[/Fl /A85] z x 64 Mi 66 090 B CARRIED 6 461 558 784 -> too_large 40 280 064
[/A85] z x 8 Mi 8.4 MB CARRIED 933 085 184 -> too_large 62 484 480
[/Fl /A85 /Fl] z x 32 Mi 33 488 B samples_invalid 3 519 180 800 -> too_large 43 438 080
The picture was CARRIED in three of the four: not a bound that fired late, no
bound at all. Doubling the `z` run trebles the old cost and leaves the new one
where it was.
WHY THIS FORM. `assets.MAX_FILTER_DECODE_BYTES` (512 MiB) is what decoding ONE
link may cost -- a separate number from `MAX_IMAGE_BYTES`, because that one
bounds the picture and this one bounds producing it. `FlateDecode` is measured
as it is paid; every other permitted filter carries a MEASURED cost ratio
(`assets.PDF_FILTER_COST_RATIO`) checked against its input BEFORE its decoder
is called, since those decoders take a whole string and return a whole string.
A filter with no ratio is refused unread. The budget TRAVELS: a deflate link
is inflated under the smaller of the picture's bound and what the next link's
decoder may be handed, or `[/Fl /A85]` pays 256 MiB for a refusal.
A chunked ASCII85 decoder written here was the alternative and was FELLED: it
would bound `_check_stream_cost` and not the run, because `stream.get_data()`
decodes the whole chain again with pdfminer's own decoder, and it would make
this package rather than pdfminer the authority on an image's bytes. The cap
is the only number that bounds that. `resource.setrlimit(RLIMIT_AS)` was
MEASURED before anything was built on it, as the order required, and is not
usable: Darwin 26.6.2 raises `ValueError: current limit exceeds maximum limit`
and does not enforce it. No child-process cap exists.
THE CAP IS READ OFF THE CORPORA, the posture `MAX_IMAGE_PIXELS` has: over the
9 668 image objects of the 77 PDFs on this machine, 16 decode through an
ASCII85 link and the largest input to one is 450 739 bytes, against a cap of
about 5.0 MB.
A PROPERTY TEST REPLACES THE LIST OF KNOWN SHAPES: every chain of length 1-3
over the ten filters pdfminer decodes, 1 110 of 1 110, both payload fills,
each delivered under the bound or refused with a published code and never paid
for on the way (`tracemalloc`, which counts allocations and is not disturbed
by load). Known-positive beside it: 258 of 258 chains over the permitted
filters still carry a small image.
MAJOR -- the backstop had no test. `check_payload` at the end of
`_check_stream_cost` could be deleted with the whole suite green, because the
second one after `get_data()` gives the same code one step later. The two
differ in whether the payment was made, so the test asserts `get_data` was
never called.
10 OF 10 MUTANTS KILLED, control green, each killer named in the report. Four
survived a first pass and two tests exist because of it.
NOT ONE PICTURE CHANGES HANDS, MEASURED BY NAME: `_pdf_images` over every PDF
on this machine from both pinned trees -- 9 306 -> 9 306 carried over 77
files, 50 -> 50 on R761, 0 of 78 files moving a count and 0 moving a code.
R761 also settles a question raised while this order was open: 50 objects, 29
[/DCTDecode], 21 [/FlateDecode], 0 ASCII85 links -- so round 2's count of 580
`[/FlateDecode /ASCII85Decode]` objects is reproducible from nothing on this
machine. It changes no decision; a bomb shape does not need a corpus.
Version stays 0.10.1, no tag. README, CHANGELOG, CLAUDE.md and errors.py
corrected TO what the code does; the round-2 report carries a correction block
rather than a rewrite. Report:
docs/2026-09-18-utgangsbudsjett-per-ledd.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 3 of the 0.10.1 review. Rounds 1-3 each bound a NUMBER and the bomb
moved to the next one: the declared size, then the first `FlateDecode`, then
every `FlateDecode`. It now lives in a link this package had documented as
safe. `ASCII85Decode` was called bounded "by its own input because it
shrinks"; `z` is ASCII85's shorthand for four zero bytes, so one input byte
becomes four, and `base64.a85decode` appends one 4-byte object per group to a
list, so the DECODER costs about a hundred bytes of memory per byte of input
(measured on CPython 3.14: 101.4x at 1 MiB, 96.1x at 4 MiB, 94.5x at 16 MiB).
Measured on the pinned `0c3c490` tree, its own interpreter, idle machine: a
33 475-byte PDF decoding an image through `/Filter [/FlateDecode
/ASCII85Decode]` cost 3 827 003 392 bytes of peak RSS -- 114 000x the file --
and the picture was CARRIED, with no rejection at all.
Seven tests red, three green:
* the two bombs above, in their own interpreters, at the shipped bound;
* the cost ratio as a re-measurable known-negative, so the constant cannot
rot the first time CPython changes `a85decode`;
* the input cap against the corpora (9 668 image objects over 77 PDFs, 16
behind an `ASCII85Decode` link, largest input 450 739 bytes);
* THE PROPERTY, both payload fills: every chain of length 1-3 over the ten
filters pdfminer decodes -- 1 110 of them -- is either delivered under the
bound or refused with a code in the published vocabulary, and never paid
for on the way, which `tracemalloc` measures because that is where
`a85decode`'s cost lives.
Green but previously uncovered, which is the MAJOR of the same checkpoint:
`check_payload` at the END of `_check_stream_cost` could be deleted with the
whole suite still passing, because the second one after `get_data()` gives
the same code one step later. The two differ in whether the payment was made,
so the test asserts `get_data` was never called. The known-positive beside
the property -- every bounded chain still carries a 64-byte image -- is green
too, and a rule that refuses everything would pass the property alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two findings of the 18.09 PM checkpoint of `0f308c1`. Red tests landed
first in `3b587ea`; this is what turns them green.
BLOCKER -- `_check_inflated` read `filters[0]`, measured that one link and
returned, which is not a bound: a PDF decodes a stream through a LIST of
filters. Measured in paired subprocesses from two pinned trees, idle machine:
[/FlateDecode] 400 MB 408 516 B 59 232 256 -> 62 017 536 B
[/FlateDecode x2] 400 MB 1 636 B 886 554 624 -> 52 367 360 B
[/FlateDecode x3] 400 MB 1 070 B 889 393 152 -> 61 390 848 B
[/FlateDecode x2] 1,2 GB 2 927 B 2 567 204 864 -> 60 403 712 B
542 000x the file at two links, and the picture WAS refused at the end -- by
`check_payload` after `get_data()`, once the memory was spent. The single-link
row is the control and does not move. It also left the 16 corpus objects behind
an `[/ASCII85Decode /FlateDecode]` chain unmeasured, since `filters[0]` is not
`FlateDecode` there.
`_check_stream_cost` walks every link. THREE CLASSES and no fourth
(`extract.bounded_pdf_filters`, pinned by a test): `FlateDecode` MEASURED, a
link with another expanding link behind it inflated under the same bound and
handed on; `ASCII85Decode`/`ASCIIHexDecode` bounded by their own input because
they SHRINK; `DCTDecode`/`JPXDecode`/`JBIG2Decode` PASS THROUGH. Everything
else -- `LZWDecode`, `RunLengthDecode`, `CCITTFaxDecode`, `/Crypt`, anything
written later -- is refused UNREAD with a new code `asset_pdf_unbounded`,
decided before the FIRST link is decoded so a document cannot make the run pay
for the links in front of the one we cannot bound. An encrypted stream is
deciphered and then measured, where `stream.decipher is not None` used to
return unmeasured; 0 of 5 142 objects here are in an encrypted document, which
is why nothing caught it.
NOT ONE PICTURE CHANGES HANDS, AND IT IS MEASURED BY NAME. Every PDF on this
machine -- 78 documents, K2 in both trinn1 and trinn2, the shipped fixtures and
R761 -- run through `_pdf_images` page by page from both pinned trees:
images carried 9 356 -> 9 356
documents losing one 0 of 78
documents gaining one 0 of 78
asset_pdf_unsupported 322 -> 314
asset_pdf_unbounded 0 -> 8
The 8 are the 4 `CCITTFaxDecode` stencil masks (`/ImageMask true`,
`/BitsPerComponent 1`), counted twice because trinn1 and trinn2 hold the same
document. They were refused before and are refused now, one step earlier and
under a code that says why.
MAJOR -- `check_payload(len(data))` after `get_data()` is the counted refusal
four documentation surfaces point at, and deleting exactly that line passed all
2 132 tests. It is reachable through a stream pdfminer has ALREADY decoded
(`decode()` sets `rawdata` to `None`), which is now the ONLY case outside the
bound and has a test.
Eight mutations, one line each, every one DEAD, with the unmutated tree run
first as the control: first-link-only, loop dropped, inequality reversed,
encrypted skipped, backstop deleted, unknown filter passed through,
intermediate link not carried forward, whole check removed.
`tools/okf_accounting_gate.py` gains one line, the new code in
`REJECTION_CODES` -- what a rejection code requires and nothing more. Gate
unchanged: exit 1, GATE RED rows 2, 3, 6. Version stays 0.10.1, untagged.
Suite after `git add` against a clean tree: `uv run pytest -q` ->
2152 passed, 1 skipped (226 s). ruff, ruff format --check, mypy --strict clean.
Report: docs/2026-09-18-filterkjeden-og-backstoppen.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
