open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(mutants): repair M21, add X3 [mutation-ok], and the round's report

Browse source 
The `if False:` in this diff is a mutation DEFINITION -- a string in
`MUTANTS`, applied only to a throwaway copy of the tree inside the
harness and restored in a `finally`. No branch in this repository is
pinned by it.

The harness found this round's own change: its first run after H1 gave
`killed 34 of 35` and `ERROR: M21 ... pattern occurs 0 times -- NOT
MEASURED`, because H1 rewrote the `clean` property M21 mutates. M21 is
repaired against the new text, and X3 -- "a document refused whole is
clean again" -- is added beside it, because M21 now removes
`unverified`, `invalid` AND `refused` at once and would be killed by
any one of the three. Final run: killed 36 of 36, 0 survived, 0
errors, exit 0.

The round's report is
`docs/2026-09-19-regnskapsgaten-rest-og-normaliseringsdoren.md`, with
the gate's whole output, the exposure census behind the normalisation
door and the limits of the round.

CHANGELOG: this round's entries are folded into the UNTAGGED `[0.10.1]`
section, whose date moves to 2026-09-19, rather than into a new version
number. `v0.10.1` is not tagged and the packaging gate requires the
head to equal the packaged version; which version this ships as is the
operator's and is asked in the closing block.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 06:32:35 +02:00
commit 43331fc4d4
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
3 changed files with 253 additions and 34 deletions
Download patch file Download diff file Expand all files Collapse all files

58
CHANGELOG.md
View file

@ -5,7 +5,7 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.10.1] — 2026-09-19
### Added
@ -28,37 +28,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
report and `log.md` now say how many of M documents the run persisted nothing
of. The exit code is unchanged — it belongs to the whole run.
### Fixed
- **The content-accounting gate: a document refused whole is never clean.**
Its elements are all booked as coded rejections, so u = 0 and d = 0, and
`refused_whole` asks its question only for a corpus that persisted NOTHING —
one refused source beside an accepted one read as clean with the content
gone. `Unit.refused` is that loss with its own column and the document's code
in the note.
- **The STS JSON role map reads the publisher's own tags.** `count_sts_json`
compared the raw tag string where the XML witness has always used `_local`,
so `mml:math` reached `tag == "math"` on nothing — 74 formulas in N200
Vegbygging:2024 counted as 0. And the publisher's JSON writes a figure's
caption as `figcaption` under the `graphic`, not as the `fig/caption`
NISO-STS writes — 430 of them over the eight deliveries measured. No other
count moves, measured role by role over those eight and the committed twins.
- **The mutation harness is a gate.** A surviving mutant now exits 1; the run
ended `2 if errors else 0`, so `killed 0 of 1` exited 0. PM's X2 mutant — a
report may declare a document rejected while the bundle holds it — is in the
set and is killed by a new test driven from both sides.
- **The skipped-row guard measures the machine, not the argument.** Row 6 is
SKIPPED exactly when the corpora the arguments name are absent, so asking the
arguments made the branch unreachable.
- **Row 6 says when a corpus measures no element class at all.** On N200 the
build proposes 0 plans and exits 2 before the accounting door, so 16 549
elements land as unaccounted with no declared fate — a finding about the run
that read as a finding about the build.
## [0.10.1] — 2026-09-18
### Added
- **`okf build --accounting PATH`: content accounting per element.** Before
extraction, every source document is inventoried in a per-format element
vocabulary: headings, paragraphs, tables, cells, images, and so on. After
@ -241,6 +210,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- **The content-accounting gate: a document refused whole is never clean.**
Its elements are all booked as coded rejections, so u = 0 and d = 0, and
`refused_whole` asks its question only for a corpus that persisted NOTHING —
one refused source beside an accepted one read as clean with the content
gone. `Unit.refused` is that loss with its own column and the document's code
in the note.
- **The STS JSON role map reads the publisher's own tags.** `count_sts_json`
compared the raw tag string where the XML witness has always used `_local`,
so `mml:math` reached `tag == "math"` on nothing — 74 formulas in N200
Vegbygging:2024 counted as 0. And the publisher's JSON writes a figure's
caption as `figcaption` under the `graphic`, not as the `fig/caption`
NISO-STS writes — 430 of them over the eight deliveries measured. No other
count moves, measured role by role over those eight and the committed twins.
- **The mutation harness is a gate.** A surviving mutant now exits 1; the run
ended `2 if errors else 0`, so `killed 0 of 1` exited 0. PM's X2 mutant — a
report may declare a document rejected while the bundle holds it — is in the
set and is killed by a new test driven from both sides.
- **The skipped-row guard measures the machine, not the argument.** Row 6 is
SKIPPED exactly when the corpora the arguments name are absent, so asking the
arguments made the branch unreachable.
- **Row 6 says when a corpus measures no element class at all.** On N200 the
build proposes 0 plans and exits 2 before the accounting door, so 16 549
elements land as unaccounted with no declared fate — a finding about the run
that read as a finding about the build.
- **An inline PDF image gets a stable name (0.10.1).** pdfminer names an
inline image (`BI … EI`) from `id()` of a Python object, so a pointer line
changed between two runs of one build and two concept files of the reference

215
docs/2026-09-19-regnskapsgaten-rest-og-normaliseringsdoren.md Normal file
View file

@ -0,0 +1,215 @@
# The accounting gate's remainder, and one normalisation door
`llm-ingestion-okf`, 2026-09-19. Order `20260918T110211Z-9941518407`, raised
from PM's re-measurement of `ed8d9d7`
(verdict: PARTIAL). Six holes, each with a red test written in its own commit,
plus the operator's decision of 2026-09-18 about U+00AD.
Nothing is tagged, pushed or released here.
## 1. What the round closes
| # | Hole | Closed by | Red first |
|---|---|---|---|
| H1 | a partly refused corpus is "clean" | `Unit.refused` + `Accounting.refused` | `5d1060d` -> `d27ca50` |
| H2 | one of B-1's six refusals held by no test | mutant X2 + a two-sided test | `a59898e` -> `a94b0c5` |
| H3 | `math`/`figure_caption` do not fire on a real delivery | `_local` on the JSON tag + a `figcaption` clause | `37599a3` -> `d31ab73` |
| H4 | the mutation harness is not a gate | `verdict(survived, errors)` | `6abe4b4` -> `f5b263f` |
| H5 | the skipped-row guard is unreachable | it measures the DEFAULT sources | `253c4fb` -> `8b4f409` |
| H6 | N200 contributes one blank red | `measures_no_class` in row 6 | `e6d9f16` -> `280d6bd` |
| — | R761 refused whole over 71 soft hyphens | `extract.normalise_extracted` | `68cfa90` -> `eebaf53` |
## 2. The normalisation door (operator decision 2026-09-18)
`llm-ingestion-guard` 1.4.0 keeps U+00AD in `_ZERO_WIDTH_CPS` beside U+200B,
U+200C, U+200D and U+FEFF, and `output:zero-width-present` is an **any-tier
carrier**: `fail_secure` at every trust level, with no sanitisation and no
exception. R761 Prosesskoden:2025 carries **71 U+00AD and 0** of the four real
zero-width characters (U+2060 also 0). All 71 are Norwegian hyphenation points
inside words -- `ar[SHY]beider`, `bitu[SHY]men`, `asfalt[SHY]betong` -- so a
701-page process code was unreadable for the whole chain over typography. The
verdict was formally right and materially a false positive.
Of the three ways out, the operator chose one door in okf, before the guard,
that removes the character and BOOKS the number. The other two were felled:
weakening the guard's rule is the security repository's call and takes a screen
away from every consumer; delivering a cleaned corpus fixes one document and no
other.
**Where it is.** `extract.normalise_extracted(text) -> (text, removed)`, applied
once at the end of `extract_document`, so both entry points and both sides of a
plan's `text_sha256` see one string. `_pdf_units` measures its page offsets
through the same door: that table is CHARACTER offsets rebuilt from the pages
while the text it indexes comes back normalised, and a table built against the
other rendering names the wrong page with full confidence. Removing a character
removes no newline, so every line-indexed rule downstream -- the proposer's
grammars, `xml_outline`, `pdf_outline`'s per-page line check -- is unmoved by
construction.
**What it says.** `ExtractedDocument.soft_hyphens` carries the count out,
`InboxResult.normalised` carries it per document, and `account_run` reads it off
the RUN rather than recounting the source -- a second count would be a second
reader. It reaches the accounting JSON as `normalised_soft_hyphen` at both
levels and `log.md` as a `**Normalisation**` bullet.
**What it does not touch.** The guard. The other four characters -- they carry
no typographic job in running text, so removing one would be a decision about
what the guard screens for, taken in the wrong repository. U+00A0 NBSP, which
is not in the guard's set at all (6 633 of them in R761).
**Measured, on R761's own delivery, through the DEFAULT gate:**
```
exit 0, wall 25.2 s
run normalised_soft_hyphen: 71
refused: 0 unaccounted: 0 double_booked: 0
doc 14ce59dc-....xml status persisted normalised_soft_hyphen 71
LOG: * **Accounting**: 1 document(s) and 109 other file(s); 31096 elements
found in the sources; 0 unaccounted, 0 double-booked; 0 of 1 document(s)
refused whole.
LOG: * **Normalisation**: 71 soft hyphen(s) (U+00AD) removed from 1 of 1
document(s) before the persist gate. No other character is touched.
```
**Reach, measured rather than argued.** 0 of the **78** readable documents of
the K2 reference corpus carry U+00AD or any of the five other characters (the 8
unreadable ones raise before extraction, so the denominator is 78 of 86); 0 in
the pinned K2 bundle's concept bodies; 0 across `tests/fixtures`, `examples`,
`skills`, `docs`, `src`, README and CHANGELOG. The door cannot have moved a byte
anyone here has measured.
**Known-negative, in the suite.** A source carrying U+200B is still
`fail_secure`, still exits 1, and the character is still in the extracted text.
Text preservation is stated as an EXACT invariant and never a share: the
extracted text of the hyphenated source EQUALS the extracted text of the same
source written without the hyphens.
## 3. H1 -- a document refused whole is never clean
A refused document books every element as a coded rejection, so `u = 0` and
`d = 0` and the unit balances by construction; `refused_whole` asks its question
only for a corpus that persisted NOTHING. PM measured the ordinary case: one
refused source beside one accepted one gave `clean = 4 of 4`,
`refused_whole = None`, `okf build` exit 0 and three elements gone unseen.
`Unit.refused` is a fifth column and the only one that is not a defect in the
report. Row 3's reason carries `N element(s) lost with R of D document(s)
refused whole`, and each unclean unit's detail line carries `refused=` with the
document's own rejection code in the note.
On the build side, `Accounting.refused` goes into the JSON and into `log.md`.
**The exit code does not move**: it belongs to the whole run, and a corpus
holding one unreadable file among many is ordinary. The order allowed either,
and this is the half that does not break every build that passes today.
`test_a_corpus_refused_whole_under_the_default_gate_is_red` kept its point and
lost its premise: it now asserts `u = 0` and `d = 0` rather than a clean unit.
## 4. H3 -- the role map reads the publisher's own tags
Two independent causes, one line each.
- `count_sts_xml` takes every tag through `_local`, which strips a
Clark-notation namespace AND a prefix. `count_sts_json` compared the raw
string, so `mml:math` reached `tag == "math"` on nothing: **74** formulas in
N200, 2 811 characters, counted as 0.
- The publisher's JSON writes a figure's caption as `figcaption` under the
`graphic`, not as the `fig/caption` NISO-STS writes: **49** in N200, counted
as 0.
The `figcaption` clause is in the JSON map ALONE. The two maps are written twice
on purpose so row 5 can see a hole in either, and the XML delivery has never
been measured carrying one.
**No other count moves**, measured role by role over the eight JSON deliveries
on this machine and the three committed twins:
| delivery | before -> after |
|---|---|
| N100-2022 | `figure_caption` 0 -> 123 |
| N100-2023 | `figure_caption` 0 -> 127 |
| N101-2025 | `figure_caption` 0 -> 62 |
| N200-2024 | `figure_caption` 0 -> 49, `math` 0 -> 74 |
| N303-2026 | `figure_caption` 0 -> 43, `math` 0 -> 3 |
| N500-2024 | `figure_caption` 0 -> 27, `math` 0 -> 88 |
| N601-2022 | no change |
| R761-2025 | no change (row 5's pair) |
| the two existing twins | no change |
The known-negative for stripping a prefix is N101's **574 `tbx:`** tags: not one
of their local names is a role here, and it is in the suite.
## 5. H2, H4, H5, H6
**H2.** PM's X2 mutant -- delete the refusal of a report that declares a
document `rejected` while the bundle holds a concept from it -- is now in
`tools/okf_gate_mutants.py`. Reproduced before the test: **98 passed, rc 0**,
the pattern applied exactly once. The test drives it from both sides, so a gate
refusing EVERY rejected document would not pass either.
**H4.** `verdict(survived, errors)`: 2 when a mutant could not be applied (it
was never measured, and that outranks everything), 1 when one survived, 0
otherwise. The run ended `2 if errors else 0`, so `killed 0 of 1` was an exit 0.
**The harness caught this round's own change.** Its first run reported
`killed 34 of 35` and `ERROR: M21 ... pattern occurs 0 times -- NOT MEASURED`:
H1 had rewritten the `clean` property M21 mutates. M21 is repaired, and X3 --
"a document refused whole is clean again" -- is added beside it, because M21
removes `unverified`, `invalid` AND `refused` at once and would be killed by
any one of the three. Final run: **killed 36 of 36, 0 survived, 0 errors, exit
0**.
**H5.** The skipped-row guard measures `real_corpora(R761_DEFAULT,
N200_DEFAULT)`. Row 6 is SKIPPED exactly when the corpora the ARGUMENTS name are
absent, so asking the arguments made the branch unreachable. Two tests, so a
guard that fires on everything fails too: with the default source present and
the arguments pointing elsewhere the run exits 1 and names the row; with no
corpus on the machine it exits 0.
**H6.** `measures_no_class(units)` returns a sentence when EVERY document of a
corpus has no declared fate, and row 6 prints it beside that corpus's tally.
Reproduced 2026-09-19: `okf build` on N200 proposes 0 plans, prints `FAILED - no
segmentation plans` and exits 2 BEFORE the accounting door -- **no accounting
file is written at all**. What it would take is a capability and not a
threshold: `.json` is read as generic JSON, and the publisher's STS delivery in
that form would have to reach the markdown grammar `_extract_xml` writes for the
XML one. `standardContent` occurs **0** times in `src/` and 4 in the witness,
with the query validated against that known positive.
## 6. The gate, after
`python3 tools/okf_accounting_gate.py` -> **exit 1, `GATE RED: rows 2, 3, 6`**.
| row | k of M | status | moved this round |
|---|---|---|---|
| 1 file types with a fasit fixture | 13 of 13 | GREEN | no |
| 2 source inventory before build | 8 of 13 | RED | no |
| 3 accounting after build | 17 of 22 | RED | no |
| 4 a rejected document is reported honestly | 1 of 1 | GREEN | no |
| 5 two witnesses agree | 4 of 4 | GREEN | no |
| 6 real corpora | 110 of 111 | RED | **the reason did** |
Rows 2 and 3 are the reference corpus's own losses, which the operator decided
on 2026-09-17 stay red until they are fixed.
Row 6 keeps its count and changes its cause. **R761 now passes the DEFAULT
gate**: `exit 0, 1 of 1 document(s) persisted, 50 asset file(s); 31 081 carried
element(s) found in the bundle, 0 claimed and not found`, `u = 0, d = 0` under
both gates. The only `refused whole` left is N200, and that is the build exiting
2 on 0 plans -- H6's blank, now stated in the row.
## 7. Limits of this round
- **Rows 2 and 3 are untouched.** Nothing here fixes a loss in the reference
corpus; the round closes the instrument's holes and one refusal.
- **The 43-document reference corpus was not rebuilt.** The exposure argument
for the normalisation door is a census over the corpus's extracted text (0 of
78) and over the pinned bundle's bytes (0), not a rebuild.
- **Red first is per commit and per hole, not per assertion.** H2's red is the
mutant surviving, not a failing test: the check it protects already worked,
and a test for working code cannot be red.
- **`test_r761s_own_seventy_one_soft_hyphens...` skips** where the corpus is
absent. The committed twin fixture is what holds the rule elsewhere.
- **The witness is still a second implementation of the same definitions.**
A definition wrong for a format is wrong on both sides at once, and the gate
says so in its own output.

14
tools/okf_gate_mutants.py
View file

@ -179,8 +179,18 @@ MUTANTS: tuple[Mutant, ...] = (
Mutant(
"M21 an unverified booking is clean",
GATE,
"return not (self.unaccounted or self.double or self.unverified or self.invalid)",
"return not (self.unaccounted or self.double)",
" self.unaccounted or self.double or self.unverified "
"or self.invalid or self.refused",
" self.unaccounted or self.double",
),
# H1's own column, mutated on its own: M21 above removes `unverified`,
# `invalid` AND `refused` at once, so it would be killed by any one of the
# three. This one takes only the fifth.
Mutant(
"X3 a document refused whole is clean again",
GATE,
' if status == "rejected" and not persisted and total > 0:',
" if False:",
),
Mutant(
"M22 any asset file at all proves a carry",