Kjell Tore Guttormsen
23ef6bbef3
A public repository should not carry the layout of the machine it was written
on, and it should not carry the identifiers of a queue nobody outside can see.
Six places in material this span added or changed did.
Neutral text, same facts: a private queue path in the EOL report, a private
note path in two docstrings and one report header, and four order identifiers
opening four reports. Where a sentence lost its source it says `internal
measurement note` -- the measurement is still named, the path is not.
One of them was output, not prose: row 9 of the retrieval gate printed a
`~/corpora/...` path to stdout on every run. It says the bundles exist on the
machine the row was written against; the row already takes `--k2 SET SHA
BUNDLE` from anyone who holds them.
Measured with the patterns as given, and both validated against a
known-positive on `df5a118` first -- an unvalidated pattern reporting zero is
not a measurement:
* `~/\.claude|/Users/|coord/.*/orders` over the 74 files the span touches:
**9 -> 6**. The six left are deliberate: two README lines telling a reader to
install a skill into their OWN `~/.claude/skills`, one prose line arguing
that `/Users/...` is not more portable than a relative path, and three lines
of `tests/test_retrieval_gate.py`'s own known-positive for this exact check,
which removing would disarm it.
* `[0-9]{8}T[0-9]{6}Z-[0-9]+` over the 38 files the span ADDED: **4 -> 0**.
A note on the method, because the first run of the first grep reported 0 and
was wrong: zsh does not word-split an unquoted parameter, so
`git grep -- $FILES` passed 74 names as one argument and matched no file at
all. Both numbers above come from `$(git diff --name-only ...)`, which zsh does
split, and the known-positive is what made the false zero visible.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
215 lines
11 KiB
Markdown
215 lines
11 KiB
Markdown
# The content-accounting gate, hardened: the judge opens the bundle itself
|
|
|
|
2026-09-18. Raised by an independent adversarial review of `0b00de4`,
|
|
commissioned after the accounting was built. The review's
|
|
verdict was that the gate is "an honest RED today, but cannot be trusted as a
|
|
judge once it turns green": it was a calculator over a report the judged party
|
|
writes. This is what changed, measured.
|
|
|
|
Nothing in `src/` moved. The instrument was hardened; the thing it judges was
|
|
not touched.
|
|
|
|
## 1. BLOCKER B-1 — the gate never opened a concept file
|
|
|
|
`okf_accounting_gate.account()` compared BOOKED NUMBERS with the witness's
|
|
counts. Reproduced here before any code moved, as two tests over the real
|
|
fixture bundle: a report that changes not one byte of the bundle and books
|
|
every element as `carried` gave `GATE GREEN`, exit 0 — and so did booking
|
|
every element as `rejected`.
|
|
|
|
**The witness now gives every element the PIECES OF TEXT IT IS MADE OF, and
|
|
the gate looks for each of them in the concept bodies the run wrote.** Pieces
|
|
rather than one joined string, and that distinction is load-bearing: a reader
|
|
writes a heading's ATX marker and a picture's pointer block between the parts
|
|
of a container, so an STS `<sec>` is never one contiguous run in the bundle
|
|
even when every word of it is there. Measured — with one joined string per
|
|
element, 2 of the 13 fixture documents reported false losses; with pieces, 0.
|
|
|
|
The normalisation is the one the build already uses and no looser (strip
|
|
non-alphanumerics, casefold, remove a converter attribute block such as
|
|
`{.mark}`), and it is implemented in the gate rather than imported from
|
|
`accounting.py`: a judge borrowing the judged party's own comparison agrees
|
|
with it by construction.
|
|
|
|
Five more refusals, each driven from both sides by a test:
|
|
|
|
- a negative booking (`carried: 25, rejected: {x: -15}` of 10 read as clean);
|
|
- a document declared `persisted` that no concept in the bundle names;
|
|
- a document declared `rejected` that the bundle holds a concept from;
|
|
- a rejection code outside a closed list, at the document AND at the element;
|
|
- an `accounting_version` the gate does not read.
|
|
|
|
And: **a document the build PERSISTED whose report carries nothing from it is
|
|
never clean.** "Everything rejected" satisfied `u = 0, d = 0`.
|
|
|
|
**m-1:** an asset proved a carry by NAME, so a zero-byte file called
|
|
`<sha12>-x.png` read as a carry. It is now checked on the BYTES and on the
|
|
content address. The first attempt re-implemented the build's full naming rule
|
|
and reported **50 of 50** carried R761 images as missing — the build
|
|
lowercases the basename, folds separator runs and takes the suffix from the
|
|
bytes. The readable tail is the build's cosmetic choice and is deliberately
|
|
not reproduced here.
|
|
|
|
## 2. MAJOR M-1 and M-2 — what the witnesses could not see
|
|
|
|
Rows 2 and 3 require the build's inventory to EQUAL the witness's, so what the
|
|
witness does not count, nothing can lose visibly. Thirteen classes were added,
|
|
each with a red test first:
|
|
|
|
| format | now counted |
|
|
|---|---|
|
|
| `.docx` | header/footer, comment, endnote, text box |
|
|
| `.pptx` | speaker note, hidden slide (`show="0"`, no longer an ordinary slide) |
|
|
| `.xlsx` | formula, hidden sheet |
|
|
| `.odt` | header/footer (they live in `styles.xml`), annotation |
|
|
| `.xml` | `mixed-citation`, `mml:math`, `fig`, a figure's caption |
|
|
| `.pdf` | the page's own TEXT (it counted pages and picture placements only) |
|
|
|
|
A text box's paragraphs are the box's own, or the text is booked twice. A
|
|
hidden sheet's state lives in `workbook.xml` and is reached through the
|
|
relationship id, so the sheet part itself says nothing about it. An annotation
|
|
counted as prose makes the accounting demand that a reader carry a note the
|
|
author wrote to themselves.
|
|
|
|
**M-2:** the two STS witnesses shared ONE role map, so row 5 — "two witnesses
|
|
agree" — could not see a hole in it. They are written apart now. Measured on
|
|
R761 2026-09-18: the XML delivery places a section's label at `sec/label`
|
|
**7 714** times and inside a title **0** times; the JSON delivery puts
|
|
**2 760** of them inside the title. The XML map was narrowed to the XML fact,
|
|
and a JSON twin fixture carrying the nested placement holds the difference
|
|
live — without it, swapping the maps changed nothing measurable.
|
|
|
|
## 3. MAJOR M-3 — 20 of 63 element types had a count of zero
|
|
|
|
A witness cannot be caught being wrong about something it never sees; that is
|
|
why six of seven witness mutants survived the original suite. Seven hand-built
|
|
documents close it (`tests/fixtures/accounting/make_accounting_fixtures.py`),
|
|
every element type now occurs at least once, and **all twenty documents carry
|
|
a hand count** read off the fixture's own bytes — four did before. The
|
|
operator's own proposed `.xlsx image` exception could not be exercised at all
|
|
until now.
|
|
|
|
Building them found three defects in the fixtures themselves, each of which
|
|
would have reported a loss the build never had: a shared string table not
|
|
related to the workbook (every `t="s"` cell converts empty), a `graphicData`
|
|
with no `uri` (pandoc does not see a table), and an `odt` `styles.xml` without
|
|
`<office:styles/>` (pandoc refuses the document).
|
|
|
|
## 4. Row 6 and the verdict line
|
|
|
|
- **R761 refused whole under the default gate is now RED.** It was GREEN with
|
|
31 096 of 31 096 elements booked as coded rejections and `okf build` exiting
|
|
1 unseen: every element had a fate, so `u = 0, d = 0`. The row asks the
|
|
question separately now.
|
|
- **A skipped row never leaves the verdict unqualified.** `GATE GREEN` now
|
|
reads `GATE GREEN (row 6 not run: <reason>)`, and the exit code is non-zero
|
|
locally when a corpus source exists on the machine and the row did not run.
|
|
- **N200 Vegbygging:2024 is a second real corpus.** R761 holds **0** `fig`,
|
|
**0** formulas and **0** references, so the gate's only real corpus could not
|
|
have found the hole in the role map. N200 carries **194** citations, **49**
|
|
figures and **135** footnotes. A `.json` file whose root carries an STS node
|
|
tree is counted as STS rather than as keys and leaves — the container is not
|
|
the content.
|
|
|
|
## 5. MAJOR M-4 — mutation
|
|
|
|
The review's 26 mutants, ported to the code as it stands, plus 8 written for
|
|
the new checks: **34 of 34 killed** (21 of the review's 26 were killed before
|
|
the last seven tests were added; all 26 are killed now, against the order's bar
|
|
of 23). The harness is `tools/okf_gate_mutants.py`, it runs on a copy of the
|
|
tree, and **a pattern that does not match is an ERROR and exit 2** — a mutant
|
|
that could not be applied was never measured, and counting it as killed is the
|
|
same mistake as reading an empty search as an absence. That fired once, on
|
|
M13, after a refactor moved the line it edits.
|
|
|
|
## 6. The gate is REDDER than before, and that is the movement
|
|
|
|
At the shipped defaults, `python3 tools/okf_accounting_gate.py`, 3 m 51 s:
|
|
|
|
```
|
|
1 file types with a fasit fixture | 13 of 13 | GREEN
|
|
2 source inventory before build | 8 of 13 | RED
|
|
3 accounting after build | 17 of 22 | RED
|
|
4 a rejected document is reported | 1 of 1 | GREEN
|
|
5 two witnesses agree | 4 of 4 | GREEN
|
|
6 real corpora | 110 of 111| RED
|
|
GATE RED: rows 2, 3, 6
|
|
```
|
|
|
|
Row 3: **u = 19 unaccounted, d = 2 double-booked, 0 booked carried and not in
|
|
the bundle, 0 declarations the gate refuses** — and **314 carried elements
|
|
FOUND in the bundle**, 6 carrying no text the gate can check. The zero matters
|
|
as much as the nineteen: nothing the build DOES book as carried failed the
|
|
bundle check, so every red is the build's and none is the instrument's.
|
|
|
|
**The new "not carried" classes, per file type, with counts on the fixtures** —
|
|
the raw material for the next capability order and for the operator's
|
|
exception list:
|
|
|
|
| file type | not carried | on the fixtures |
|
|
|---|---|---|
|
|
| `.docx` | comment, endnote, footnote, header/footer, text box | 1, 1, 1, 2, 1 |
|
|
| `.odt` | annotation, header/footer, image | 1, 2, 1 |
|
|
| `.pptx` | hidden slide, speaker note | 1, 1 |
|
|
| `.xlsx` | formula, hidden sheet, image | 1, 1, 1 |
|
|
| `.xml` (STS) | citation, figure, figure caption, math | 1, 1, 1, 1 |
|
|
|
|
Two are DOUBLE-booked rather than lost: a hidden slide and a hidden sheet are
|
|
carried into the bundle but booked as an ordinary slide and an ordinary sheet.
|
|
A third is a genuine double count in this package: **a text box's paragraph is
|
|
counted twice**, once inside the box and once in the paragraph carrying it.
|
|
|
|
On the real corpora: R761 under `--gate none` is **31 081 carried elements
|
|
found in the bundle, 0 claimed and not found, 65 the gate cannot check**, over
|
|
a document holding 31 096. N200 is **16 549 elements with no fate at all**,
|
|
because `okf build` exits 2 on it (see § 7).
|
|
|
|
`okf build --accounting` over the fixture corpus now exits 1 with **6
|
|
unaccounted** in the build's OWN vocabulary — its first real finding, and it
|
|
only became reachable when the fixtures carried the constructs.
|
|
|
|
## 7. Two facts about `okf build` this run establishes
|
|
|
|
**R761 is felled 100 % by ONE rule, and it is not tier-dependent.** The
|
|
document is refused whole under the default `guard-trusted-source` with
|
|
`fail_secure`. Asked directly, guard 1.4.0 answers
|
|
`any-tier: invisible carrier (output:zero-width-present)`, severity HIGH,
|
|
assessment SEVERE — and `PRESET_USER_UPLOAD` gives the same disposition, so no
|
|
tier this package offers persists it. The cause is measured rather than
|
|
inferred: the document carries **71 U+00AD soft hyphens**, and removing them
|
|
takes the same text to `WARN`, which persists. Whether that is intended is
|
|
below.
|
|
|
|
**N200 gets 0 plans and exit 2.** The JSON delivery is fenced verbatim, so no
|
|
line-anchored boundary grammar fires — the same behaviour a structureless
|
|
`.txt` produces, documented in `CLAUDE.md`. It is not an accounting defect, and
|
|
the gate reports it as what it is: 16 549 elements with no fate.
|
|
|
|
## 8. What this gate still cannot check
|
|
|
|
Printed on every run, beside the verdict, because a gate that reports only the
|
|
build's gaps invites a green row to be read as a guarantee:
|
|
|
|
- a short element often stands elsewhere in the same document, so finding it
|
|
proves it is present and not that THIS one is;
|
|
- two pointed files with identical bytes are one content-addressed asset, so
|
|
one of them losing its pointer is invisible here (m-5);
|
|
- an image embedded in a binary container has no source file to hash;
|
|
- absence is never verified: an element booked REJECTED is not looked for;
|
|
- the witness is a second implementation of the same definitions, so a
|
|
definition that is wrong for a format is wrong on both sides at once.
|
|
|
|
Every witness also prints, per file type, the named list of what it still does
|
|
not count. `APPROVED_EXCEPTIONS` was read by no row at all (m-3); each pair is
|
|
now checked against the witness's vocabulary and the run says, per pair, why it
|
|
moves no denominator — and would say `WARNING` if one ever did.
|
|
|
|
## 9. Reproduction
|
|
|
|
```
|
|
uv run python tools/okf_accounting_gate.py # 3 m 51 s, exit 1
|
|
uv run python tools/okf_gate_mutants.py # 34 of 34 killed, exit 0
|
|
uv run pytest tests/test_accounting_gate.py tests/test_content_accounting.py -q
|
|
uv run python tools/okf_witness.py tests/fixtures/accounting/corpus
|
|
python3 tests/fixtures/accounting/make_accounting_fixtures.py
|
|
```
|