open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(accounting): content-accounting gate for okf build, written red

Browse source 
Capability loop step 3, no capability. tools/okf_accounting_gate.py asks,
per README file type, how many of the elements a SOURCE holds okf build
books as carried / pointer / coded rejection, with unaccounted and
double-booked both required to be 0. Exit 1 today on rows 2, 3, 4 and 6.

The fasit is tools/okf_witness.py (stdlib + pdfplumber + poppler, no
package import; tested on the live import graph), committed as
tests/fixtures/accounting/*inventory.json over one fixture per type.

Measured: no source inventory (0 of 13); two graphics/ files carried
through documents AND counted extractor_unknown (50 on R761 under
--gate none); a refused document logged "0 carried of 0 found"; R761
refused whole because guard 1.4.0 treats its 71 U+00AD soft hyphens as an
invisible carrier (asked of the security repo). The two R761 witnesses
agree once STS labels are counted by role, not tag.

Report: docs/2026-09-17-innholdsregnskapet-rod-gate.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-17 15:40:37 +02:00
commit 0b00de4408
26 changed files with 2598 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

28
tests/fixtures/README.md vendored
View file

@ -175,6 +175,34 @@ The version range that carries this lives in `pyproject.toml`'s
`[project.optional-dependencies] extract`, with the same reasoning at the
declaration site.
## The content-accounting fixtures (`accounting/`)
The fasit side of `tools/okf_accounting_gate.py`. `accounting/corpus/` holds
one document per row of README's file-type table (13 of 13) plus a `graphics/`
directory next to them that the HTML, STS and markdown documents point at --
the layout under which a picture is carried through a document AND booked as a
rejected file. `accounting/rejected/` holds one HTML document with a
zero-width space in its prose, which the guard refuses at every tier, and the
image it points at.
`inventory.json` and `rejected-inventory.json` are what `tools/okf_witness.py`
counts in those two directories, committed as data and regenerated only with
that tool:
```
python3 tools/okf_witness.py tests/fixtures/accounting/corpus > tests/fixtures/accounting/inventory.json
python3 tools/okf_witness.py tests/fixtures/accounting/rejected > tests/fixtures/accounting/rejected-inventory.json
```
`witness/prosess-84-sts.twin.json` is the STS document written by hand in the
publisher's JSON node form (`standardContent`, nodes with `e`/`t`/`x`), so the
two STS witnesses can be compared on a fixture as well as on R761. Eight of
the thirteen documents are byte copies of fixtures documented above
(`image-inbox/`, `k2-office/`, `prisark.xlsx`); the other five
(`notat.md`, `logg.txt`, `mengder.csv`, `parametre.json`, `side.htm`) are
written here, and `notat.md` carries a fenced `# ...` line that is not a
heading.
## What these fixtures do not cover
Structured table recovery. Measured on real Vegnormalene, only 45 of 196