feat(accounting): okf build accounts for every source element

okf build --accounting PATH inventories every source document before
extraction, in the gate's per-format vocabulary, and after the run gives
each element one fate (carried / pointer / coded rejection), written as
JSON and summarised in log.md. "carried" is checked against the written
concept bodies, so a gate that drops a line is found (test). Exit 1 on
anything unaccounted or double-booked. Opt-in: +744 s (+19 %) on the
43-document reference corpus, and that corpus fails the check on 24 real
losses (22 images on text-less PDF pages, 2 docx Title paragraphs).

Changed without the flag:
- okf build exits 1 when it extracted documents and persisted none.
  Door B and corpus.measure are unchanged. One test relied on exit 0.
- An image file carried through a persisted document is its own K1b
  column, no longer also extractor_unknown. The set is what the resolver
  actually carried (ExtractedDocument.files), never a byte match.

tools/okf_accounting_gate.py (checks untouched) is green on all six rows,
R761 110 of 110 under both gates.

Report: docs/2026-09-17-innholdsregnskapet-bygget.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-17 18:35:20 +02:00
commit 864570b320
13 changed files with 1751 additions and 59 deletions

View file

@ -309,6 +309,36 @@ this package chose, and it would silently drop somebody's small table.
<!-- cli-default-assets: on -->
`--accounting PATH` accounts for the CONTENT, not only the files. Before
extraction, every source document is inventoried in a per-format element
vocabulary: headings, paragraphs, tables, cells, images, pages, and so on.
After the run, each element gets exactly one fate:
- **carried:** all of its text is in the concepts written for the document,
or its image is in `assets/`;
- **pointer:** a remote image, which is never fetched, or a markdown image
reference kept verbatim;
- **a coded rejection:** the gate's or the reader's code.
The result goes to PATH as JSON and into `log.md`. The build exits 1 when an
element has no fate or has two. A document the gate refuses is logged as
`<file>: <M> elements found in the source, 0 carried: document rejected
`<code>``, and the `Images` bullet then counts what the sources declare.
"Carried" means the text is present, not that it is in the right place. A
short element such as a section label can be found elsewhere in the same
document. The judge is `tools/okf_accounting_gate.py`, which compares the
inventory against an independent witness.
Two things hold with or without the flag:
- `okf build` exits 1 when it extracted at least one document and persisted
none.
- An image file beside a document is counted once. If a persisted document
carried it, it is in the conservation identity's own column
(`merged + files carried through a document + coded rejections = N`);
otherwise it is a coded rejection.
`--frontmatter KEY=VALUE` (repeatable) stamps a key on every concept of the
run, for what the operator knows and the document does not say — an edition,
a publisher's address. It splits on the first `=` and writes the value