docs(front-page): the gate numbers the gate actually prints, and a breaking point that was measured

Four claims on the front page were false on this commit, and one of them was a
number no division ever produced.

**The retrieval gate.** README reported it RED on rows 3, 4, 5, 7, 8 and 9,
with row 3 at 2 of 5 and row 4 at 3 of 6. Run on this commit it is RED on rows
5, 7, 8 and 9, with row 3 at 5 of 5 and row 4 at 6 of 6: `f81683e` made a
withheld concept carry the rule that actually decided it, and `05cb190` gave
the payload a `coverage` block, and neither updated the table. Row 8 is `0 of 3
| NOT RUN` on the default run and was published as `44 of 64 questions`, which
is what it scores the day all three private sets are handed to it -- now
labelled with the day and the machine rather than printed as a row. The same
four figures were stale in `CLAUDE.md`.

**The breaking point in a generated skill.** `int(LIMIT / per_withheld) if
per_withheld else 0` printed `At roughly 0 concepts the bookkeeping alone
reaches the 120000-byte limit` whenever the generation run withheld nothing --
the absence of a measurement, rendered as one, and read as a bundle that breaks
before it holds anything. A run with no withheld entry has no slope to
extrapolate from, so the sentence is withheld with its reason. The shipped
`skills/okf-consume/SKILL.md` is generated with the question its
`references/README.md` names, withholds nothing, and carried exactly that `0`;
it is regenerated. Two arms in the test, because one would pass on an empty
set: the bundles that withhold something must still state a positive figure.

The sentence for that arm also stopped saying `**4 bytes** for 3 concepts`
where the 4 bytes were the cost of 0 withheld entries. It is now `for N of M
concepts`, which moves two generated skills' line counts and therefore the
published comparison: 280 of 312 and 310 -> 281 of 313 and 311, re-measured,
with the 62 differing lines unchanged.

**Four tools.** A single-bundle server exposes three: `okf_list` is absent
where there is nothing to list. README's table already said so in a cell; the
heading and the CHANGELOG did not.

**What `--accounting` accounts for.** The account is over the element classes
each format's vocabulary names, verified against `accounting._READERS` rather
than against the report: a file whose suffix has no reader is accounted at file
level only, `.docx` reads `document.xml` and `footnotes.xml` (so headers,
footers, endnotes and comments are outside), `.pptx` reads the slides (so
speaker notes are outside), `.xlsx` reads the worksheets (so cell comments are
outside and a cell contributes its cached value, never its formula), and `.rtf`
skips its header and footer groups. A hidden slide or sheet IS counted -- it
lives in the same part as a visible one. Nothing is built for this; the list is
what `0 unaccounted` does not claim.

Gates re-run on the commit: retrieval `GATE RED: rows 5, 7, 8, 9` (exit 1),
MCP `GATE RED: rows 2` (exit 1), both matching what is now written.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-20 15:40:20 +02:00
commit d300338e4d
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
9 changed files with 203 additions and 754 deletions

View file

@ -464,6 +464,27 @@ short element such as a section label can be found elsewhere in the same
document. The judge is `tools/okf_accounting_gate.py`, which compares the
inventory against an independent witness.
**The account covers the element classes the vocabulary knows, and no others.**
`accounting._READERS` names twelve suffixes, each with its own tuple of classes
(`.md`: heading, paragraph, table, table_row, image, code_block; `.pdf`: image
and page only, which is the approved exception below). Two consequences are
stated here rather than left to be discovered, because "0 unaccounted" reads
like a statement about the document and is a statement about those classes:
- **A file whose suffix has no reader is accounted at FILE level only**
carried, merged or rejected — never element by element.
- **Parts of a document that no vocabulary names are not counted, so content
there can go missing under exit 0 and `0 unaccounted`.** Verified against the
readers: `.docx` reads `word/document.xml` and `word/footnotes.xml`, so
headers, footers, endnotes and comments are outside; `.pptx` reads
`ppt/slides/slideN.xml`, so speaker notes, masters and layouts are outside;
`.xlsx` reads the worksheets, the shared strings and the drawings, so cell
comments are outside and a cell contributes its cached value or inline
string, never its formula; `.rtf` skips the `header`, `footer`, `info`,
`pict`, `stylesheet`, `fonttbl` and `colortbl` groups. A hidden slide or
sheet IS counted — it lives in the same part as a visible one. Nothing here
is built for now: the list is what the account does not claim.
**Two operator decisions, 2026-09-17.** The accounting stays OPT-IN until the
losses it reports on the reference corpus are fixed, because a default-on door
would fail builds that pass today. And of the three exceptions the gate
@ -1018,30 +1039,33 @@ set names a consumer's documents and this repository is public. **The corpus
is pinned the same way** (`SPECS_SHA256`): every row counts against those
documents, so moving them without moving the pin is exit 2.
**It is RED today, on rows 3, 4, 5, 7, 8 and 9**, and each of those is a
**It is RED today, on rows 5, 7, 8 and 9**, and each of those is a
finding rather than a defect in the gate:
| row | what it asks | today |
|---|---|---|
| 1 | hit@payload, one fasit entry = one unit | 10 of 10 |
| 2 | every miss carries exactly one class, each forced by its own fixture | 7 of 7 |
| 3 | the `rule` the payload prints for a withheld fasit is the true one | 2 of 5 |
| 4 | an uncovered question comes back marked, a covered one does not | 3 of 6 |
| 3 | the `rule` the payload prints for a withheld fasit is the true one | 5 of 5 |
| 4 | an uncovered question comes back marked, a covered one does not | 6 of 6 |
| 5 | a hold-out set, frozen and with its threshold written first | 0 of 1 |
| 6 | every delivery confirmed against the bundle's own bytes | 10 of 10 |
| 7 | mechanical mutants of the ranking and the cut, felled | 12 of 14 |
| 8 | the three real sets, from path + sha256 | 44 of 64 questions |
| 8 | the three real sets, from path + sha256 | 0 of 3 sets, NOT RUN without `--real` |
| 9 | K2 | 0 of 6, no gold set exists |
Row 3 is the one to read first: in a bundle built from ONE source document,
every concept past the first two carries that document's `source_file`, so a
concept the RANK had already lost is withheld as `source_quota_exceeded`. The
gate decides the truth with the same cut run without the quota, and a
consumer reading `rule` today gets the wrong reason. Row 4 is the second: the
payload has no key a consumer can read as "this bundle does not answer that",
so an uncovered question comes back with excerpts and no statement — the only
honest case today is the one where nothing matched at all and nothing was
delivered.
Rows 3 and 4 were this gate's two findings and both are closed, which is what
a gate written before the capability is for. Row 3: in a bundle built from ONE
source document, every concept past the first two carries that document's
`source_file`, so a concept the RANK had already lost came back withheld as
`source_quota_exceeded`. A drop now keeps the rule the same cut without the
quota would have given it, and only a candidate that cut would have delivered
is named as the quota's — 2 of 5 to 5 of 5. Row 4: the payload had no key a
consumer could read as "this bundle does not answer that", so an uncovered
question came back with excerpts and no statement. `coverage` states the terms
the pre-pass read, the terms no concept in the bundle answers and the terms no
delivered excerpt answers — facts, no verdict, and the one bar is the gate's
own `UNANSWERED_BAR = 2/3` — 3 of 6 to 6 of 6.
Row 7 reports two survivors with what they moved rather than with a shrug:
killing the document prior and flattening the fusion (`RRF_K`) each moved
@ -1064,10 +1088,13 @@ Granularity is stated on every line and the two forms are never summed: a set
naming a citation is measured at citation granularity, a set naming only a
section is measured at concept granularity. **Row 8's own headline is
therefore at QUESTION granularity** — the one unit all three sets share —
with the two unit totals printed below it, each with its own denominator.
Measured 2026-09-19 against the three real sets on one machine: **44 of 64
questions**, and below it *7 of 29 at citation granularity, 38 of 50 at
concept granularity*.
with the two unit totals printed below it, each with its own denominator. The
table above reports the gate's DEFAULT run, where row 8 is `0 of 3` and NOT
RUN because the sets are not here; the last run that was given all three, on
one machine 2026-09-19, scored **44 of 64 questions**, and below it *7 of 29
at citation granularity, 38 of 50 at concept granularity*. That figure is not
reproducible from this repository alone, which is why it is labelled with the
day and the machine rather than printed as a row.
## Consume in Claude Code
@ -1172,7 +1199,9 @@ by name**: it discovers them per call, so a bundle you add, remove or rebuild
while the server is running is picked up by the next call. No restart, no
configuration edit, no code change.
Four tools, and each one's description says why it exists:
Four tools on a multi-bundle server and **three** on a single-bundle one —
`okf_list` is absent where there is nothing to list — and each one's description
says why it exists:
| tool | what it answers |
|---|---|
@ -1225,8 +1254,8 @@ reader to run `okf card <bundle>` first. The card is **derived on every run and
never written into the bundle**, so there is no second artefact that can
disagree with the bytes beside it.
Measured on two unrelated bundles: two per-bundle skills are identical on 280
of 312 and 310 lines. The 62 lines that differ are exactly identity, concept
Measured on two unrelated bundles: two per-bundle skills are identical on 281
of 313 and 311 lines. The 62 lines that differ are exactly identity, concept
count, the conditional-field table, the whole-bundle cost and the breaking
point — the five things a rebuild invalidates.