docs(front-page): the gate numbers the gate actually prints, and a breaking point that was measured
Four claims on the front page were false on this commit, and one of them was a number no division ever produced. **The retrieval gate.** README reported it RED on rows 3, 4, 5, 7, 8 and 9, with row 3 at 2 of 5 and row 4 at 3 of 6. Run on this commit it is RED on rows 5, 7, 8 and 9, with row 3 at 5 of 5 and row 4 at 6 of 6: `f81683e` made a withheld concept carry the rule that actually decided it, and `05cb190` gave the payload a `coverage` block, and neither updated the table. Row 8 is `0 of 3 | NOT RUN` on the default run and was published as `44 of 64 questions`, which is what it scores the day all three private sets are handed to it -- now labelled with the day and the machine rather than printed as a row. The same four figures were stale in `CLAUDE.md`. **The breaking point in a generated skill.** `int(LIMIT / per_withheld) if per_withheld else 0` printed `At roughly 0 concepts the bookkeeping alone reaches the 120000-byte limit` whenever the generation run withheld nothing -- the absence of a measurement, rendered as one, and read as a bundle that breaks before it holds anything. A run with no withheld entry has no slope to extrapolate from, so the sentence is withheld with its reason. The shipped `skills/okf-consume/SKILL.md` is generated with the question its `references/README.md` names, withholds nothing, and carried exactly that `0`; it is regenerated. Two arms in the test, because one would pass on an empty set: the bundles that withhold something must still state a positive figure. The sentence for that arm also stopped saying `**4 bytes** for 3 concepts` where the 4 bytes were the cost of 0 withheld entries. It is now `for N of M concepts`, which moves two generated skills' line counts and therefore the published comparison: 280 of 312 and 310 -> 281 of 313 and 311, re-measured, with the 62 differing lines unchanged. **Four tools.** A single-bundle server exposes three: `okf_list` is absent where there is nothing to list. README's table already said so in a cell; the heading and the CHANGELOG did not. **What `--accounting` accounts for.** The account is over the element classes each format's vocabulary names, verified against `accounting._READERS` rather than against the report: a file whose suffix has no reader is accounted at file level only, `.docx` reads `document.xml` and `footnotes.xml` (so headers, footers, endnotes and comments are outside), `.pptx` reads the slides (so speaker notes are outside), `.xlsx` reads the worksheets (so cell comments are outside and a cell contributes its cached value, never its formula), and `.rtf` skips its header and footer groups. A hidden slide or sheet IS counted -- it lives in the same part as a visible one. Nothing is built for this; the list is what `0 unaccounted` does not claim. Gates re-run on the commit: retrieval `GATE RED: rows 5, 7, 8, 9` (exit 1), MCP `GATE RED: rows 2` (exit 1), both matching what is now written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
bf697bfcad
commit
d300338e4d
9 changed files with 203 additions and 754 deletions
69
README.md
69
README.md
|
|
@ -464,6 +464,27 @@ short element such as a section label can be found elsewhere in the same
|
|||
document. The judge is `tools/okf_accounting_gate.py`, which compares the
|
||||
inventory against an independent witness.
|
||||
|
||||
**The account covers the element classes the vocabulary knows, and no others.**
|
||||
`accounting._READERS` names twelve suffixes, each with its own tuple of classes
|
||||
(`.md`: heading, paragraph, table, table_row, image, code_block; `.pdf`: image
|
||||
and page only, which is the approved exception below). Two consequences are
|
||||
stated here rather than left to be discovered, because "0 unaccounted" reads
|
||||
like a statement about the document and is a statement about those classes:
|
||||
|
||||
- **A file whose suffix has no reader is accounted at FILE level only** —
|
||||
carried, merged or rejected — never element by element.
|
||||
- **Parts of a document that no vocabulary names are not counted, so content
|
||||
there can go missing under exit 0 and `0 unaccounted`.** Verified against the
|
||||
readers: `.docx` reads `word/document.xml` and `word/footnotes.xml`, so
|
||||
headers, footers, endnotes and comments are outside; `.pptx` reads
|
||||
`ppt/slides/slideN.xml`, so speaker notes, masters and layouts are outside;
|
||||
`.xlsx` reads the worksheets, the shared strings and the drawings, so cell
|
||||
comments are outside and a cell contributes its cached value or inline
|
||||
string, never its formula; `.rtf` skips the `header`, `footer`, `info`,
|
||||
`pict`, `stylesheet`, `fonttbl` and `colortbl` groups. A hidden slide or
|
||||
sheet IS counted — it lives in the same part as a visible one. Nothing here
|
||||
is built for now: the list is what the account does not claim.
|
||||
|
||||
**Two operator decisions, 2026-09-17.** The accounting stays OPT-IN until the
|
||||
losses it reports on the reference corpus are fixed, because a default-on door
|
||||
would fail builds that pass today. And of the three exceptions the gate
|
||||
|
|
@ -1018,30 +1039,33 @@ set names a consumer's documents and this repository is public. **The corpus
|
|||
is pinned the same way** (`SPECS_SHA256`): every row counts against those
|
||||
documents, so moving them without moving the pin is exit 2.
|
||||
|
||||
**It is RED today, on rows 3, 4, 5, 7, 8 and 9**, and each of those is a
|
||||
**It is RED today, on rows 5, 7, 8 and 9**, and each of those is a
|
||||
finding rather than a defect in the gate:
|
||||
|
||||
| row | what it asks | today |
|
||||
|---|---|---|
|
||||
| 1 | hit@payload, one fasit entry = one unit | 10 of 10 |
|
||||
| 2 | every miss carries exactly one class, each forced by its own fixture | 7 of 7 |
|
||||
| 3 | the `rule` the payload prints for a withheld fasit is the true one | 2 of 5 |
|
||||
| 4 | an uncovered question comes back marked, a covered one does not | 3 of 6 |
|
||||
| 3 | the `rule` the payload prints for a withheld fasit is the true one | 5 of 5 |
|
||||
| 4 | an uncovered question comes back marked, a covered one does not | 6 of 6 |
|
||||
| 5 | a hold-out set, frozen and with its threshold written first | 0 of 1 |
|
||||
| 6 | every delivery confirmed against the bundle's own bytes | 10 of 10 |
|
||||
| 7 | mechanical mutants of the ranking and the cut, felled | 12 of 14 |
|
||||
| 8 | the three real sets, from path + sha256 | 44 of 64 questions |
|
||||
| 8 | the three real sets, from path + sha256 | 0 of 3 sets, NOT RUN without `--real` |
|
||||
| 9 | K2 | 0 of 6, no gold set exists |
|
||||
|
||||
Row 3 is the one to read first: in a bundle built from ONE source document,
|
||||
every concept past the first two carries that document's `source_file`, so a
|
||||
concept the RANK had already lost is withheld as `source_quota_exceeded`. The
|
||||
gate decides the truth with the same cut run without the quota, and a
|
||||
consumer reading `rule` today gets the wrong reason. Row 4 is the second: the
|
||||
payload has no key a consumer can read as "this bundle does not answer that",
|
||||
so an uncovered question comes back with excerpts and no statement — the only
|
||||
honest case today is the one where nothing matched at all and nothing was
|
||||
delivered.
|
||||
Rows 3 and 4 were this gate's two findings and both are closed, which is what
|
||||
a gate written before the capability is for. Row 3: in a bundle built from ONE
|
||||
source document, every concept past the first two carries that document's
|
||||
`source_file`, so a concept the RANK had already lost came back withheld as
|
||||
`source_quota_exceeded`. A drop now keeps the rule the same cut without the
|
||||
quota would have given it, and only a candidate that cut would have delivered
|
||||
is named as the quota's — 2 of 5 to 5 of 5. Row 4: the payload had no key a
|
||||
consumer could read as "this bundle does not answer that", so an uncovered
|
||||
question came back with excerpts and no statement. `coverage` states the terms
|
||||
the pre-pass read, the terms no concept in the bundle answers and the terms no
|
||||
delivered excerpt answers — facts, no verdict, and the one bar is the gate's
|
||||
own `UNANSWERED_BAR = 2/3` — 3 of 6 to 6 of 6.
|
||||
|
||||
Row 7 reports two survivors with what they moved rather than with a shrug:
|
||||
killing the document prior and flattening the fusion (`RRF_K`) each moved
|
||||
|
|
@ -1064,10 +1088,13 @@ Granularity is stated on every line and the two forms are never summed: a set
|
|||
naming a citation is measured at citation granularity, a set naming only a
|
||||
section is measured at concept granularity. **Row 8's own headline is
|
||||
therefore at QUESTION granularity** — the one unit all three sets share —
|
||||
with the two unit totals printed below it, each with its own denominator.
|
||||
Measured 2026-09-19 against the three real sets on one machine: **44 of 64
|
||||
questions**, and below it *7 of 29 at citation granularity, 38 of 50 at
|
||||
concept granularity*.
|
||||
with the two unit totals printed below it, each with its own denominator. The
|
||||
table above reports the gate's DEFAULT run, where row 8 is `0 of 3` and NOT
|
||||
RUN because the sets are not here; the last run that was given all three, on
|
||||
one machine 2026-09-19, scored **44 of 64 questions**, and below it *7 of 29
|
||||
at citation granularity, 38 of 50 at concept granularity*. That figure is not
|
||||
reproducible from this repository alone, which is why it is labelled with the
|
||||
day and the machine rather than printed as a row.
|
||||
|
||||
## Consume in Claude Code
|
||||
|
||||
|
|
@ -1172,7 +1199,9 @@ by name**: it discovers them per call, so a bundle you add, remove or rebuild
|
|||
while the server is running is picked up by the next call. No restart, no
|
||||
configuration edit, no code change.
|
||||
|
||||
Four tools, and each one's description says why it exists:
|
||||
Four tools on a multi-bundle server and **three** on a single-bundle one —
|
||||
`okf_list` is absent where there is nothing to list — and each one's description
|
||||
says why it exists:
|
||||
|
||||
| tool | what it answers |
|
||||
|---|---|
|
||||
|
|
@ -1225,8 +1254,8 @@ reader to run `okf card <bundle>` first. The card is **derived on every run and
|
|||
never written into the bundle**, so there is no second artefact that can
|
||||
disagree with the bytes beside it.
|
||||
|
||||
Measured on two unrelated bundles: two per-bundle skills are identical on 280
|
||||
of 312 and 310 lines. The 62 lines that differ are exactly identity, concept
|
||||
Measured on two unrelated bundles: two per-bundle skills are identical on 281
|
||||
of 313 and 311 lines. The 62 lines that differ are exactly identity, concept
|
||||
count, the conditional-field table, the whole-bundle cost and the breaking
|
||||
point — the five things a rebuild invalidates.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue