feat(quality): okf quality --fasit, boundary recall against a declared structure

The bundle-only gate returned UNMEASURED and exit 3 on the very arm it was
built for: no metric computable from a bundle alone reaches boundary recall.
`boundary_share` -- declared boundaries that became a concept, over declared
boundaries -- is the one metric measured that orders the arms correctly, and it
needs the publisher's own structure, so it arrives as an input.

Measurement first, threshold after, which is what the order asked for.

P1, the normalisation, derived rather than guessed: stripping all whitespace
and lowercasing reproduces the fasit's own `norm` from its own `title` on
2 761 of 2 761 rows (alphanumerics-only scores 58). P1's own bar is 99 % on the
known-good arm and the literal reading of it reaches 22 of 2 761 -- not because
the normalisation is wrong but because okf's default route moves the numbering
token a publisher glues into a heading over into the concept id. The pair form
(concept's own directory, residual title) reaches 2 737, either reaches 2 759
(99.9 %). Both forms ship and neither is a fallback: `r761-2025-d1` is the
control in the opposite direction at 2 727 literal, 0 paired.

P2, the single corpus, is in the OUTPUT and not only in the document: the bar
is declared `corpora = 1`, every boundary row prints `N = 1 corpus`, and the
line states that `--fasit` is the caller's ASSERTION that this bundle is a
build of the document the fasit describes -- the posture `okf consume --ref`
has. Measured: the K2 reference and `n100-2023` score 0 of 2 761 and read FAIL,
which is the assertion being wrong rather than the bundle being bad.

One bar, at the pinned artifact's own value, 2 759/2 761. It is tight and the
cost is published rather than tuned away: 2 of 4 R761 builds fall under it
(2 752 and 2 727), while any bar between 41.6 % and 98.8 % separates the
known-bad arm from every R761 build measured. The known-bad arm
(`860019-mdb-100`) is 1 148 of 2 761 -- FAIL and exit 1, where the bundle-only
gate gave exit 3.

A fasit is validated at the door: not a list, a row missing `title` or `norm`,
or anything that is not JSON exits 2 with the reason, never a quiet UNMEASURED.
A fasit under five rows is UNMEASURED -- the document floor in the fasit's own
unit.

Without `--fasit` the command is byte-for-byte what it was, held by a test.
`okf check` is untouched; no version bump and no tag. 17 tests red on
assertions before the implementation, the two new doc pins each driven red and
back. Suite 1 869 passed / 1 skipped / 1 870 collected (base 5e5d01c: 1 851).

docs/2026-09-12-g37-terskler.md SS 7 carries the premises re-measured, the
seven bundles, the interval any bar could sit in, and the honesty limits --
including the correction of SS 2's own grep claim, which went false in the
commit that wrote it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-13 07:27:33 +02:00
commit b6da09cc97
7 changed files with 868 additions and 16 deletions

View file

@ -685,11 +685,42 @@ five documents on both sides — the bundle's and its own — because a `1/1` is
not a rate.
**What a `PASS` is not.** It is a regression bar against a pinned artifact, not
a claim that the cut found the document's own structure. Boundary recall and
hit@k need a fasit and are outside a bundle-only gate; the measurement that
says so, the corpora behind each number, and three candidate metrics that were
measured and not shipped are in
[`docs/2026-09-12-g37-terskler.md`](docs/2026-09-12-g37-terskler.md).
a claim that the cut found the document's own structure. hit@k needs a question
set and is not asked here; the measurement that says so, the corpora behind
each number, and three candidate metrics that were measured and not shipped are
in [`docs/2026-09-12-g37-terskler.md`](docs/2026-09-12-g37-terskler.md).
### Boundary recall: `--fasit`
Give the command the boundaries the source itself declares and it adds one
whole-bundle row, `boundary_share` — the share of them that became a concept:
```sh
okf quality .okf/my-bundle --fasit declared-sections.json
```
The fasit is a JSON list whose rows carry `title` and the key they are matched
on, `norm` (all whitespace stripped, lowercased). An unreadable one exits **2**,
never `UNMEASURED`. A boundary counts as recovered in either of two forms — a
concept whose normalised title equals `norm`, or the pair of the concept's own
directory and its residual title — because the numbering token a publisher glues
into a heading lands in the concept *id* on one route and in the *title* on
another: measured on one 2 761-section standard, the first form alone reaches
**22 of 2 761** where the two together reach **2 759**.
<!-- quality-boundary-threshold: 2759/2761 -->
| metric | threshold | N |
|---|---|---|
| `boundary_share` | 2 759/2 761 | 2 761 declared boundaries, **1 corpus** |
**`--fasit` is an assertion**, the way `okf consume --ref` is: it says this
bundle is a build of the document the fasit describes. A bundle of another
product scores near zero and reads `FAIL` — that is the assertion being wrong,
not the bundle. The bar itself rests on **one product**, which the output says
on every run. Both facts, the arm it separates (1 148 of 2 761 against 2 759 of
2 761) and the interval any bar could sit in are in
[`docs/2026-09-12-g37-terskler.md`](docs/2026-09-12-g37-terskler.md) § 7.
## Consume in Claude Code