Kjell Tore Guttormsen
b6da09cc97
The bundle-only gate returned UNMEASURED and exit 3 on the very arm it was
built for: no metric computable from a bundle alone reaches boundary recall.
`boundary_share` -- declared boundaries that became a concept, over declared
boundaries -- is the one metric measured that orders the arms correctly, and it
needs the publisher's own structure, so it arrives as an input.
Measurement first, threshold after, which is what the order asked for.
P1, the normalisation, derived rather than guessed: stripping all whitespace
and lowercasing reproduces the fasit's own `norm` from its own `title` on
2 761 of 2 761 rows (alphanumerics-only scores 58). P1's own bar is 99 % on the
known-good arm and the literal reading of it reaches 22 of 2 761 -- not because
the normalisation is wrong but because okf's default route moves the numbering
token a publisher glues into a heading over into the concept id. The pair form
(concept's own directory, residual title) reaches 2 737, either reaches 2 759
(99.9 %). Both forms ship and neither is a fallback: `r761-2025-d1` is the
control in the opposite direction at 2 727 literal, 0 paired.
P2, the single corpus, is in the OUTPUT and not only in the document: the bar
is declared `corpora = 1`, every boundary row prints `N = 1 corpus`, and the
line states that `--fasit` is the caller's ASSERTION that this bundle is a
build of the document the fasit describes -- the posture `okf consume --ref`
has. Measured: the K2 reference and `n100-2023` score 0 of 2 761 and read FAIL,
which is the assertion being wrong rather than the bundle being bad.
One bar, at the pinned artifact's own value, 2 759/2 761. It is tight and the
cost is published rather than tuned away: 2 of 4 R761 builds fall under it
(2 752 and 2 727), while any bar between 41.6 % and 98.8 % separates the
known-bad arm from every R761 build measured. The known-bad arm
(`860019-mdb-100`) is 1 148 of 2 761 -- FAIL and exit 1, where the bundle-only
gate gave exit 3.
A fasit is validated at the door: not a list, a row missing `title` or `norm`,
or anything that is not JSON exits 2 with the reason, never a quiet UNMEASURED.
A fasit under five rows is UNMEASURED -- the document floor in the fasit's own
unit.
Without `--fasit` the command is byte-for-byte what it was, held by a test.
`okf check` is untouched; no version bump and no tag. 17 tests red on
assertions before the implementation, the two new doc pins each driven red and
back. Suite 1 869 passed / 1 skipped / 1 870 collected (base 5e5d01c: 1 851).
docs/2026-09-12-g37-terskler.md SS 7 carries the premises re-measured, the
seven bundles, the interval any bar could sit in, and the honesty limits --
including the correction of SS 2's own grep claim, which went false in the
commit that wrote it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
20 KiB
G37 — the thresholds, per file type, before the gate
Order 20260912T202210Z-7596115025-from-.claude, part 1. Base d310dbb
(v0.8.5). Measured 2026-09-12 evening local time; the session crossed midnight,
so the commit lands 2026-09-13. Python 3.14.0, guard 1.4.0, no model call and
no network anywhere in what follows.
The order's own words for what this part must produce: "tabell filtype ×
metrikk × terskel × N × kilde", metrics restricted to what is already measured
or measurable without a model, and "en filtype uten måling får terskel
UNMEASURED, aldri et gjettet tall".
§ 0 — measured / not measured
Measured here. Every count in §§ 2–5: concept and document counts per file
type over nine bundles, one-concept-document shares, empty-body counts,
duplicate-title shares in two forms, short-concept shares, and which of the
four evidence corpora carry a source_file key at all. Each is a shell or
okf quality command printed beside it.
Not measured here. Boundary recall against a fasit (needs a fasit; § 6),
hit@k (needs a question set and a fasit; § 6), anything about .txt, .csv,
.json, .htm, .pptx, .odt, .rtf beyond the fixtures already in the
tree, and whether any threshold below is the right bar — ratification is the
operator's, and this document sets regression bars against a pinned artifact,
which is a different thing (§ 4).
§ 1 — the three metric names, defined here because nothing defines them
The order names "grenseandel", "struktur-null-andel", "tomme konsepter",
Bundle.skipped and "hit@k". Two of those five have no definition anywhere in
this repository:
Measured on the base commit rather than on the working tree, because this document and the module beside it introduce both words:
git grep -c "grenseandel\|boundary share" d310dbb -- docs src→ 0 filesgit grep -ci "struktur.null\|structure.null" d310dbb -- docs src→ 0 files- known-positive control for the query form:
git grep -c "hit@8" d310dbb -- docs→ 19 files, so it can find.
So the definitions below are mine, written here so the gate and this document mean the same thing by them:
| name | definition | what it needs |
|---|---|---|
boundary_share |
recovered boundaries ÷ boundaries the source itself declares | a fasit |
structure_null_share |
documents of a type yielding exactly ONE concept ÷ documents of that type in the bundle | the bundle alone |
empty_concepts |
concepts whose body holds zero non-whitespace characters ÷ concepts of that type | the bundle alone |
hit@k |
questions whose gold document reaches the payload at cut k ÷ questions |
a fasit and a question set |
| run log | N, merged and coded rejections, from the bundle's own log.md |
a bundle Door B wrote |
Bundle.skipped does not exist under that name: the door's counts reach a
bundle through corpus.CorpusReport, whose written form is the run log above
(N, extracted, gated, persisted, substantive, degenerate,
rejected, plus per-code counts). It is a whole-bundle figure and carries
no file type, which is the limit in § 5.
§ 2 — the order's own numbers, re-measured first
The coordination note that arrived with the order asked for exactly this, and three of the five premises moved.
| premise in the order | re-measured | verdict |
|---|---|---|
| 13 extensions in the registry | _CORE_EXTRACTORS 7 + _OPTIONAL_EXTRACTORS 6 = 13 |
holds |
four evidence corpora under ~/repos/vegnormal-okf/build/ferdig/ |
present: n100-2023, n200-2024, n500-2024, r761-2025 — plus r761-2025-generisk and r761-2025-v080, two more R761 builds the order does not name |
holds, and is incomplete |
| "N100 skal gi PASS på de målte typene" | `grep -rl '^source_file:' n100-2023 --include='*.md' | wc -l→ **0 of 446 concepts**. Same forn200-2024(0 of 1 133),n500-2024(0 of 270) andr761-2025` (0 of 2 756) |
| "41,6 % grenser" | not in this repository when the order arrived — measured on the base commit, git grep -c "41[.,]6" d310dbb -- docs src → 0 files. It is vegnormal-okf's: docs/2026-09-10-r2-r761-produkt.md:452, 1 148 of 2 761 declared STS sections, against the XML arm's 2 759 of 2 761 |
holds, and the owner is another repo |
| "0 av 6 hit@k" on the HTML arm | two published numbers for the same 828-document bundle: 0 of 6 (vegnormal, their six fasit questions, 10.09) and 2 of 6 → 4 of 6 with --source-quota 2 (this repo, docs/2026-09-10-k3-runde11-html-og-dominans.md:200,217, questions S1–S6) |
holds for one instrument, not for the other |
The last row is the reason the gate reports a denominator on every line: two true numbers about one bundle, four apart, because the question set differs.
§ 3 — what each corpus can answer
Command: okf quality <bundle> (part 2 of this order), cross-checked against
grep -rh '^source_file:' <bundle> --include='*.md' | sed 's/.*\.//' | sort | uniq -c. The index walk and the directory listing agree on 453, 2 761, 3 206
and 446 concepts — four bundles, both methods.
| bundle | producer | concepts | file types present |
|---|---|---|---|
K2-bundle-default-20260912 |
Door B (pinned reference) | 453 | .pdf 32 docs, .docx 5, .xlsx 2 |
r761-2025-generisk |
Door B | 2 761 | .xml 1 doc |
sk2-bundle-default |
Door B | 2 182 | .pdf 1 doc |
860019-mdb-100 |
Door B | 3 206 | .md 828 docs |
n100-2023 |
vegnormal's own pipeline | 446 | none — no source_file |
n200-2024 |
vegnormal's own pipeline | 1 133 | none |
n500-2024 |
vegnormal's own pipeline | 270 | none |
r761-2025 |
vegnormal's own pipeline | 2 756 | none |
b3 |
— | — | no index.md; the gate refuses to walk a directory (SS 9.2) and exits 2 |
The four corpora the order names as the evidence base carry no file type. The per-file-type thresholds below therefore rest on the K2 reference bundle and, for context only, on the three single-type bundles.
§ 4 — the table
Threshold semantics, stated before the numbers: a threshold here is a
regression bar against a pinned artifact, set at the value measured on that
artifact and never at a rounder number nearby. PASS means no worse than the
reference on this metric. It is not a claim that the bundle is good, and it is
not the operator's ratified quality bar — the K3 method refuses to set one
inside the measurement that produces it ("setting one inside the same work that
produces the first measurement would be fitting the bar to the number").
A rate needs a denominator. MIN_DOCUMENTS_FOR_A_THRESHOLD = 5, and it binds
both the threshold's N and the judged bundle's: this repository's own honesty
limit is "a 1/1 is not a rate" (docs/2026-09-08-k3-runde2-per-filtype.md).
| file type | metric | threshold | N | source |
|---|---|---|---|---|
| every type | empty_concepts |
0 | 8 602 concepts over 4 Door B bundles, all 0 | definitional: corpus.CorpusReport calls a zero-character merge degenerate |
.pdf |
structure_null_share |
8/32 | 32 documents | K2-bundle-default-20260912 |
.docx |
structure_null_share |
2/5 | 5 documents | K2-bundle-default-20260912 |
.xlsx |
— | UNMEASURED | 2 documents | below the floor of 5 |
.xml |
— | UNMEASURED | 1 document | below the floor of 5 |
.md |
— | UNMEASURED | 828 documents, structure_null_share 362/828 measured |
the corpus is one generator's cut of one product, and it is the bundle under suspicion; a bar read off it would pass it by construction |
.html |
— | UNMEASURED | 0 bundles in this repository | the 828-document HTML measurement was made outside it |
.htm, .txt, .csv, .json, .pptx, .odt, .rtf |
— | UNMEASURED | 0 corpus documents | no corpus class in extract._EVIDENCE; .pptx/.odt/.rtf are constructed, N = 1–2 |
Candidates measured and not shipped
Three fasit-free metrics were measured over the same four Door B bundles, with
the known-bad arm (860019-mdb-100, 1 148 of 2 761 boundaries) and the
known-good arm (r761-2025-generisk, 2 759 of 2 761) named in advance:
| candidate | known-bad arm | known-good arm | pdf arm | K2 reference | verdict |
|---|---|---|---|---|---|
| duplicate titles within a document | 0/3 206 (0.0 %) | 349/2 761 (12.6 %) | 355/2 182 | 11/453 | wrong direction |
| concepts under 40 characters | 181/3 206 (5.6 %) | 402/2 761 (14.6 %) | 1/2 182 | 25/453 | wrong direction |
| duplicate titles across the whole bundle | 1 211/3 206 (37.8 %) | 349/2 761 (12.6 %) | 355/2 182 (16.3 %) | 26/453 (5.7 %) | orders correctly, no bar |
The third orders all four bundles correctly and is still not shipped: any bar that separates them sits between 16.3 % and 37.8 %, which is a number read off the two bundles it would judge. That is the operator's call and it is stated in STATE rather than taken here.
§ 5 — three limits that are properties of the instrument
- A rejected document leaves no trace in the bundle. The pinned K2 bundle
holds 32
.pdfdocuments; the corpus held 33, and one was rejectedextractor_empty_pdf. The gate's denominator is the bundle's, never the corpus's, and it prints the run log (N = 43, merged = 39, coded rejections = 4) beside its own numbers so the two are never read as one. A bundle without a log says "no run log" rather than reporting zero rejections. - The run log carries no file type.
codesis per rejection code, not per extension, so "this type failed to extract entirely" is invisible to a bundle-only gate. Naming that would be a change incorpus.py, not here. - The reference bundle passes its own bars by construction.
okf qualityonK2-bundle-default-20260912returningPASSon.pdfand.docxis a control that the gate reproduces its own reference — nothing more.
§ 6 — what would reach the defect that started this
The 41.6 % arm is UNMEASURED under this gate, and that is the honest answer
rather than a hidden one: the metric that fells it is boundary_share, which
needs the publisher's own declared structure. R761 has one (2 761 STS <sec>
titles, ~/repos/vegnormal-okf/build/sk2-fasit-2761.json), and 4 of the 9
bundles above have no fasit at all. A --fasit input, a threshold on
boundary_share, and the operator's bar are one further round, named in STATE
and not begun here. That round is § 7 below (G37b, order
20260912T223334Z-132479088, 2026-09-13).
§ 7 — G37b: --fasit and boundary_share
Order 20260912T223334Z-132479088-from-.claude, on base 5e5d01c (v0.9.0,
pushed, untagged). Measured 2026-09-13. Python 3.14.0, no model call, no
network. The order's own instruction was measurement first: two premises
measured before any threshold is set.
§ 7.0 — the correction this section owes § 2
The row above published grep -rn "41[.,]6" docs/ src/ → 0 as a present-tense
fact. It stopped being true in the same commit, because the document making the
claim writes the number twice. Re-measured today the working tree returns 2
hits, both in this file. The measurement it was meant to be is the one on the
base commit, and it is now written that way. This is the repository's own trap
(memory/falsifiser-egne-negative-paastander.md) firing on the file that
records traps.
§ 7.1 — P1: the normalisation, measured on a known-positive first
The fasit is ~/repos/vegnormal-okf/build/sk2-fasit-2761.json — 665 134 bytes,
a JSON list of exactly 2 761 objects, each carrying title and norm
(sec_type, depth, label, outline_page and id as well). Every norm is
unique, and so is every title.
The normalisation is derived, not guessed. Strip all whitespace, then
lowercase, reproduces the fasit's own norm from its own title on 2 761 of
2 761 rows. Two candidates were run first and fell on the same file:
lower-casing and keeping only alphanumerics scores 58 of 2 761 (it eats the
. in 2.1Hovedprosesser and the - in 3Sted- og elementkode). Unicode NFC
makes no difference here — 2 761 of 2 761 titles and norms are already NFC — and
the rule is the one vegnormal-okf's own measuring script applies, so a number
produced here and a number produced there are the same number.
P1's own bar is 99 % on the known-good arm, and the literal reading of it
fails — for a reason that is measured and is not the normalisation. On
r761-2025-generisk (the declared-structure arm, 2 761 concepts):
| match form | recovered of 2 761 |
|---|---|
literal: normalised concept title equals the fasit's norm |
22 (0.8 %) |
pair: (concept's own directory, normalised residual title) |
2 737 (99.1 %) |
| either | 2 759 (99.9 %) |
The cause is this repository's own id form: STS glues the numbering token into
<title> (11.1Fastmerker), and okf's default route moves that token into the
concept id (11-1/…) while keeping the residual as the title. A gate scoring
the literal form alone would report a 99.9 % arm as 0.8 % and call it a
segmentation defect. Both forms therefore ship, neither as a fallback for the
other: the literal form wants the title with its token, the pair form without,
and no single bundle can offer both. The r761-2025-d1 build is the control in
the opposite direction — literal 2 727, paired 0.
That decomposition is not this round's invention either. It is the M8 correction
vegnormal-okf took verbatim from this repository's own round-14 report; the
instrument here reproduces their two forms so the two repositories cannot
silently measure different things.
Known-positive delivered: 2 759 of 2 761 (99.9 %), the published number to
the unit. Known-negative delivered: 860019-mdb-100 at 1 148 of 2 761
(41.6 %), also to the unit — the arm the bundle-only gate returned
UNMEASURED/exit 3 for.
§ 7.2 — P2: the threshold rests on one product, and says so
The fasit describes one document: R761 Prosesskoden:2025. Every number
below is that product's. The bar is declared with corpora = 1 in the code
(quality.BOUNDARY_THRESHOLD), the row prints N = 1 corpus on every run, and
the printed line also states what --fasit is: an assertion by the caller
that this bundle is a build of the document the fasit describes — the same
posture okf consume --ref has.
That is not a formality. Measured: the pinned 43-document K2 reference bundle
scores 0 of 2 761 and n100-2023 scores 0 of 2 761. Both read FAIL,
and in both cases the right reading is the assertion was wrong, not the
bundle is bad. A gate that could tell those two apart would need a bar on
"does this fasit describe this bundle", and that bar would have to be read off
the bundles it judges.
§ 7.3 — every bundle measured, with the fasit
okf quality <bundle> --fasit ~/repos/vegnormal-okf/build/sk2-fasit-2761.json
| bundle | concepts | literal | paired | recovered of 2 761 | verdict | rc |
|---|---|---|---|---|---|---|
r761-2025-generisk (reference) |
2 761 | 22 | 2 737 | 2 759 (99.9 %) | PASS | 0 |
r761-2025-v080 |
2 761 | 22 | 2 737 | 2 759 (99.9 %) | PASS | 0 |
r761-2025 |
2 756 | 20 | 2 732 | 2 752 (99.7 %) | FAIL | 1 |
r761-2025-d1 (vegnormal's own pipeline) |
2 728 | 2 727 | 0 | 2 727 (98.8 %) | FAIL | 1 |
860019-mdb-100 (known-bad arm) |
3 206 | 1 130 | 28 | 1 148 (41.6 %) | FAIL | 1 |
n100-2023 (another product) |
446 | 0 | 0 | 0 | FAIL | 1 |
K2-bundle-default-20260912 (another product) |
453 | 0 | 0 | 0 | FAIL | 1 |
§ 7.4 — the bar, and why it sits where it sits
| metric | threshold | N | source |
|---|---|---|---|
boundary_share |
2 759/2 761 | 2 761 declared boundaries, 1 corpus | r761-2025-generisk against sk2-fasit-2761.json |
Set at the value measured on the pinned artifact rather than at a rounder
number nearby — the same rule the two bars in § 4 follow, and the same
semantics: a regression bar, not a quality claim. PASS means no worse
than that build of that product.
It is a tight bar and the cost is stated rather than hidden: two of the four R761 builds measured fall under it, at 2 752 and 2 727 of 2 761 — 0.25 % and 1.2 % below the reference. The class separation is nevertheless wide: any bar between 1 148/2 761 (41.6 %) and 2 727/2 761 (98.8 %) separates the known-bad arm from every R761 build measured. The shipped bar is the strictest point of that interval because it is the only one in it that is read off a pinned artifact instead of off the bundles being judged. Moving it inside that interval is the operator's call, and the interval is published here so the call can be made on numbers.
A floor applies here too, in the fasit's own unit:
MIN_DECLARED_FOR_A_THRESHOLD is the same 5 as the document floor, so a fasit
of four rows gives UNMEASURED rather than a share.
§ 7.5 — what did not change
- Without
--fasit, nothing moves.okf quality <bundle>on860019-mdb-100still prints one.mdrow, no boundary row, and exits 3. A test holds it, and the word "boundary" does not appear in the per-type half of the output. - No version bump, no tag, no push. v0.9.0 was already bumped by G37 and is still untagged; the tag is the operator's.
okf checkis untouched. So isvegnormal-okf, which was read and not written.
§ 7.6 — honesty limits of this section
- N = 1 product. Every number in § 7.3 is R761's. Nothing here says what
boundary_sharedoes on a second declared structure, because no second one exists in reach. - The bar fails builds nobody calls bad. Named above rather than tuned away. The alternative — a bar at 2 727 — is a number read off a bundle the gate judges.
- A zero is ambiguous by construction. 0 of 2 761 means this fasit does not describe this bundle on both bundles where it occurred, and the gate cannot distinguish that from a total segmentation failure. It prints both match-form counts so a reader can see which it is (a real build of the product scores in the thousands on one of the two forms).
- One file was read outside this repository beyond the fasit. The order said
to read only the fasit from
vegnormal-okf; deriving the pair-key form required reading their measuring script (experiments/sk2_grenser.py, read-only). The alternative was to guess the match rule, which is what P1 exists to forbid. Nothing there was written. boundary_shareis still not a hit@k. It asks whether a declared section became a concept, never whether a question reaches it.
Honesty limits
- Two thresholds, and one of them rests on five documents.
.docxat 2/5 is exactly at the floor this document sets for itself. It is a bar against a pinned artifact, not a rate anyone should quote. structure_null_sharehas never fired on a real bundle. Of the five Door B bundles measured, none is worse than its reference; the rule'sFAILside is exercised on constructed bundles intests/test_quality.pyonly.empty_conceptsis 0 on 8 602 of 8 602 concepts. The bar is definitional and has never fired on real data either. A metric that never fires is not evidence that nothing is wrong.- One rule was found by running the gate, not by reading it. A one-document
PDF bundle scored 0 of 1 against the 32-document reference and read as
PASS. The floor now binds both denominators; the version of this document written before that run would have published the wrong rule. - Nothing here measures answer quality. hit@k asks whether a gold document reached the payload; this gate does not even ask that.