llm-ingestion-okf/tools
Kjell Tore Guttormsen c3b645bccf fix(consume): the compound-word miss is a degenerate signal's tie-break, behind a flag
The consumer's question about `vann- og frostsikring` in a subsea tunnel
delivered 0 of the 16 concepts covering it, best of them at fused rank 14.
Reproduced with the denominator, then decomposed per signal before anything
was built.

It is not a matcher miss. `normalise("vann- og frostsikring")` already returns
`('vann', 'frostsikring')` on HEAD, the prefix rule already bridges the
inflections, and the best covering concept already answers 7 of 7 question
tokens -- more than any delivered one. A tokeniser rule had nothing to widen.

It is the fusion, but not a weight. RRF ranks every concept in every signal,
including a signal that scored them all the same, and the declared
`(-score, concept_id)` tie-break then orders that group by id. On N500 the
document prior has TWO distinct values over 270 concepts, so the third signal
contributed alphabetical UUID order spread from 1/61 to 1/329 -- enough to put
a concept leading the body signal behind concepts sharing only `tunnel` and
`vann`.

`--tie-shared-rank` lets concepts a signal scores equally share that group's
first rank. The miss closes: best covering 14 -> 3, 2 of 16 delivered. OFF BY
DEFAULT, by the order's own rule: the three requirement lookups hold at rank 1
and the K2 digest holds, but hit@8 over the six published questions falls 5 of
6 to 4 of 6. Decomposed rather than guessed -- K2's prior is coarse (6 values
over 39 documents) rather than degenerate, and one gold sat early in its tie
group. That benefit was never a measurement, but it is a published row.

`--withheld-titles` gives each withheld entry the concept's title, so a reader
can see WHAT was withheld without reading the bundle. 11 lines of code; the
bytes are why it is off. It grows an N500 payload 37.9 % and takes the
629-concept K2 bundle's BOOKKEEPING to 122 704 B -- past the 120 000-byte limit
itself -- which would falsify the breaking point published in the tracked
`skills/okf-consume/SKILL.md` on the day it shipped.

Defaults measured, not asserted: six payload digests built from a frozen
`ff79cfa` (`git archive`, `__file__` checked) and from this tree with both
flags omitted are 6 of 6 identical, and `okf_skill.py` output is identical
apart from the paths each copy writes about itself. Contract checker exit 0 on
eight payloads, both values.

One known-positive did not reproduce and is reported rather than matched: the
order's S7 literal `2ae46f68`/169 573 B is stale by three excerpt-form commits;
HEAD measures `c759a657`/171 614 B.

Suite 1388 -> 1397. Report: docs/2026-09-08-rangeringsbom-sammensatte-ord.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 18:53:18 +02:00
..
okf_adjudicate.py fix(tools): do not blame the plan for a verdict okf_adjudicate built 2026-09-05 07:35:14 +02:00
okf_cid_measure.py feat(tools): a re-measurable CID-glyph-share instrument for K2 2026-09-04 20:23:34 +02:00
okf_consume.py fix(consume): the compound-word miss is a degenerate signal's tie-break, behind a flag 2026-09-08 18:53:18 +02:00
okf_consume_measure.py test(consume): hit@8 over six questions against a random-ranker baseline 2026-09-07 09:37:14 +02:00
okf_contract_check.py feat(consume): give every excerpt the name and the address an answer must cite 2026-09-08 15:01:41 +02:00
okf_corpus_run.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
okf_fidelity.py test(fidelity): K2 on pptx, odt and rtf -- 59/59 and 20/20, on a set we wrote 2026-09-07 05:22:12 +02:00
okf_outline_measure.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
okf_propose_segments.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
okf_skill.py feat(consume): carry every source_* key by prefix, and generate a skill per bundle 2026-09-08 15:13:22 +02:00
okf_table_measure.py feat(tools): a re-measurable grid-table reach instrument for K3 [skip-docs] 2026-09-07 10:56:12 +02:00
okf_watch.py feat(watch): weekly OKF upstream watch that can prove it found nothing 2026-08-23 20:38:37 +02:00