fix(consume): the compound-word miss is a degenerate signal's tie-break, behind a flag

The consumer's question about `vann- og frostsikring` in a subsea tunnel
delivered 0 of the 16 concepts covering it, best of them at fused rank 14.
Reproduced with the denominator, then decomposed per signal before anything
was built.

It is not a matcher miss. `normalise("vann- og frostsikring")` already returns
`('vann', 'frostsikring')` on HEAD, the prefix rule already bridges the
inflections, and the best covering concept already answers 7 of 7 question
tokens -- more than any delivered one. A tokeniser rule had nothing to widen.

It is the fusion, but not a weight. RRF ranks every concept in every signal,
including a signal that scored them all the same, and the declared
`(-score, concept_id)` tie-break then orders that group by id. On N500 the
document prior has TWO distinct values over 270 concepts, so the third signal
contributed alphabetical UUID order spread from 1/61 to 1/329 -- enough to put
a concept leading the body signal behind concepts sharing only `tunnel` and
`vann`.

`--tie-shared-rank` lets concepts a signal scores equally share that group's
first rank. The miss closes: best covering 14 -> 3, 2 of 16 delivered. OFF BY
DEFAULT, by the order's own rule: the three requirement lookups hold at rank 1
and the K2 digest holds, but hit@8 over the six published questions falls 5 of
6 to 4 of 6. Decomposed rather than guessed -- K2's prior is coarse (6 values
over 39 documents) rather than degenerate, and one gold sat early in its tie
group. That benefit was never a measurement, but it is a published row.

`--withheld-titles` gives each withheld entry the concept's title, so a reader
can see WHAT was withheld without reading the bundle. 11 lines of code; the
bytes are why it is off. It grows an N500 payload 37.9 % and takes the
629-concept K2 bundle's BOOKKEEPING to 122 704 B -- past the 120 000-byte limit
itself -- which would falsify the breaking point published in the tracked
`skills/okf-consume/SKILL.md` on the day it shipped.

Defaults measured, not asserted: six payload digests built from a frozen
`ff79cfa` (`git archive`, `__file__` checked) and from this tree with both
flags omitted are 6 of 6 identical, and `okf_skill.py` output is identical
apart from the paths each copy writes about itself. Contract checker exit 0 on
eight payloads, both values.

One known-positive did not reproduce and is reported rather than matched: the
order's S7 literal `2ae46f68`/169 573 B is stale by three excerpt-form commits;
HEAD measures `c759a657`/171 614 B.

Suite 1388 -> 1397. Report: docs/2026-09-08-rangeringsbom-sammensatte-ord.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 18:53:18 +02:00
commit c3b645bccf
5 changed files with 630 additions and 6 deletions

View file

@ -609,6 +609,51 @@ def test_a_concept_whose_verified_cannot_be_read_is_withheld_by_name() -> None:
assert dict(withheld)["dyp/nivaa/blokkform-verifisert"] == "verified_unreadable"
def test_a_withheld_entry_names_what_was_dropped_under_the_flag() -> None:
# A reader who is told 262 concepts were withheld, by id and rule alone,
# cannot tell WHAT was withheld without reading the bundle -- which SS 2.2
# forbids. The title closes that, and it is emitted only where the concept
# carries one.
payload = okf_consume.build_payload(
FIXTURE, question="Hvordan skal prisene fylles ut?", withheld_titles=True
)
entries = payload["withheld"]
assert isinstance(entries, list) and entries
titled = [entry for entry in entries if "title" in entry]
assert titled, "no withheld entry carried a title, so the rule measures nothing"
concepts = {concept.concept_id: concept for concept in _fixture_concepts()}
for entry in entries:
concept = concepts[str(entry["concept_id"])]
if concept.title:
assert entry["title"] == concept.title
else:
assert "title" not in entry
def test_no_withheld_entry_names_anything_without_the_flag() -> None:
# The default is what every consumer already runs, and this is the
# measurement that keeps it theirs: a title on every withheld entry grew a
# 270-concept payload by 37.9 % and pushed a 629-concept bundle's
# bookkeeping past the budget limit itself.
payload = okf_consume.build_payload(FIXTURE, question="Hvordan skal prisene fylles ut?")
entries = payload["withheld"]
assert isinstance(entries, list) and entries
assert all(set(entry) == {"concept_id", "rule"} for entry in entries)
def test_the_withheld_title_flag_costs_bytes_and_the_default_pays_none() -> None:
question = "Hvordan skal prisene fylles ut?"
off = okf_consume.serialise(okf_consume.build_payload(FIXTURE, question=question))
explicit_off = okf_consume.serialise(
okf_consume.build_payload(FIXTURE, question=question, withheld_titles=False)
)
on = okf_consume.serialise(
okf_consume.build_payload(FIXTURE, question=question, withheld_titles=True)
)
assert off == explicit_off
assert len(on.encode("utf-8")) > len(off.encode("utf-8"))
def test_delivered_and_withheld_partition_the_considered_set() -> None:
delivered, withheld, considered = _cut_fixture()
delivered_ids = {excerpt["concept_id"] for excerpt in delivered}
@ -1224,7 +1269,12 @@ def test_a_cost_question_reaches_no_price_concept_without_the_flag() -> None:
counts, withheld = payload["denominators"], payload["withheld"]
assert isinstance(counts, dict) and isinstance(withheld, list)
assert counts["delivered"] == 0
assert {"concept_id": "krav/pristabell", "rule": "no_lexical_match"} in withheld
# By the two fields this test is about, not by the whole entry: the entry
# also carries the concept's title, and pinning the exact dict here would
# make an unrelated widening of the withheld form fail a cost-question test.
assert {(entry["concept_id"], entry["rule"]) for entry in withheld} >= {
("krav/pristabell", "no_lexical_match")
}
def test_the_cost_vocabulary_flag_bridges_a_question_and_a_document_that_share_no_word() -> None: