open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(consume): the compound-word miss is a degenerate signal's tie-break, behind a flag

Browse source 
The consumer's question about `vann- og frostsikring` in a subsea tunnel
delivered 0 of the 16 concepts covering it, best of them at fused rank 14.
Reproduced with the denominator, then decomposed per signal before anything
was built.

It is not a matcher miss. `normalise("vann- og frostsikring")` already returns
`('vann', 'frostsikring')` on HEAD, the prefix rule already bridges the
inflections, and the best covering concept already answers 7 of 7 question
tokens -- more than any delivered one. A tokeniser rule had nothing to widen.

It is the fusion, but not a weight. RRF ranks every concept in every signal,
including a signal that scored them all the same, and the declared
`(-score, concept_id)` tie-break then orders that group by id. On N500 the
document prior has TWO distinct values over 270 concepts, so the third signal
contributed alphabetical UUID order spread from 1/61 to 1/329 -- enough to put
a concept leading the body signal behind concepts sharing only `tunnel` and
`vann`.

`--tie-shared-rank` lets concepts a signal scores equally share that group's
first rank. The miss closes: best covering 14 -> 3, 2 of 16 delivered. OFF BY
DEFAULT, by the order's own rule: the three requirement lookups hold at rank 1
and the K2 digest holds, but hit@8 over the six published questions falls 5 of
6 to 4 of 6. Decomposed rather than guessed -- K2's prior is coarse (6 values
over 39 documents) rather than degenerate, and one gold sat early in its tie
group. That benefit was never a measurement, but it is a published row.

`--withheld-titles` gives each withheld entry the concept's title, so a reader
can see WHAT was withheld without reading the bundle. 11 lines of code; the
bytes are why it is off. It grows an N500 payload 37.9 % and takes the
629-concept K2 bundle's BOOKKEEPING to 122 704 B -- past the 120 000-byte limit
itself -- which would falsify the breaking point published in the tracked
`skills/okf-consume/SKILL.md` on the day it shipped.

Defaults measured, not asserted: six payload digests built from a frozen
`ff79cfa` (`git archive`, `__file__` checked) and from this tree with both
flags omitted are 6 of 6 identical, and `okf_skill.py` output is identical
apart from the paths each copy writes about itself. Contract checker exit 0 on
eight payloads, both values.

One known-positive did not reproduce and is reported rather than matched: the
order's S7 literal `2ae46f68`/169 573 B is stale by three excerpt-form commits;
HEAD measures `c759a657`/171 614 B.

Suite 1388 -> 1397. Report: docs/2026-09-08-rangeringsbom-sammensatte-ord.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 18:53:18 +02:00
commit c3b645bccf
5 changed files with 630 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

102
tools/okf_consume.py
View file

@ -1004,6 +1004,7 @@ def concept_scores(
cost_vocabulary: bool = False,
weights: Mapping[str, float] | None = None,
lookup: bool = True,
tie_shared_rank: bool = False,
) -> list[tuple[Concept, float, int]]:
"""Every concept, ordered best first, fused from three signals by RRF.
@ -1030,6 +1031,30 @@ def concept_scores(
separately: a concept that answers nothing in the question, sitting in a
document that does, is a GUESS, and a guess is the one thing a declared cut
must not deliver.
**`tie_shared_rank` is the one rule this fusion has for a signal that does
not separate**, off by default, and it is a correction to what the declared
tie-break does rather than a weight. RRF consumes ranks, so a rank is
produced for EVERY concept in EVERY signal -- including a signal that gave
them all the same score. The tie-break then orders that group by
`concept_id`, and the fusion reads the result as if it were a measurement.
MEASURED 2026-09-08 on the N500 bundle (270 concepts): the document prior
has **two** distinct values there and 269 concepts share one, so that
signal contributed the concepts' UUIDs in alphabetical order, spread across
`1/61` to `1/329`. The best concept covering `vann- og frostsikring` in a
subsea tunnel answered **7 of 7** question tokens and led the body signal
at rank 6; it fused to rank 14, outside the cut, behind concepts sharing
only `tunnel` and `vann` whose ids sorted earlier. With shared ranks it
fuses to rank 3.
The rule takes the FIRST position of a score group rather than its middle.
Both were measured on the same four cases; the middle put the same concept
at rank 5 where the first puts it at 3, and neither changed the three
known-positive lookups. First is kept because it is the reading under which
a signal that separates nothing contributes an identical constant to every
concept -- which is the whole claim -- where the middle still varies with
the size of the group a concept lands in.
"""
question_tokens = normalise(question)
bridge = cost_vocabulary and question_uses_cost_vocabulary(question)
@ -1065,8 +1090,23 @@ def concept_scores(
# Sort by score descending, then by id ascending -- the declared
# tie-break, applied before a rank is ever read.
order = sorted(signal, key=lambda key: (-signal[key], key))
for position, concept_id in enumerate(order, start=1):
fused[concept_id] += 1.0 / (RRF_K + position)
if not tie_shared_rank:
for position, concept_id in enumerate(order, start=1):
fused[concept_id] += 1.0 / (RRF_K + position)
continue
# SHARED RANK: every concept a signal scores EQUALLY takes that score
# group's first position, so the signal contributes the same amount to
# each of them and orders none of them. See this function's docstring
# and `tests/test_tie_shared_rank.py` for what it is for.
start = 0
while start < len(order):
stop = start
while stop < len(order) and signal[order[stop]] == signal[order[start]]:
stop += 1
contribution = 1.0 / (RRF_K + start + 1)
for concept_id in order[start:stop]:
fused[concept_id] += contribution
start = stop
lexical = (
{
concept.concept_id: int(signals[0][concept.concept_id] + signals[1][concept.concept_id])
@ -1356,12 +1396,30 @@ def build_payload(
cost_vocabulary: bool = False,
reserve_top_rank: bool = False,
rarity_weight: bool = False,
tie_shared_rank: bool = False,
withheld_titles: bool = False,
) -> dict[str, object]:
"""One bundle plus one question, cut to one contract-conformant payload.
Pure with respect to the clock and the network: the same
`(bundle_root, question, k, limit, cost_vocabulary, reserve_top_rank,
rarity_weight)` at the same bytes returns the same object, every time.
rarity_weight, tie_shared_rank, withheld_titles)` at the same bytes returns
the same object, every time.
**`withheld_titles` (default off) names what was dropped.** A `withheld`
entry carries `concept_id` and `rule` and no title, so a reader told that
262 concepts were withheld cannot tell WHAT was withheld without reading
the bundle -- which SS 2.2 forbids. The title closes that.
OFF BY DEFAULT BY MEASUREMENT, not by taste. Measured 2026-09-08: on N500
the payload grows 41 364 -> 57 023 bytes (+37.9 %), and on the 629-concept
K2 bundle the bookkeeping -- everything that is not an excerpt -- grows to
**122 704 bytes, past the 120 000-byte limit itself**. The instantiated
skill publishes that breaking point as "~75 KB at 629 concepts, reached at
roughly 8 000 concepts"; on by default would make that sentence false and
would move every consumer's bytes for a field none of them asked for.
Whether the naming is worth the bookkeeping is the caller's call, and the
flag is how it stays one.
"""
case, expected, measured = known_positive()
if expected != measured:
@ -1400,7 +1458,9 @@ def build_payload(
),
cost_vocabulary=cost_vocabulary,
weights=weights,
tie_shared_rank=tie_shared_rank,
)
titles_by_id = {concept.concept_id: concept.title for concept in concepts}
matched = sum(1 for _, _, lexical in ranked if lexical > 0)
delivered, withheld, reserved = cut(ranked, k=k, limit=limit, reserve_top_rank=reserve_top_rank)
spent = sum(excerpt_weight(excerpt) for excerpt in delivered)
@ -1460,7 +1520,19 @@ def build_payload(
},
"question": question,
"excerpts": list(delivered),
"withheld": [{"concept_id": concept_id, "rule": rule} for concept_id, rule in withheld],
# Emitted only under the flag, and then only where the concept carries
# a title, so a bundle whose concepts have none produces the same bytes
# either way. See this function's docstring for the measurement that
# keeps the default off.
"withheld": [
{"concept_id": concept_id, "rule": rule}
| (
{"title": titles_by_id[concept_id]}
if withheld_titles and titles_by_id.get(concept_id)
else {}
)
for concept_id, rule in withheld
],
}
@ -1523,6 +1595,26 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"one 96->103 worse. See docs/2026-09-08-sjeldenhetsvekt.md"
),
)
parser.add_argument(
"--tie-shared-rank",
action="store_true",
help=(
"let concepts a signal scores EQUALLY share that score group's "
"first rank, so a signal that separates nothing contributes the "
"same constant to each of them instead of ordering them by id. OFF "
"by default. See docs/2026-09-08-rangeringsbom-sammensatte-ord.md"
),
)
parser.add_argument(
"--withheld-titles",
action="store_true",
help=(
"give each withheld entry the concept's title, so a reader can see "
"WHAT was withheld without reading the bundle. OFF by default: it "
"grew a 270-concept payload by 37.9 %% and pushed a 629-concept "
"bundle's bookkeeping past the budget limit itself"
),
)
parser.add_argument("--out", type=Path, default=None, help="write here instead of stdout")
parser.add_argument(
"--ref",
@ -1557,6 +1649,8 @@ def main(argv: list[str] | None = None) -> int:
cost_vocabulary=args.cost_vocabulary,
reserve_top_rank=args.reserve_top_rank,
rarity_weight=args.rarity_weight,
tie_shared_rank=args.tie_shared_rank,
withheld_titles=args.withheld_titles,
)
except ConsumeError as error:
print(f"okf_consume: FAILED - {error}", file=sys.stderr)