feat(consume): a question that accounts for a concept's WHOLE title takes it first

`title_covered_hits` names the concepts whose every title token is a token of
the question, and `concept_scores` reads them before the rest -- a PARTITION
below the lookup partition, never a fourth RRF signal.

The arithmetic that rules a signal out is the same one `lookup_hits` was made
a partition on: RRF consumes ranks alone, so with shared ranks a rule whose
positive group is SMALL separates least of all. Measured on a 2 761-concept
bundle of one standard, the group is 1, 1, 1, 1, 9, 2, 0 and 0 concepts over
the eight questions -- as a signal it moved hit@1 not at all.

By EQUALITY and never by shared prefix: four shared leading characters take
the group from 1 to 6 on one question and 9 to 31 on another, and the
answering section falls to candidate rank 6 with the known-positive to 2.

ON by default with the opt-out `--no-title-covered`. Suite: 1580 passed,
1 skipped; the pinned K2 bundle green and unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 22:08:57 +02:00
commit 6bb21891b8

View file

@ -1158,6 +1158,77 @@ DEFAULT_TIE_SHARED_RANK = True
DEFAULT_STEM_PREFIX = True DEFAULT_STEM_PREFIX = True
#: Round 16. Whether a concept whose WHOLE title the question accounts for is
#: read before the concepts the fusion ranked above it. ON since 2026-09-11.
#:
#: THE DEFECT IT REPAIRS. Both lexical signals are unnormalised COVERAGE
#: COUNTS -- one per question token the candidate answers to -- so nothing in
#: the fusion measures how much of the CANDIDATE the question accounts for. A
#: section titled with the question's subject and nothing else scores exactly
#: what a narrower section titled with that subject PLUS a qualifier scores,
#: and then loses to it on the body count. Measured 2026-09-11 on a
#: 2 761-concept bundle of one standard, where the answering section carries
#: the bare term as its title on three of six scored questions: `Hovedprosesser`
#: behind `Hovedprosess 81 ...`, `Armering` behind `Armering av ...`,
#: `Inspeksjon` behind `Enkel inspeksjon`.
#:
#: WHY A PARTITION AND NOT A FOURTH SIGNAL, measured rather than argued. RRF
#: consumes RANKS ONLY, and with shared ranks a signal whose positive group is
#: SMALL separates least of all: the group takes position 1 and everyone else
#: position `len(group) + 1`, so a rule firing on 1 concept of 2 761 is worth
#: `1/61 - 1/62` to it -- an order of magnitude under the body-signal gap it
#: has to close. Measured as a signal on that bundle it moved hit@1 not at all
#: (3 of 6); as a partition it reaches 6 of 6 candidate rank 1 with the
#: known-positive still at rank 1. `lookup_hits` is the same shape and was made
#: a partition on the same arithmetic.
#:
#: LIKE `--tie-shared-rank` AND `--stem-prefix`, THIS MOVES A PAYLOAD WITH NO
#: BUNDLE CHANGING. A consumer pinned to the previous excerpt order needs
#: `--no-title-covered`.
DEFAULT_TITLE_COVERED = True
def title_covered_hits(concepts: Sequence["Concept"], question: str) -> tuple[str, ...]:
"""The concepts whose ENTIRE title the question accounts for.
Every token of the title is a token of the question, so the title says
nothing the question did not ask about. That is a statement about the
CANDIDATE -- the complement of the coverage counts, which are statements
about the question -- and it is the one thing separating a section titled
with the subject from a narrower section titled with the subject plus a
qualifier.
**BY EQUALITY, never by shared prefix, and that is measured rather than
assumed.** `tokens_match` accepts four shared leading characters, which
would admit `Anchorage` beside `Anchoring` and, on the 2 761-concept
bundle, took the group from 1 concept to 6 on one question and from 9 to 31
on another -- the answering section falling to candidate rank 6 and the
known-positive to rank 2. Under equality both hold rank 1. The precedent is
`tokens_match`'s own: an identifier matches by equality alone, for the same
reason -- a prefix rule built for compounds says nothing true about a name.
Returns the EMPTY tuple when no concept qualifies, which is what makes the
rule invisible to every question that names no section outright. Byte
sorted, so several holders of one title arrive in a declared order.
Reads `title` alone and not the concept id: an id segment is this
library's reduction of a title, so counting it would let the same words
qualify a concept twice, and the document uuid that the id carries on a
single-document bundle is in no question ever asked.
"""
question_tokens = set(normalise(question))
if not question_tokens:
return ()
return tuple(
sorted(
concept.concept_id
for concept in concepts
if (title_tokens := normalise(concept.title))
and all(token in question_tokens for token in title_tokens)
)
)
def concept_scores( def concept_scores(
concepts: Sequence[Concept], concepts: Sequence[Concept],
question: str, question: str,
@ -1167,6 +1238,7 @@ def concept_scores(
weights: Mapping[str, float] | None = None, weights: Mapping[str, float] | None = None,
lookup: bool = True, lookup: bool = True,
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK, tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
title_covered: bool = DEFAULT_TITLE_COVERED,
stems: frozenset[str] | None = None, stems: frozenset[str] | None = None,
) -> list[tuple[Concept, float, int]]: ) -> list[tuple[Concept, float, int]]:
"""Every concept, ordered best first, fused from three signals by RRF. """Every concept, ordered best first, fused from three signals by RRF.
@ -1301,6 +1373,18 @@ def concept_scores(
) )
by_id = {concept.concept_id: concept for concept in concepts} by_id = {concept.concept_id: concept for concept in concepts}
ranked_ids = sorted(fused, key=lambda key: (-fused[key], key)) ranked_ids = sorted(fused, key=lambda key: (-fused[key], key))
covered = set(title_covered_hits(concepts, question)) if title_covered else set()
if covered:
# A PARTITION, not a signal, and it lands BELOW the lookup partition
# so a question that NAMES a concept still reads that one first. See
# `DEFAULT_TITLE_COVERED` for the arithmetic that rules a signal out,
# and `tests/test_title_covered.py` for the mechanism on a fixture.
#
# STABLE: both parts keep the order the fusion gave them, so nothing
# here depends on dict order.
ranked_ids = [key for key in ranked_ids if key in covered] + [
key for key in ranked_ids if key not in covered
]
named = set(lookup_hits(concepts, question)) if lookup else set() named = set(lookup_hits(concepts, question)) if lookup else set()
if named: if named:
# THE LOOKUP LANDS BEFORE THE FUSION'S OUTPUT IS READ, and it is a # THE LOOKUP LANDS BEFORE THE FUSION'S OUTPUT IS READ, and it is a
@ -1627,6 +1711,7 @@ def build_payload(
reserve_top_rank: bool = False, reserve_top_rank: bool = False,
rarity_weight: bool = False, rarity_weight: bool = False,
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK, tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
title_covered: bool = DEFAULT_TITLE_COVERED,
withheld_titles: bool = False, withheld_titles: bool = False,
stem_prefix: bool = DEFAULT_STEM_PREFIX, stem_prefix: bool = DEFAULT_STEM_PREFIX,
source_quota: int | None = DEFAULT_SOURCE_QUOTA, source_quota: int | None = DEFAULT_SOURCE_QUOTA,
@ -1635,7 +1720,8 @@ def build_payload(
Pure with respect to the clock and the network: the same Pure with respect to the clock and the network: the same
`(bundle_root, question, k, limit, cost_vocabulary, reserve_top_rank, `(bundle_root, question, k, limit, cost_vocabulary, reserve_top_rank,
rarity_weight, tie_shared_rank, withheld_titles)` at the same bytes returns rarity_weight, tie_shared_rank, title_covered, withheld_titles)` at the
same bytes returns
the same object, every time. the same object, every time.
**`withheld_titles` (default off) names what was dropped.** A `withheld` **`withheld_titles` (default off) names what was dropped.** A `withheld`
@ -1703,6 +1789,7 @@ def build_payload(
cost_vocabulary=cost_vocabulary, cost_vocabulary=cost_vocabulary,
weights=weights, weights=weights,
tie_shared_rank=tie_shared_rank, tie_shared_rank=tie_shared_rank,
title_covered=title_covered,
stems=stems, stems=stems,
) )
titles_by_id = {concept.concept_id: concept.title for concept in concepts} titles_by_id = {concept.concept_id: concept.title for concept in concepts}
@ -1883,6 +1970,26 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
dest="stem_prefix", dest="stem_prefix",
help="The rule's explicit opt-out, reproducing the pre-round-10 matcher", help="The rule's explicit opt-out, reproducing the pre-round-10 matcher",
) )
parser.add_argument(
"--title-covered",
action="store_true",
default=DEFAULT_TITLE_COVERED,
help=(
"read a concept whose WHOLE title the question accounts for before "
"the concepts the fusion ranked above it. ON since 2026-09-11. "
"Measured on a 2 761-concept bundle of one standard: hit@1 over six "
"questions 3 of 6 -> 6 of 6 with the known-positive holding rank 1, "
"where none of the six existing reading-side flags moved that "
"number at all. The title is read by EQUALITY, never by shared "
"prefix. See docs/2026-09-11-k3-runde16-hele-tittelen-tar-ruten.md"
),
)
parser.add_argument(
"--no-title-covered",
action="store_false",
dest="title_covered",
help="The rule's explicit opt-out, reproducing the pre-round-16 excerpt order",
)
parser.add_argument( parser.add_argument(
"--source-quota", "--source-quota",
type=int, type=int,
@ -1951,6 +2058,7 @@ def main(argv: list[str] | None = None) -> int:
reserve_top_rank=args.reserve_top_rank, reserve_top_rank=args.reserve_top_rank,
rarity_weight=args.rarity_weight, rarity_weight=args.rarity_weight,
tie_shared_rank=args.tie_shared_rank, tie_shared_rank=args.tie_shared_rank,
title_covered=args.title_covered,
stem_prefix=args.stem_prefix, stem_prefix=args.stem_prefix,
source_quota=args.source_quota, source_quota=args.source_quota,
withheld_titles=args.withheld_titles, withheld_titles=args.withheld_titles,