feat(consume): a question that accounts for a concept's WHOLE title takes it first
`title_covered_hits` names the concepts whose every title token is a token of the question, and `concept_scores` reads them before the rest -- a PARTITION below the lookup partition, never a fourth RRF signal. The arithmetic that rules a signal out is the same one `lookup_hits` was made a partition on: RRF consumes ranks alone, so with shared ranks a rule whose positive group is SMALL separates least of all. Measured on a 2 761-concept bundle of one standard, the group is 1, 1, 1, 1, 9, 2, 0 and 0 concepts over the eight questions -- as a signal it moved hit@1 not at all. By EQUALITY and never by shared prefix: four shared leading characters take the group from 1 to 6 on one question and 9 to 31 on another, and the answering section falls to candidate rank 6 with the known-positive to 2. ON by default with the opt-out `--no-title-covered`. Suite: 1580 passed, 1 skipped; the pinned K2 bundle green and unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
14ae36ad2e
commit
6bb21891b8
1 changed files with 109 additions and 1 deletions
|
|
@ -1158,6 +1158,77 @@ DEFAULT_TIE_SHARED_RANK = True
|
||||||
DEFAULT_STEM_PREFIX = True
|
DEFAULT_STEM_PREFIX = True
|
||||||
|
|
||||||
|
|
||||||
|
#: Round 16. Whether a concept whose WHOLE title the question accounts for is
|
||||||
|
#: read before the concepts the fusion ranked above it. ON since 2026-09-11.
|
||||||
|
#:
|
||||||
|
#: THE DEFECT IT REPAIRS. Both lexical signals are unnormalised COVERAGE
|
||||||
|
#: COUNTS -- one per question token the candidate answers to -- so nothing in
|
||||||
|
#: the fusion measures how much of the CANDIDATE the question accounts for. A
|
||||||
|
#: section titled with the question's subject and nothing else scores exactly
|
||||||
|
#: what a narrower section titled with that subject PLUS a qualifier scores,
|
||||||
|
#: and then loses to it on the body count. Measured 2026-09-11 on a
|
||||||
|
#: 2 761-concept bundle of one standard, where the answering section carries
|
||||||
|
#: the bare term as its title on three of six scored questions: `Hovedprosesser`
|
||||||
|
#: behind `Hovedprosess 81 ...`, `Armering` behind `Armering av ...`,
|
||||||
|
#: `Inspeksjon` behind `Enkel inspeksjon`.
|
||||||
|
#:
|
||||||
|
#: WHY A PARTITION AND NOT A FOURTH SIGNAL, measured rather than argued. RRF
|
||||||
|
#: consumes RANKS ONLY, and with shared ranks a signal whose positive group is
|
||||||
|
#: SMALL separates least of all: the group takes position 1 and everyone else
|
||||||
|
#: position `len(group) + 1`, so a rule firing on 1 concept of 2 761 is worth
|
||||||
|
#: `1/61 - 1/62` to it -- an order of magnitude under the body-signal gap it
|
||||||
|
#: has to close. Measured as a signal on that bundle it moved hit@1 not at all
|
||||||
|
#: (3 of 6); as a partition it reaches 6 of 6 candidate rank 1 with the
|
||||||
|
#: known-positive still at rank 1. `lookup_hits` is the same shape and was made
|
||||||
|
#: a partition on the same arithmetic.
|
||||||
|
#:
|
||||||
|
#: LIKE `--tie-shared-rank` AND `--stem-prefix`, THIS MOVES A PAYLOAD WITH NO
|
||||||
|
#: BUNDLE CHANGING. A consumer pinned to the previous excerpt order needs
|
||||||
|
#: `--no-title-covered`.
|
||||||
|
DEFAULT_TITLE_COVERED = True
|
||||||
|
|
||||||
|
|
||||||
|
def title_covered_hits(concepts: Sequence["Concept"], question: str) -> tuple[str, ...]:
|
||||||
|
"""The concepts whose ENTIRE title the question accounts for.
|
||||||
|
|
||||||
|
Every token of the title is a token of the question, so the title says
|
||||||
|
nothing the question did not ask about. That is a statement about the
|
||||||
|
CANDIDATE -- the complement of the coverage counts, which are statements
|
||||||
|
about the question -- and it is the one thing separating a section titled
|
||||||
|
with the subject from a narrower section titled with the subject plus a
|
||||||
|
qualifier.
|
||||||
|
|
||||||
|
**BY EQUALITY, never by shared prefix, and that is measured rather than
|
||||||
|
assumed.** `tokens_match` accepts four shared leading characters, which
|
||||||
|
would admit `Anchorage` beside `Anchoring` and, on the 2 761-concept
|
||||||
|
bundle, took the group from 1 concept to 6 on one question and from 9 to 31
|
||||||
|
on another -- the answering section falling to candidate rank 6 and the
|
||||||
|
known-positive to rank 2. Under equality both hold rank 1. The precedent is
|
||||||
|
`tokens_match`'s own: an identifier matches by equality alone, for the same
|
||||||
|
reason -- a prefix rule built for compounds says nothing true about a name.
|
||||||
|
|
||||||
|
Returns the EMPTY tuple when no concept qualifies, which is what makes the
|
||||||
|
rule invisible to every question that names no section outright. Byte
|
||||||
|
sorted, so several holders of one title arrive in a declared order.
|
||||||
|
|
||||||
|
Reads `title` alone and not the concept id: an id segment is this
|
||||||
|
library's reduction of a title, so counting it would let the same words
|
||||||
|
qualify a concept twice, and the document uuid that the id carries on a
|
||||||
|
single-document bundle is in no question ever asked.
|
||||||
|
"""
|
||||||
|
question_tokens = set(normalise(question))
|
||||||
|
if not question_tokens:
|
||||||
|
return ()
|
||||||
|
return tuple(
|
||||||
|
sorted(
|
||||||
|
concept.concept_id
|
||||||
|
for concept in concepts
|
||||||
|
if (title_tokens := normalise(concept.title))
|
||||||
|
and all(token in question_tokens for token in title_tokens)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def concept_scores(
|
def concept_scores(
|
||||||
concepts: Sequence[Concept],
|
concepts: Sequence[Concept],
|
||||||
question: str,
|
question: str,
|
||||||
|
|
@ -1167,6 +1238,7 @@ def concept_scores(
|
||||||
weights: Mapping[str, float] | None = None,
|
weights: Mapping[str, float] | None = None,
|
||||||
lookup: bool = True,
|
lookup: bool = True,
|
||||||
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
|
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
|
||||||
|
title_covered: bool = DEFAULT_TITLE_COVERED,
|
||||||
stems: frozenset[str] | None = None,
|
stems: frozenset[str] | None = None,
|
||||||
) -> list[tuple[Concept, float, int]]:
|
) -> list[tuple[Concept, float, int]]:
|
||||||
"""Every concept, ordered best first, fused from three signals by RRF.
|
"""Every concept, ordered best first, fused from three signals by RRF.
|
||||||
|
|
@ -1301,6 +1373,18 @@ def concept_scores(
|
||||||
)
|
)
|
||||||
by_id = {concept.concept_id: concept for concept in concepts}
|
by_id = {concept.concept_id: concept for concept in concepts}
|
||||||
ranked_ids = sorted(fused, key=lambda key: (-fused[key], key))
|
ranked_ids = sorted(fused, key=lambda key: (-fused[key], key))
|
||||||
|
covered = set(title_covered_hits(concepts, question)) if title_covered else set()
|
||||||
|
if covered:
|
||||||
|
# A PARTITION, not a signal, and it lands BELOW the lookup partition
|
||||||
|
# so a question that NAMES a concept still reads that one first. See
|
||||||
|
# `DEFAULT_TITLE_COVERED` for the arithmetic that rules a signal out,
|
||||||
|
# and `tests/test_title_covered.py` for the mechanism on a fixture.
|
||||||
|
#
|
||||||
|
# STABLE: both parts keep the order the fusion gave them, so nothing
|
||||||
|
# here depends on dict order.
|
||||||
|
ranked_ids = [key for key in ranked_ids if key in covered] + [
|
||||||
|
key for key in ranked_ids if key not in covered
|
||||||
|
]
|
||||||
named = set(lookup_hits(concepts, question)) if lookup else set()
|
named = set(lookup_hits(concepts, question)) if lookup else set()
|
||||||
if named:
|
if named:
|
||||||
# THE LOOKUP LANDS BEFORE THE FUSION'S OUTPUT IS READ, and it is a
|
# THE LOOKUP LANDS BEFORE THE FUSION'S OUTPUT IS READ, and it is a
|
||||||
|
|
@ -1627,6 +1711,7 @@ def build_payload(
|
||||||
reserve_top_rank: bool = False,
|
reserve_top_rank: bool = False,
|
||||||
rarity_weight: bool = False,
|
rarity_weight: bool = False,
|
||||||
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
|
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
|
||||||
|
title_covered: bool = DEFAULT_TITLE_COVERED,
|
||||||
withheld_titles: bool = False,
|
withheld_titles: bool = False,
|
||||||
stem_prefix: bool = DEFAULT_STEM_PREFIX,
|
stem_prefix: bool = DEFAULT_STEM_PREFIX,
|
||||||
source_quota: int | None = DEFAULT_SOURCE_QUOTA,
|
source_quota: int | None = DEFAULT_SOURCE_QUOTA,
|
||||||
|
|
@ -1635,7 +1720,8 @@ def build_payload(
|
||||||
|
|
||||||
Pure with respect to the clock and the network: the same
|
Pure with respect to the clock and the network: the same
|
||||||
`(bundle_root, question, k, limit, cost_vocabulary, reserve_top_rank,
|
`(bundle_root, question, k, limit, cost_vocabulary, reserve_top_rank,
|
||||||
rarity_weight, tie_shared_rank, withheld_titles)` at the same bytes returns
|
rarity_weight, tie_shared_rank, title_covered, withheld_titles)` at the
|
||||||
|
same bytes returns
|
||||||
the same object, every time.
|
the same object, every time.
|
||||||
|
|
||||||
**`withheld_titles` (default off) names what was dropped.** A `withheld`
|
**`withheld_titles` (default off) names what was dropped.** A `withheld`
|
||||||
|
|
@ -1703,6 +1789,7 @@ def build_payload(
|
||||||
cost_vocabulary=cost_vocabulary,
|
cost_vocabulary=cost_vocabulary,
|
||||||
weights=weights,
|
weights=weights,
|
||||||
tie_shared_rank=tie_shared_rank,
|
tie_shared_rank=tie_shared_rank,
|
||||||
|
title_covered=title_covered,
|
||||||
stems=stems,
|
stems=stems,
|
||||||
)
|
)
|
||||||
titles_by_id = {concept.concept_id: concept.title for concept in concepts}
|
titles_by_id = {concept.concept_id: concept.title for concept in concepts}
|
||||||
|
|
@ -1883,6 +1970,26 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
||||||
dest="stem_prefix",
|
dest="stem_prefix",
|
||||||
help="The rule's explicit opt-out, reproducing the pre-round-10 matcher",
|
help="The rule's explicit opt-out, reproducing the pre-round-10 matcher",
|
||||||
)
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--title-covered",
|
||||||
|
action="store_true",
|
||||||
|
default=DEFAULT_TITLE_COVERED,
|
||||||
|
help=(
|
||||||
|
"read a concept whose WHOLE title the question accounts for before "
|
||||||
|
"the concepts the fusion ranked above it. ON since 2026-09-11. "
|
||||||
|
"Measured on a 2 761-concept bundle of one standard: hit@1 over six "
|
||||||
|
"questions 3 of 6 -> 6 of 6 with the known-positive holding rank 1, "
|
||||||
|
"where none of the six existing reading-side flags moved that "
|
||||||
|
"number at all. The title is read by EQUALITY, never by shared "
|
||||||
|
"prefix. See docs/2026-09-11-k3-runde16-hele-tittelen-tar-ruten.md"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--no-title-covered",
|
||||||
|
action="store_false",
|
||||||
|
dest="title_covered",
|
||||||
|
help="The rule's explicit opt-out, reproducing the pre-round-16 excerpt order",
|
||||||
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--source-quota",
|
"--source-quota",
|
||||||
type=int,
|
type=int,
|
||||||
|
|
@ -1951,6 +2058,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||||
reserve_top_rank=args.reserve_top_rank,
|
reserve_top_rank=args.reserve_top_rank,
|
||||||
rarity_weight=args.rarity_weight,
|
rarity_weight=args.rarity_weight,
|
||||||
tie_shared_rank=args.tie_shared_rank,
|
tie_shared_rank=args.tie_shared_rank,
|
||||||
|
title_covered=args.title_covered,
|
||||||
stem_prefix=args.stem_prefix,
|
stem_prefix=args.stem_prefix,
|
||||||
source_quota=args.source_quota,
|
source_quota=args.source_quota,
|
||||||
withheld_titles=args.withheld_titles,
|
withheld_titles=args.withheld_titles,
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue