open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(consume): a covered title rises only past titles that answer no more of the question

Browse source 
Round 16's partition read every concept whose WHOLE title the question
accounts for before everything the fusion ranked above it. That is a claim
about the covered title's PRECISION, and it overrode the fusion even against
a title answering MORE of the question. Measured on a 26-concept bundle of
five documents: the question names a section by three title tokens and holds
a neighbour's whole one-token title (1 of 9 question tokens); the fusion put
the named section at rank 1, the partition moved the neighbour over it.

A covered concept now RISES through the fusion's order and stops beneath the
first concept whose title answers more question tokens, by equality, than it
holds, or beneath a covered concept the fusion put above it. With nothing
above it answering more it reaches the top exactly as before. Same flag
(--title-covered / --no-title-covered), no new parameter, no new constant.

Measured before this commit, delivered ranks from build_payload:
- known-negative: rank 2 -> 1; the payload equals --no-title-covered's
- three own probes on that bundle: 1, 1, 1 (unchanged from round 16)
- R761 XML, 2 761 concepts: hit@1/8/50 6/6 - 6/6 - 6/6 at default k and at
  --k 50, KP rank 1; 8 of 8 payloads byte-identical to 7cca9e0 at BOTH k
- payloads byte-identical to 7cca9e0: K2 pinned 6/6, Arm B 6/6, N100/N200/
  N500 15/15; tests/test_default_bundle_pin.py 7 passed, file untouched
- candidates measured beside it: min title length (R761 hit@1 3/6), share of
  the question (holds only for 1/9 < s <= 1/6), order inside the group
  (group of one: no effect), stop list (no title involved is one)

Suite on the staged set: 1600 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 01:32:49 +02:00
commit e15de71aa4
3 changed files with 99 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

36
CHANGELOG.md
View file

@ -49,6 +49,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
bytes are unchanged -- measured, both tracked bundles byte-identical before
and after on the same interpreter.
### Fixed
- **`--title-covered` no longer lifts a short title over a title that answers
more of the question.** 0.8.1's partition read every concept whose WHOLE
title the question accounts for before everything the fusion ranked above
it. That is a claim about the covered title's PRECISION, and it overrode the
fusion even against a title answering MORE of the question: measured on a
26-concept bundle of five tender documents, a question naming a section by
three of its title tokens also held a neighbour's whole one-token title, and
the neighbour took rank 1 from the section the question names. A covered
concept now RISES through the fusion's order and stops beneath the first
concept whose title answers more question tokens, by equality, than it
holds. Same flag, no new parameter, no new constant.
- **What the rule was built for does not move.** On the 2 761-concept bundle
of one standard no covered concept had such a title above it, so all 8
payloads are byte-identical to 0.8.1's at default `k` AND at `--k 50`;
hit@1/8/50 stays 6/6 · 6/6 · 6/6 with the known-positive at rank 1.
- **Where the rule never fires nothing moves either**, measured on the bytes:
the pinned K2 bundle 6 of 6 payloads identical, Arm B 6 of 6, the three N
bundles 15 of 15.
- **Four other repairs were measured and not taken**: a minimum title length
sold hit@1 back to 3 of 6; a share of the question holds only in a band
set by the question's word count (1/9 < s <= 1/6); an order inside the
covered group cannot act on a group of one; a closed stop list touches no
title involved and would be a new vocabulary to maintain.
- **0.8.1's unbounded order is reproducible by no flag.** It differs from
this one only where a covered concept has such a title above it -- 1 of
the 4 questions measured on that bundle, 0 of 8 on the standard, 0 of 27
elsewhere. `--no-title-covered` still reproduces the pre-0.8.1 order.
- **One constructed variant still reads the short title first, and no rule
reading titles alone separates it.** A shortened form of the same question
shares ONE token with each of the two titles; a form that blocks on any
token the covered title lacks fixes it and takes one of the standard's
scored questions, the same shape with the opposite answer, from rank 1 to
3. Report: `docs/2026-09-11-k3-runde17-dekningen-stopper-ved-en-bredere-tittel.md`.
## [0.8.1] — 2026-09-10
### Added

16
README.md
View file

@ -454,6 +454,22 @@ bundle and on **0 of 21** measured cells across the pinned K2 bundle, Arm B and
the three N bundles, whose payloads are byte-identical either way.
`docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md`.
**Since round 17 the partition is bounded by RECALL**, under the same flag. A
covered title states its PRECISION — it says nothing the question did not ask —
and round 16 let that claim override the fusion even against a title answering
MORE of the question: on a 26-concept bundle of five tender documents, a
question naming a section by three of its title tokens also held a neighbour's
whole one-token title, and the neighbour took rank 1. A covered concept now
rises through the fusion's order and stops beneath the first concept whose
title answers more question tokens, by equality, than it holds. On the
standard's bundle nothing stood above a covered concept answering more, so its
8 payloads are byte-identical at both `k`; the 27 payloads where the rule never
fires are byte-identical too. A minimum title length, a share of the question,
an order inside the group and a stop list were measured beside it and not
taken. 0.8.1's unbounded order is reproducible by no flag;
`--no-title-covered` still gives the pre-0.8.1 order.
`docs/2026-09-11-k3-runde17-dekningen-stopper-ved-en-bredere-tittel.md`.
It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
`sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a
validated known-positive), `denominators`, `excerpts` and `withheld` — and every

53
src/llm_ingestion_okf/consume.py
View file

@ -1185,6 +1185,23 @@ DEFAULT_STEM_PREFIX = True
#: LIKE `--tie-shared-rank` AND `--stem-prefix`, THIS MOVES A PAYLOAD WITH NO
#: BUNDLE CHANGING. A consumer pinned to the previous excerpt order needs
#: `--no-title-covered`.
#:
#: ROUND 17 BOUNDS THE PARTITION BY RECALL, under the same flag. The partition
#: states the covered title's PRECISION -- it says nothing the question did not
#: ask -- and round 16 let that claim override the fusion even against a title
#: answering MORE of the question. Measured 2026-09-11 on a 26-concept bundle
#: of five tender documents: a question naming a section by three of its title
#: tokens also held a neighbour's whole one-token title, and the partition
#: moved that neighbour from fusion rank 2 to rank 1 over the section the
#: question names. A covered concept now RISES only past concepts whose titles
#: answer no more question tokens, by equality, than it holds. On the
#: 2 761-concept bundle no covered concept had such a title above it, so all
#: eight payloads there are byte-identical to round 16's. Four other repairs
#: were measured against it: a minimum title length (hit@1 there back to 3 of
#: 6), a share of the question (holds only in a band set by the question's
#: word count, 1/9 < s <= 1/6), an order inside the group (the known-negative's
#: group is ONE) and a stop list (no title involved is a function word). See
#: `docs/2026-09-11-k3-runde17-dekningen-stopper-ved-en-bredere-tittel.md`.
DEFAULT_TITLE_COVERED = True
@ -1380,11 +1397,33 @@ def concept_scores(
# `DEFAULT_TITLE_COVERED` for the arithmetic that rules a signal out,
# and `tests/test_title_covered.py` for the mechanism on a fixture.
#
# STABLE: both parts keep the order the fusion gave them, so nothing
# here depends on dict order.
ranked_ids = [key for key in ranked_ids if key in covered] + [
key for key in ranked_ids if key not in covered
]
# BOUNDED BY RECALL since round 17: a covered concept RISES through the
# fusion's order and stops beneath the first concept whose title
# answers MORE question tokens, by equality, than the covered title
# holds -- or beneath a covered concept the fusion put above it. With
# nothing above it answering more, it reaches the top exactly where
# round 16's plain partition put it. `tests/test_title_covered_rise.py`
# holds the known-negative that bound exists for.
#
# STABLE: the covered concepts keep the order the fusion gave them, and
# so does everything else, so nothing here depends on dict order.
asked = frozenset(question_tokens)
answered = {
concept_id: len(set(normalise(by_id[concept_id].title)) & asked)
for concept_id in ranked_ids
}
risen: list[str] = []
for concept_id in ranked_ids:
stop = len(risen)
if concept_id in covered:
while (
stop
and risen[stop - 1] not in covered
and answered[risen[stop - 1]] <= answered[concept_id]
):
stop -= 1
risen.insert(stop, concept_id)
ranked_ids = risen
named = set(lookup_hits(concepts, question)) if lookup else set()
if named:
# THE LOOKUP LANDS BEFORE THE FUSION'S OUTPUT IS READ, and it is a
@ -1976,7 +2015,9 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
default=DEFAULT_TITLE_COVERED,
help=(
"read a concept whose WHOLE title the question accounts for before "
"the concepts the fusion ranked above it. ON since 2026-09-10. "
"the concepts the fusion ranked above it, stopping beneath any whose "
"title answers MORE question tokens than it holds (round 17). ON "
"since 2026-09-10. "
"Measured on a 2 761-concept bundle of one standard: hit@1 over six "
"questions 3 of 6 -> 6 of 6 with the known-positive holding rank 1, "
"where none of the six existing reading-side flags moved that "