docs(k3): round 16, the whole title takes the route precision could not
The report, the CHANGELOG entry under [Unreleased] and the README's ranking section. No version bump and no tag: v0.8.0 was released this same day. R761, delivered ranks, before -> after: hit@1/8/50 3/6 - 5/6 - 5/6 -> 6/6 - 6/6 - 6/6 at default k and 3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6 at --k 50, with the known-positive at rank 1 on all four and the known-negative still not a hit. --no-title-covered on the new binary reproduces the old row exactly. Nothing else moves, measured on the bytes: the pinned K2 bundle and Arm B keep (1,1,1,1,1,5) with payloads byte-identical on 6 of 6 questions each, n100/n200/n500 payloads byte-identical on 5 each, the 828-file HTML corpus at 828 plans / 0 unreadable / 6 015 md with diff -rq empty, the five-document folder diff -r-identical at 26/52, and okf project still byte-equal to okf build. README's test count corrected to what this round measures: 1583 collected, 1582 passed, 1 skipped. The published 1575 was the passed count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ff06b92c35
commit
4d1b5222b7
4 changed files with 392 additions and 9 deletions
52
README.md
52
README.md
|
|
@ -408,6 +408,46 @@ number that had stopped being true in the same commit. A flag's "off by
|
|||
measurement" is a measurement of a *configuration*, not a property of the flag.
|
||||
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
|
||||
|
||||
`--title-covered` is **on** by default since 2026-09-10 (opt out with
|
||||
`--no-title-covered`), and it is the fourth widening here that alters a payload
|
||||
with no bundle changing. A concept whose EVERY title token is a token of the
|
||||
question is read before the concepts the fusion ranked above it. The defect it
|
||||
repairs is that both lexical signals are unnormalised COVERAGE COUNTS: they
|
||||
measure how much of the question a candidate answers, and nothing measures how
|
||||
much of the CANDIDATE the question accounts for, so a section titled with the
|
||||
question's subject alone scores exactly what a narrower section titled with that
|
||||
subject plus a qualifier scores — and then loses on the body count, because a
|
||||
longer title and a longer body can only reach more of the question. Measured on
|
||||
a 2 761-concept bundle of one standard, where **none of the six flags above
|
||||
moved the number at all**: hit@1/8/50 **3 of 6 · 5 of 6 · 5 of 6 → 6 of 6 · 6 of
|
||||
6 · 6 of 6** at default `k`, the same 6 of 6 at `--k 50`, the known-positive
|
||||
holding rank 1 at both and the known-negative still not a hit. The three
|
||||
recovered rows go 4 → 1, not-delivered → 1 and 3 → 1.
|
||||
|
||||
It is a PARTITION and not a fourth RRF signal, and the arithmetic is the
|
||||
reason: RRF consumes ranks alone, so with shared ranks a rule whose positive
|
||||
group has `m` members is worth `1/61 − 1/(61 + m)`, and a rule firing on ONE
|
||||
concept of 2 761 is worth 0.00026 against a body-signal gap of 0.0029 — **a
|
||||
precise rule is worth least under this fusion, backwards from what precision is
|
||||
for**. Measured as a signal the same predicate moves hit@1 not at all; as a
|
||||
partition it reaches 6 of 6. `lookup_hits` is the same shape for the same
|
||||
measured reason, and it still wins: this partition lands below it. The title is
|
||||
read by EQUALITY, never by shared prefix — four shared leading characters take
|
||||
the group from 1 to 6 on one question and 9 to 31 on another, with the answering
|
||||
section falling to candidate rank 6 and the known-positive to 2.
|
||||
|
||||
Two candidate repairs were measured and felled first. Pivoted length
|
||||
normalisation of the body signal collapses at every value swept (b = 0.25, 0.5,
|
||||
0.75, 1.0 give hit@8 3, 1, 1 and 0 of 6, and at 1.0 the known-positive falls to
|
||||
rank 49), because the median concept holds 22 tokens against a mean of 60 — so
|
||||
length normalisation promotes thousands of tiny concepts over the section that
|
||||
treats the subject. Title *precision* as a signal reaches candidate hit@1 5 of 6
|
||||
and takes the known-positive from rank 1 to 4 every time it does. The reach of
|
||||
what shipped is narrow and stated as such: it fires on 6 of 8 questions on that
|
||||
bundle and on **0 of 21** measured cells across the pinned K2 bundle, Arm B and
|
||||
the three N bundles, whose payloads are byte-identical either way.
|
||||
`docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md`.
|
||||
|
||||
It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
|
||||
`sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a
|
||||
validated known-positive), `denominators`, `excerpts` and `withheld` — and every
|
||||
|
|
@ -764,12 +804,14 @@ From a checkout, the test suite runs with:
|
|||
.venv/bin/python -m pytest
|
||||
```
|
||||
|
||||
The suite is the verification surface for everything above: **1575 tests**,
|
||||
run on 2026-09-10 against the `v0.8.0` release commit with the `[extract]`
|
||||
extra installed.
|
||||
The suite is the verification surface for everything above: **1583 tests
|
||||
collected, 1582 passed and 1 skipped**, run on 2026-09-10 with the `[extract]`
|
||||
extra installed. Both numbers are given because they are two measurements: the
|
||||
figure published before this round was the PASSED count, and `pytest
|
||||
--collect-only -q` reported one more.
|
||||
(The figure stood at 596 until 2026-09-09 — measured 2026-08-21 and never
|
||||
updated as the suite grew — and at 1515 until this release: a count is a
|
||||
measurement with a date on it.)
|
||||
updated as the suite grew — at 1515 until the `v0.8.0` release and at 1575
|
||||
through it: a count is a measurement with a date on it.)
|
||||
Without the extra the same suite skips the tests covering the parser path;
|
||||
that split was last counted on 2026-08-21 as 589 passed and 7 skipped and has
|
||||
**not** been re-measured since. The tests holding the fail-fast rejection for
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue