docs(k3): round 16, the whole title takes the route precision could not
The report, the CHANGELOG entry under [Unreleased] and the README's ranking section. No version bump and no tag: v0.8.0 was released this same day. R761, delivered ranks, before -> after: hit@1/8/50 3/6 - 5/6 - 5/6 -> 6/6 - 6/6 - 6/6 at default k and 3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6 at --k 50, with the known-positive at rank 1 on all four and the known-negative still not a hit. --no-title-covered on the new binary reproduces the old row exactly. Nothing else moves, measured on the bytes: the pinned K2 bundle and Arm B keep (1,1,1,1,1,5) with payloads byte-identical on 6 of 6 questions each, n100/n200/n500 payloads byte-identical on 5 each, the 828-file HTML corpus at 828 plans / 0 unreadable / 6 015 md with diff -rq empty, the five-document folder diff -r-identical at 26/52, and okf project still byte-equal to okf build. README's test count corrected to what this round measures: 1583 collected, 1582 passed, 1 skipped. The published 1575 was the passed count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ff06b92c35
commit
4d1b5222b7
4 changed files with 392 additions and 9 deletions
59
CHANGELOG.md
59
CHANGELOG.md
|
|
@ -7,6 +7,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- **`--title-covered` (ON by default, opt out with `--no-title-covered`): a
|
||||
question that accounts for a concept's WHOLE title reads that concept first.**
|
||||
On the 2 761-concept bundle of one standard, the answering section was
|
||||
delivered at rank 1 on **3 of 6** scored questions and **none of the reading
|
||||
side's six flags moved that number** -- the whole sweep sits at 3/6 or worse.
|
||||
Measured on that bundle, before and after: hit@1/8/50 **3/6 - 5/6 - 5/6 ->
|
||||
6/6 - 6/6 - 6/6** at default `k` and **3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6** at
|
||||
`--k 50`, with the known-positive holding rank 1 at both and the
|
||||
known-negative still not a hit. S1 4 -> 1, S5 not delivered -> 1, S6 3 -> 1.
|
||||
- **THE DEFECT IS THAT BOTH LEXICAL SIGNALS ARE UNNORMALISED COVERAGE
|
||||
COUNTS.** They measure how much of the QUESTION a candidate answers and
|
||||
nothing measures how much of the CANDIDATE the question accounts for, so a
|
||||
section titled with the question's subject alone scores what a narrower
|
||||
section titled with that subject PLUS a qualifier scores, and then loses on
|
||||
the body count. Decomposed per miss: S1 turns on `hvordan`, an interrogative
|
||||
pronoun; S5 on `hvilke` and `stilles` in a body 7x the gold's, and on
|
||||
`betonghvelv ~ betongkonstruksjoner` through the four-character stem
|
||||
`betong`; S6 on an exact tie broken by `concept_id`.
|
||||
- **A PARTITION, never a fourth RRF signal, and the arithmetic is why.** RRF
|
||||
consumes ranks alone, so with shared ranks a rule whose positive group has
|
||||
`m` members is worth `1/61 - 1/(61 + m)` -- a rule firing on ONE concept of
|
||||
2 761 is worth 0.00026 against a body gap of 0.0029. **A precise rule is
|
||||
worth LEAST under this fusion.** Measured as a signal it moves hit@1 not at
|
||||
all (3/6, both as a third and as a fourth signal); as a partition it reaches
|
||||
6/6. `lookup_hits` is the same shape for the same measured reason, and it
|
||||
still wins: the new partition lands below it, with a test and its control.
|
||||
- **By EQUALITY, never by shared prefix.** Four shared leading characters take
|
||||
the group from 1 to 6 on one question and 9 to 31 on another, with the
|
||||
answering section falling to candidate rank 6 and the known-positive to 2.
|
||||
- **TWO CANDIDATE REPAIRS WERE MEASURED AND FELLED FIRST.** Pivoted length
|
||||
normalisation of the body signal collapses at every value swept
|
||||
(b = 0.25/0.5/0.75/1.0 -> hit@8 3/6, 1/6, 1/6, 0/6, and at b = 1.0 the
|
||||
known-positive falls to rank 49): the median concept holds 22 tokens against
|
||||
a mean of 60, so length normalisation promotes thousands of tiny concepts.
|
||||
Title PRECISION as a signal reaches candidate hit@1 5/6 and takes the
|
||||
known-positive from 1 to 4 every time it does.
|
||||
- **NOTHING ELSE MOVES AND IT IS MEASURED ON THE BYTES.** The pinned K2 bundle
|
||||
keeps `(1,1,1,1,1,5)` and its 7 pin tests, Arm B keeps `(1,1,1,1,1,5)`, and
|
||||
the payloads on both are **byte-identical on 6 of 6 questions**; n100/n200/
|
||||
n500 payloads are byte-identical on 5 questions each; the 828-file HTML
|
||||
corpus still gives 828 plans, 0 unreadable and 6 015 md with `diff -rq`
|
||||
empty; the five-document folder is `diff -r`-identical at 26 concepts / 52
|
||||
md; `okf project` stays byte-equal to `okf build`. hit@k on n100/n200/n500
|
||||
is **NOT MEASURED** -- this repository holds no gold set for them, which is
|
||||
0 gold sets and not 0 hits.
|
||||
- **THIS IS THE FOURTH READING-SIDE CHANGE THAT MOVES A PAYLOAD WITH NO BUNDLE
|
||||
CHANGING.** A consumer pinned to the previous excerpt order needs
|
||||
`--no-title-covered`. The rule fires on **0 of 21** measured cells outside
|
||||
that one bundle, so "no regression" there means it never fires -- not that
|
||||
it fires harmlessly. Report:
|
||||
`docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md`.
|
||||
- `build_payload`'s signature defaults are now held equal to the consume CLI's
|
||||
argparse defaults by a test, for every same-named parameter. This is O6's
|
||||
defect in the other command: `cli.build` defaulted two flags `False` in the
|
||||
signature and `True` in argparse, and a caller reaching it as a function read
|
||||
the signature.
|
||||
|
||||
## [0.8.0] — 2026-09-10
|
||||
|
||||
### Added
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue