fix(consume): a directory every concept id shares no longer ranks them
The first fusion signal read a concept's title together with every segment of its id. On a one-document bundle every id starts with the same directory, and since K3-19 an STS document names that directory after its own number, so a question naming the document matched every concept except the one whose title already named it. Measured on a 2 761-concept bundle, the known-positive fell from rank 1 to not delivered at the default k (13 at k = 50). `shared_id_prefix` returns the leading directory segments EVERY id shares, never the leaf, and the signal reads the id below them. Where the ids share no prefix the signal reads the same string as before. Measured on a frozen export before this commit, four forms: the chosen one gives KP rank 1 at both k with S1-S6 6/6, and K2 (12 payloads), N100/N200/N500 (15) and the five-document folder (5) byte-identical. Dropping each concept's own document directory instead took a K2 hit@8 row from rank 5 to not delivered, and is not shipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
5ce8efe437
commit
9f8a1bca42
3 changed files with 88 additions and 12 deletions
22
CHANGELOG.md
22
CHANGELOG.md
|
|
@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
|
||||
- **A directory every concept id shares no longer ranks the concepts
|
||||
(K3-20).** `okf consume`'s first fusion signal read a concept's title
|
||||
together with every segment of its id. On a one-document bundle every id
|
||||
starts with the same directory, and since K3-19 an STS document names that
|
||||
directory after its own number, so a question naming the document matched
|
||||
every concept -- except the one whose title already named it, which gained
|
||||
nothing because the overlap counts a question token once. Measured on a
|
||||
2 761-concept bundle, the known-positive fell from rank 1 to not delivered
|
||||
at the default `k` (13 at `k` = 50). `consume.shared_id_prefix` now keeps
|
||||
the leading directories EVERY id shares out of that signal: the
|
||||
known-positive is rank 1 at both `k` and S1-S6 stay 6/6.
|
||||
- **Consumer cost: payloads move only on a bundle whose ids all share a
|
||||
leading directory**, which is what a one-document build produces. Where
|
||||
they share none, the signal reads the same string as before, and the
|
||||
measured multi-document bundles are byte-identical. The old order is
|
||||
reproducible by no flag.
|
||||
- **Measured and felled:** dropping each concept's own document directory
|
||||
instead took a hit@8 row on the pinned 43-document bundle from rank 5 to
|
||||
not delivered.
|
||||
|
||||
### Changed
|
||||
|
||||
- **A NISO-STS document's own identity names its directory and titles its
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue