feat(consume): weight a lexical hit by its rarity, off by default and measured

O2b asked whether a requirement number can be made worth more than a common
word by weighting each hit with the token's rarity in the bundle, with no
hand-set constant and no declared token class. It can, on one of the three
bundles, and the two it cannot are decomposed rather than guessed.

The rule is log(N/df) over the concepts' own tokens, counted with the same
four-character prefix rule a hit is scored with. It enters the RANKING and
never the GATE: `lexical` stays a count, because `krav` weighs exactly 0 on
all three bundles and a weighted gate would drop every concept matching only
that word -- which is the gate 54a0bc2 falsified for other reasons. One df
table per bundle reaches every stage that scores the question against text,
including the document prior. One pass, 0.241 s over 1 133 concepts.

Measured on four corpora, before and after, with every published figure
reproduced first: gold fused rank 96 -> 103, 9 -> 8 (withheld -> DELIVERED at
rank 8) and 35 -> 35; K2's priced sheet candidate rank 10 -> 2 with the cost
vocabulary and 251 -> 78 without; Q-good unmoved at rank 1; hit@8 5 of 6 with
every rank identical; the S7 control payload byte-identical on the default
command.

DEFAULT OFF, decided by the number and not by taste: it does not win on all
four, because N100's gold loses seven rank positions. Off means the bytes that
were already published, and that is measured -- 8 of 8 payload digests
identical against a frozen copy of 56c1205 built with git archive.

Two limits, both someone else's mechanism and both named: MIN_SHARED_PREFIX=4
makes a unique identifier read as 135-of-446 common on N100, so the weight
correctly ranks a common adjective above the exact requirement number; and RRF
consumes RANKS only, so on N500 -- where the gold already leads the one signal
that can see the identifier, and the other two cannot see it at all -- no
weighting inside a signal can move anything.

Consumption-side only, so no rebuild: the K2 bundle ref 2f82fcfe... stands.

Report: docs/2026-09-08-sjeldenhetsvekt.md. 13 new tests, red first; 8
mutations, 8 red, two of them only after the survivors were read as code -- one
exposed a fixture that put the identifier where the real corpus does not, and
the corrected fixture is what found the RRF limit. Suite 1295 -> 1308.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CLAUDE.md

@@ -268,8 +268,8 @@ and fixtures, never code.
   measurement behind it, including the control that FAILED, is
   `docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this
   repository's own choice** — the contract binds a payload, not a retrieval
-  algorithm (§ 10) — and it has TWO optional widenings, both **off by default**
-  and both keeping the default payload byte-identical. `--cost-vocabulary`: a
+  algorithm (§ 10) — and it has THREE optional widenings, all **off by default**
+  and all keeping the default payload byte-identical. `--cost-vocabulary`: a
   declared cost/price/quantity vocabulary family that bridges a question and a
   document naming money with different words, gated on the QUESTION carrying
   such a term, so a question without one is byte-identical either way. It moves
@@ -286,6 +286,18 @@ and fixtures, never code.
   the caller's decision, because deriving a limit from the corpus was measured
   and falsified — two defensible derivations, 49x apart, one of them breaking
   the known-positive. `docs/2026-09-08-blindsone-laas2-budsjett-k2.md`.
+  `--rarity-weight` is the third: each lexical hit weighs `log(N/df)` over the
+  bundle's own concepts instead of 1, so an identifier is not worth what a
+  common verb is worth. It enters the RANKING and never the GATE — `lexical`
+  stays a count, because a word every concept carries weighs exactly 0 and a
+  weighted gate is what `54a0bc2` falsified. Off by default BY MEASUREMENT: it
+  delivers one of three requirement lookups and takes a priced sheet from
+  candidate rank 10 to 2, leaves one gold unmoved and costs another seven rank
+  positions. Two limits are decomposed rather than guessed, and both are
+  someone else's mechanism: `MIN_SHARED_PREFIX = 4` makes a unique identifier
+  read as 135-of-446 common, and RRF consumes RANKS, so no weighting inside a
+  signal can move a gold that already leads it.
+  `docs/2026-09-08-sjeldenhetsvekt.md`.
 
 ## Workflow