feat(consume): weight a lexical hit by its rarity, off by default and measured

O2b asked whether a requirement number can be made worth more than a common
word by weighting each hit with the token's rarity in the bundle, with no
hand-set constant and no declared token class. It can, on one of the three
bundles, and the two it cannot are decomposed rather than guessed.

The rule is log(N/df) over the concepts' own tokens, counted with the same
four-character prefix rule a hit is scored with. It enters the RANKING and
never the GATE: `lexical` stays a count, because `krav` weighs exactly 0 on
all three bundles and a weighted gate would drop every concept matching only
that word -- which is the gate 54a0bc2 falsified for other reasons. One df
table per bundle reaches every stage that scores the question against text,
including the document prior. One pass, 0.241 s over 1 133 concepts.

Measured on four corpora, before and after, with every published figure
reproduced first: gold fused rank 96 -> 103, 9 -> 8 (withheld -> DELIVERED at
rank 8) and 35 -> 35; K2's priced sheet candidate rank 10 -> 2 with the cost
vocabulary and 251 -> 78 without; Q-good unmoved at rank 1; hit@8 5 of 6 with
every rank identical; the S7 control payload byte-identical on the default
command.

DEFAULT OFF, decided by the number and not by taste: it does not win on all
four, because N100's gold loses seven rank positions. Off means the bytes that
were already published, and that is measured -- 8 of 8 payload digests
identical against a frozen copy of 56c1205 built with git archive.

Two limits, both someone else's mechanism and both named: MIN_SHARED_PREFIX=4
makes a unique identifier read as 135-of-446 common on N100, so the weight
correctly ranks a common adjective above the exact requirement number; and RRF
consumes RANKS only, so on N500 -- where the gold already leads the one signal
that can see the identifier, and the other two cannot see it at all -- no
weighting inside a signal can move anything.

Consumption-side only, so no rebuild: the K2 bundle ref 2f82fcfe... stands.

Report: docs/2026-09-08-sjeldenhetsvekt.md. 13 new tests, red first; 8
mutations, 8 red, two of them only after the survivors were read as code -- one
exposed a fixture that put the identifier where the real corpus does not, and
the corrected fixture is what found the RRF limit. Suite 1295 -> 1308.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

README.md

@@ -121,8 +121,8 @@ cuts them to a bounded set and emits one payload. It decides nothing about the
 question; the skill that reads the payload does the judgement. It calls no
 model, opens no socket, imports nothing outside the standard library and this
 package, and takes no clock: the same bundle bytes and the same
-`(question, k, limit, cost_vocabulary, reserve_top_rank)` produce
-byte-identical output.
+`(question, k, limit, cost_vocabulary, reserve_top_rank, rarity_weight)`
+produce byte-identical output.
 
 `--cost-vocabulary` is off by default and widens one question class: it lets a
 declared list of cost/price/quantity terms bridge a question and a document that
@@ -141,6 +141,17 @@ pre-exclusion, never before — and the payload then declares
 24 measured combinations, both of them that eviction:
 `docs/2026-09-08-blindsone-laas2-budsjett-k2.md`.
 
+`--rarity-weight` is off by default and weights each lexical hit by
+`log(N/df)` over the bundle's own concepts instead of counting it as one, so a
+requirement number is not worth what a common verb is worth. The default being
+off is a measurement rather than a preference: on four corpora it took one gold
+concept from withheld to delivered and a priced sheet from candidate rank 10 to
+2, left one gold rank unmoved, and cost another seven rank positions — because
+the four-character prefix matcher makes a unique identifier read as
+135-of-446 common on that bundle. Where it cannot help is decomposed rather
+than guessed: RRF fuses RANKS, so a weight moves nothing on a signal the gold
+already leads. `docs/2026-09-08-sjeldenhetsvekt.md`.
+
 It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
 `sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a
 validated known-positive), `denominators`, `excerpts` and `withheld` — and every