fix(consume,propose): hold an identifier number as one token, give an orphaned heading's name to its table

Two consumer-reported defects, one rebuild.

The pre-pass could not see a requirement number: `_TOKEN_SPLIT_RE` split
`10.2-2` into digit runs and `MIN_TOKEN_LENGTH` removed them, so a question
naming a requirement reached the ranker carrying only the word every concept
in a standards bundle carries. Measured on three real bundles (446, 1133 and
270 concepts), the named requirement was withheld `below_k` in three of three.
Numeric groups joined by `.` or `-` are now held together, dash variants fold
to the ASCII hyphen, and the noise floor is unchanged. The gold moves from
160 to 96, 143 to 9 and 100 to 35 -- a large move, and NOT a delivery: it is
still `below_k`, because `_overlap` is a count and an exact requirement number
is worth no more than a common verb. That weighting is a separate decision.

The rule was narrowed by a measurement: a version that joined alphanumeric
groups swallowed a document slug whole and cost a hit@8 row. An equality-only
variant was measured on all three bundles and falsified -- better on one,
worse on two.

The orphan gate destroyed a heading's name: a table opening directly below a
heading left that heading with an empty body, the orphan check dropped it, and
the surviving table block kept the mechanical `Tabell linje <n>`. A table that
orphans its heading now takes that heading's title and section number.
Conditioned on the drop, on adjacency, and carrying both members -- each of
the three measured or mutation-tested.

One K2 rebuild for both, from a frozen source tree: 629 concepts, `39 + 4 = 43
= N`, 2 of 629 ids moved and both moved BACK to the names the 2026-09-03
bundle carried, 1106 of 1108 files identical to it. New ref
sha256-tree:2f82fcfea91c3bd3f8ef7147f80cd613227d3ca7975c41d88810233f3f79ab4b
-- c26eed6a... is superseded. The regression the previous session measured is
closed: candidate rank 19 -> 10, and the delivering command is now
`--cost-vocabulary --k 12` inside the default budget at 58 907 o200k against
65 912 before. The specific question is unmoved at rank 1.

The tokeniser alone leaves the K2 control question byte-identical, measured
with the bundle held fixed and both published byte counts reproduced.

Report: docs/2026-09-08-kravnummer-tokenisering.md. 8 new tests, red first;
6 mutations, 6 red, one of them only after the survivor was read as code and
a missing fixture was added. Suite 1287 -> 1295.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

tools/okf_consume.py

@@ -522,9 +522,27 @@ MIN_SHARED_PREFIX = 4
 
 _TOKEN_SPLIT_RE = re.compile(r"[^0-9a-zà-öø-ÿ]+")
 
+#: Every dash a source spells an identifier's separator with, folded to the
+#: ASCII hyphen. NFC folds NONE of them, so `10.2—2` from a document viewer and
+#: `10.2-2` from a person typing the question are two different tokens until
+#: this table runs. The set is the Unicode dash block plus the minus sign.
+_DASH_TO_HYPHEN = str.maketrans(dict.fromkeys("‐‑‒–—―−", "-"))
+
+#: An identifier: NUMERIC groups joined by `.` or `-`, with an optional letter
+#: prefix that touches its digits without a separator (`R610.4`).
+#:
+#: THE LETTERS ARE THE POINT, and this pattern was narrowed by a measurement
+#: rather than written this way. A rule that joined alphanumeric groups across
+#: a separator swallowed a whole document slug -- `...bilag-3-6-premissrapport-
+#: akustikk` became ONE token because `3-6` sits inside it -- and that
+#: document's score for a question naming its subject fell from 0.735 to 0.0,
+#: taking a hit@8 row with it. Only digits may stand on either side of a
+#: separator, so a hyphenated word keeps its words.
+_IDENTIFIER_RE = re.compile(r"[a-zà-öø-ÿ]*[0-9]+(?:[.-][0-9]+)+")
+
 
 def normalise(text: str) -> tuple[str, ...]:
-    """Text as comparable tokens: NFC first, then casefold, then split.
+    """Text as comparable tokens: NFC, casefold, dash-fold, then split.
 
     NFC FIRST is load-bearing and not tidiness. macOS hands filenames over
     decomposed, so `å` arrives as `a` + U+030A; the combining ring is not a word
@@ -532,9 +550,30 @@ def normalise(text: str) -> tuple[str, ...]:
     the term is silently lost. `æ` and `ø` have no canonical decomposition, so a
     test built on either passes while the bug is live -- which is why the
     known-positive for this function uses `å`.
+
+    IDENTIFIERS SURVIVE THE SPLIT. Splitting on every non-alphanumeric turns a
+    requirement number into digit runs, and `MIN_TOKEN_LENGTH` then removes
+    them: `Krav 10.2—2` reached the ranker as `krav` alone, a word every concept
+    in a standards bundle carries, so the ranking became a corpus-wide tie and
+    the named requirement was withheld `below_k` (measured 2026-09-08 on three
+    bundles). The floor stays -- a bare `10` matches every page number in a
+    corpus -- and the identifier is exempted from it rather than the floor
+    lowered for everyone.
     """
-    folded = unicodedata.normalize("NFC", text).casefold()
-    return tuple(token for token in _TOKEN_SPLIT_RE.split(folded) if len(token) >= MIN_TOKEN_LENGTH)
+    folded = unicodedata.normalize("NFC", text).casefold().translate(_DASH_TO_HYPHEN)
+    tokens: list[str] = []
+    position = 0
+    for match in _IDENTIFIER_RE.finditer(folded):
+        tokens.extend(_split(folded[position : match.start()]))
+        tokens.append(match.group())
+        position = match.end()
+    tokens.extend(_split(folded[position:]))
+    return tuple(tokens)
+
+
+def _split(folded: str) -> list[str]:
+    """The generic split, on text already folded by `normalise`."""
+    return [token for token in _TOKEN_SPLIT_RE.split(folded) if len(token) >= MIN_TOKEN_LENGTH]
 
 
 def tokens_match(left: str, right: str) -> bool: