fix(structure): a derived reference needs a cue, not just a number shape

STRUCTURED_V1 derived `references` from every number-shaped token in a body.
A consumer measured 12 false references out of 12 on their corpus, ten of them
version numbers in titles read as document numbers. Measured again here before
changing anything, on two corpora, through `derive_document_structure` itself:

  corpus A  a consumer's normative bundles   2 561 docs -> 2 838 subjects
  corpus B  this repository's own docs/         28 docs ->   559 subjects

The reported class reproduces, and two larger ones they did not report turn up:
702 of corpus A's subjects (24.7 %) are hexadecimal fragments of UUIDs read out
of `id="..."` attributes in embedded MathML, and corpus B lifts tokens out of
escape sequences in quoted source (`\n60` -> `N60`), licence identifiers
(`AGPL-3.0` -> `3.0`) and package pins.

A derived subject is now a bundle-local link target, or a number immediately
preceded by a cue from a closed set. Positive rather than a blacklist because
the data forces it: `V221` is a genuine document number in corpus A and
`V0.3.0` is a software version in corpus B, and they are the same token shape,
so only the words in front of them can tell them apart. The cue matches at a
word boundary (a Norwegian compound ending in `-klasse` otherwise satisfies the
cue `se`, which admitted 86 class designations) and the window is NFC-normalised
so a cue survives a decomposed filesystem. Fragment-only and brace-carrying link
targets go too: neither can name a concept, so neither is a pending pointer.

After: 2 838 -> 1 279 (A) and 559 -> 72 (B). Hand-classified against the
occurrence that actually passed the gate: 30 of 30 sampled genuine on A, 60 of
60 on B. Residual known falses: 9 of B's 72, all illustrative link targets in
prose about link syntax.

The prefix-resolution rule stays open, per the order's condition: of 2 589
documents, 2 562 carry a number and 0 of those are dotted, so unique-prefix
match has no data here to be defended against.

Cost stated rather than hidden: a corpus phrasing cross-references outside this
vocabulary derives nothing and must declare `references`. A missing reference is
visible to the reader; a false one is not.

Record: docs/plan/references-cue-rule.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATyA5Lx53N1rKzwMnuMoem

src/llm_ingestion_okf/structure.py

@@ -53,6 +53,43 @@ _NUMBER_ANYWHERE = re.compile(rf"(?<![\w.])({_NUMBER})")
 # this library's job.
 _LINK = re.compile(r"\[[^\]]*\]\(([^)\s]+)\)")
 
+# What makes a number token in running prose a POINTER rather than a quantity.
+#
+# The rule is positive — a closed set of cues — and not a blacklist, because a
+# blacklist cannot work: measured 2026-08-29, `V221` and `V240` are genuine
+# document numbers in one real corpus and `v0.3.0` is a software version in
+# another, and the two are the same token shape. Nothing structural separates
+# them; only the words in front of them do. Scanning every number instead
+# produced, on those two corpora, hexadecimal fragments of UUIDs lifted from
+# inside markup attributes, unit symbols (`kN/m2` -> `M2`), table row labels,
+# formula numbers, licence identifiers (`AGPL-3.0` -> `3.0`) and package pins.
+# Each of those stands in the index as a relation this library asserted, under
+# the producer's name. A reference the reader never sees is a smaller harm
+# than one the reader cannot tell from a fact.
+#
+# The consequence is stated rather than hidden: a corpus whose cross-references
+# are phrased outside this vocabulary — another language, a house style — gets
+# NOTHING derived, and must declare `references` itself. Silence is the honest
+# failure; a guess dressed as a relation is not.
+#
+# The leading guard is a word boundary that also covers the Norwegian letters
+# `\w` handles but `[a-z]` does not. Without it `belysningsklasse C5` ends in
+# the cue `se`, which admitted 86 class designations on the corpus measured.
+_CUE = re.compile(
+    r"(?:(?<![^\W\d_])(?:"
+    r"kapittel|kapitlene|kapitler|avsnitt|punkt|pkt|vedlegg|tabell|tabellen|"
+    r"figur|figuren|krav|h\u00e5ndbok|jf|iht|nr|se|ogs\u00e5|henhold til|"
+    r"chapter|section|clause|appendix|annex|paragraph|table|figure|requirement|"
+    r"handbook|cf|see|also"
+    r")\.?|\u00a7+)\s*$",
+    re.IGNORECASE,
+)
+
+# How far back a cue is looked for. A cue sits immediately before its number,
+# so the window only has to be long enough for the longest cue plus the
+# whitespace and punctuation that may follow it.
+_CUE_WINDOW = 24
+
 # The fields this module can infer. Named as a constant because `derived` is a
 # contract with the consumer, not an implementation detail.
 DERIVABLE_FIELDS = frozenset({"title", "number", "parent", "references"})
@@ -146,11 +183,19 @@ def _parent_of(number: str) -> str | None:
 def _scan_references(body: str, offset: int, own_number: str | None) -> tuple[str, ...]:
     """Bundle-local reference subjects, ordered by first appearance.
 
+    Two kinds of pointer, and nothing else. A markdown link is an explicit one
+    the producer authored, so it is taken as written. A number in running prose
+    is taken only when a :data:`_CUE` stands immediately in front of it —
+    everything else number-shaped is a version, a measurement, a label or an
+    identifier fragment, and emitting it puts a relation this library invented
+    into the index under the producer's name.
+
     Link targets are collected first and their spans masked with spaces before
     the number scan runs, so a link to `n500.md` yields the link target once
     rather than the target plus a phantom `N500` read out of the URL. Masking
     with spaces rather than deleting keeps every later offset aligned, which is
-    what makes "first appearance" a property of the original text.
+    what makes "first appearance" a property of the original text — and it is
+    also what keeps a cue from being read across a link it does not precede.
     """
     found: list[tuple[int, str]] = []
     masked = list(body)
@@ -159,12 +204,28 @@ def _scan_references(body: str, offset: int, own_number: str | None) -> tuple[st
         start, end = match.span(1)
         for position in range(start, end):
             masked[position] = " "
-        if target.startswith(("http://", "https://", "//", "mailto:")):
+        # A fragment-only target points inside THIS document, and a target
+        # carrying a brace is a template placeholder from prose ABOUT links
+        # (`reduce_to_id_grammar` cannot emit a brace). Neither can ever
+        # resolve to another concept, so carrying them states a relation that
+        # cannot exist rather than one not dropped yet.
+        if target.startswith(("http://", "https://", "//", "mailto:", "#")):
+            continue
+        if "{" in target or "}" in target:
             continue
         found.append((offset + start, target))
-    for match in _NUMBER_ANYWHERE.finditer("".join(masked)):
+    scanned = "".join(masked)
+    for match in _NUMBER_ANYWHERE.finditer(scanned):
         number = _normalize_number(match.group(1))
-        if number != own_number:
+        if number == own_number:
+            continue
+        # NFC first: a cue carrying a Norwegian letter arrives decomposed from
+        # a macOS filesystem, and a cue that matches only one of the two forms
+        # is a rule that holds or not depending on where the file was written.
+        window = unicodedata.normalize(
+            "NFC", scanned[max(0, match.start() - _CUE_WINDOW) : match.start()]
+        )
+        if _CUE.search(window):
             found.append((offset + match.start(), number))
 
     ordered: list[str] = []