fix(consume): a directory every concept id shares no longer ranks them

The first fusion signal read a concept's title together with every segment
of its id. On a one-document bundle every id starts with the same directory,
and since K3-19 an STS document names that directory after its own number, so
a question naming the document matched every concept except the one whose
title already named it. Measured on a 2 761-concept bundle, the known-positive
fell from rank 1 to not delivered at the default k (13 at k = 50).

`shared_id_prefix` returns the leading directory segments EVERY id shares,
never the leaf, and the signal reads the id below them. Where the ids share no
prefix the signal reads the same string as before.

Measured on a frozen export before this commit, four forms: the chosen one
gives KP rank 1 at both k with S1-S6 6/6, and K2 (12 payloads), N100/N200/N500
(15) and the five-document folder (5) byte-identical. Dropping each concept's
own document directory instead took a K2 hit@8 row from rank 5 to not
delivered, and is not shipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- **A directory every concept id shares no longer ranks the concepts
+  (K3-20).** `okf consume`'s first fusion signal read a concept's title
+  together with every segment of its id. On a one-document bundle every id
+  starts with the same directory, and since K3-19 an STS document names that
+  directory after its own number, so a question naming the document matched
+  every concept -- except the one whose title already named it, which gained
+  nothing because the overlap counts a question token once. Measured on a
+  2 761-concept bundle, the known-positive fell from rank 1 to not delivered
+  at the default `k` (13 at `k` = 50). `consume.shared_id_prefix` now keeps
+  the leading directories EVERY id shares out of that signal: the
+  known-positive is rank 1 at both `k` and S1-S6 stay 6/6.
+  - **Consumer cost: payloads move only on a bundle whose ids all share a
+    leading directory**, which is what a one-document build produces. Where
+    they share none, the signal reads the same string as before, and the
+    measured multi-document bundles are byte-identical. The old order is
+    reproducible by no flag.
+  - **Measured and felled:** dropping each concept's own document directory
+    instead took a hit@8 row on the pinned 43-document bundle from rank 5 to
+    not delivered.
+
 ### Changed
 
 - **A NISO-STS document's own identity names its directory and titles its

CLAUDE.md

@@ -137,13 +137,19 @@ one boundary rule:
   sections on R761 carry a point, **1 807** are written (2 have no `<p>`, 217
   carry `: ` and PyYAML refused exactly those frontmatters), none invented.
   SS 4.1 sets no length, so the one-paragraph limit is ours. **The directory
-  name reaches the RANKING:** `consume`'s signal 1 reads the concept id's
-  segments, so on a one-document bundle every concept now carries the
-  document's own name, and a question naming the document matches all of
-  them -- measured, the known-positive went rank 1 -> not delivered at the
-  default `k` (13 at k=50) with S1-S6 unmoved, and renaming only the
-  directory back restores rank 1. Operator question, not a patch. Report:
-  `docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`. The
+  name reached the RANKING, and K3-20 closed it in `consume`:** signal 1 read
+  every id segment, so on a one-document bundle every concept carried the
+  document's own name, and a question naming the document matched all of them
+  -- measured, the known-positive went rank 1 -> not delivered at the default
+  `k` (13 at k=50) with S1-S6 unmoved. `consume.shared_id_prefix` now keeps
+  the leading directories EVERY id shares out of that signal: KP rank 1 at
+  both `k`, S1-S6 6/6, and K2 (12 payloads), N100/N200/N500 (15) and the
+  five-document folder (5) byte-identical, because ids that share no prefix
+  read exactly as before. Dropping each concept's OWN document directory
+  instead was measured and felled -- a K2 hit@8 row went rank 5 -> not
+  delivered. Reports:
+  `docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md` and
+  `docs/2026-09-11-k3-runde20-delt-katalog-og-arvet-kontekst.md`. The
   registries are COUPLED: a row in
   `_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
   every proposal for the type, two layers away from the extractor.

src/llm_ingestion_okf/consume.py

@@ -1230,8 +1230,10 @@ def title_covered_hits(concepts: Sequence["Concept"], question: str) -> tuple[st
 
     Reads `title` alone and not the concept id: an id segment is this
     library's reduction of a title, so counting it would let the same words
-    qualify a concept twice, and the document uuid that the id carries on a
-    single-document bundle is in no question ever asked.
+    qualify a concept twice. The directory naming a single document is not
+    always a uuid no question carries -- an STS document names it with its own
+    number (K3-19) -- which is why the fusion's first signal reads the id
+    through `shared_id_prefix`.
     """
     question_tokens = set(normalise(question))
     if not question_tokens:
@@ -1246,6 +1248,48 @@ def title_covered_hits(concepts: Sequence["Concept"], question: str) -> tuple[st
     )
 
 
+def shared_id_prefix(concept_ids: Sequence[str]) -> int:
+    """How many leading DIRECTORY segments every concept id in the bundle shares.
+
+    The first fusion signal reads a concept's title together with the segments
+    of its id, and on a bundle holding ONE document every id starts with the
+    same directory. A segment every concept carries separates nothing -- the
+    claim `concept_scores` already makes for a whole signal that scores every
+    concept alike -- but here it is not even a constant: the overlap counts
+    each question token once, so when the question names that directory, a
+    concept whose TITLE already names it gains nothing from the id while every
+    sibling gains the token. The one concept distinguished by naming the
+    document loses exactly that distinction, and a concept answering nothing
+    but the directory stops being a guess.
+
+    MEASURED 2026-09-11 on a 2 761-concept, one-document bundle whose directory
+    is the document's own number (K3-19): the known-positive question names the
+    document and fell from rank 1 to not delivered at the default `k` (13 at
+    `k` = 50). With the shared prefix unread it is rank 1 at both, and the six
+    scored questions keep rank 1.
+
+    **Directories only, never the leaf, and only the prefix EVERY id shares.**
+    Where the ids share no leading directory -- the measured multi-document
+    bundles, whose documents sit in different top-level directories -- the
+    signal reads what it read before, byte for byte. That is a measurement and
+    not a caution: dropping each concept's own document directory instead,
+    shared by every concept IN that document, was measured on the pinned
+    43-document bundle and took a hit@8 row from rank 5 to not delivered.
+    Across documents that directory carries information.
+
+    `searchable_text` still reads the whole id, so the stem vocabulary does not
+    move, and under `--rarity-weight` a token every concept carries weighs
+    `log(1) == 0` there -- the same answer this rule gives.
+    """
+    directories = [concept_id.split("/")[:-1] for concept_id in concept_ids]
+    shared = 0
+    for segments in zip(*directories):
+        if any(segment != segments[0] for segment in segments):
+            break
+        shared += 1
+    return shared
+
+
 def concept_scores(
     concepts: Sequence[Concept],
     question: str,
@@ -1261,8 +1305,9 @@ def concept_scores(
     """Every concept, ordered best first, fused from three signals by RRF.
 
     The signals: (1) the question against the concept's title and the segments
-    of its id, (2) the question against the body, (3) the stage-one score of the
-    document the concept belongs to.
+    of its id below the directories every id shares (`shared_id_prefix`), (2)
+    the question against the body, (3) the stage-one score of the document the
+    concept belongs to.
 
     **Ties break lexicographically by `concept_id`, at both the per-signal sort
     and the fused sort.** Declared rather than inherited from dict insertion
@@ -1310,8 +1355,11 @@ def concept_scores(
     """
     question_tokens = normalise(question)
     bridge = cost_vocabulary and question_uses_cost_vocabulary(question)
+    # The id BELOW the directories every concept shares: a segment every
+    # concept carries separates nothing. See `shared_id_prefix`.
+    shared = shared_id_prefix([concept.concept_id for concept in concepts])
     titles = {
-        concept.concept_id: f"{concept.title} {concept.concept_id.replace('/', ' ')}"
+        concept.concept_id: f"{concept.title} {' '.join(concept.concept_id.split('/')[shared:])}"
         for concept in concepts
     }
     signals: list[dict[str, float]] = [