feat(consume): --follow-parent carries the enclosing section's text, from room the cut left

K3-21 B. The second form of `parent`: `okf consume --follow-parent`
(`consume.attach_parent_text`) puts the enclosing concept's text inside an
excerpt's `parent`, with that concept's own `sha256` so a claim resting on it
is cited as that concept. It runs AFTER the cut, on the room the cut left, in
rank order, so the delivered set, its order, the withheld list and the
denominators are the same with the flag as without it -- inherited text
cannot displace an excerpt, the mechanism a consumer measured when copied-in
ancestor text pushed the right section to withheld place 504 and 1 069. A
text that does not fit is cut to the longest prefix that does and marked
`truncated`; a parent the payload already holds, or one a higher-ranked
excerpt already carried, travels once. OFF; the defaults are chosen on the
measurement that follows this commit.

`delivered_text` is the one normalisation an excerpt's `text` and a parent's
share. Contract SS 8 point 6 gains the MAY; the template tells the reader
what `text`, `sha256` and `truncated` mean. README and CLAUDE.md name the
flag.

Moved on purpose: the SS 7.4 known-positive again (14 455 / 14 083 / 372 ->
14 721 / 14 346 / 375), and `skills/okf-consume/` regenerated with it.
`tests/test_parent_text.py::test_no_room_means_no_text_and_no_lost_excerpt`
changed from its red form: it asked through `build_payload` at `limit ==
spent`, where the knapsack's 500 B buckets admit nothing at all
(`budget_admits_nothing`); it now holds the rule at `attach_parent_text`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CLAUDE.md

@@ -635,7 +635,12 @@ and fixtures, never code.
   absolute form), appended AFTER structure derivation -- read as body text it
   became a second, unresolved `references` edge -- and screened on its own.
   `okf check`'s seventeenth rule, `parent_unfollowable`, holds the form.
-  Whether the build default moves is K3-21 B's measurement, not this line's.
+  `okf consume --follow-parent` (K3-21 B) is the second form: `parent` also
+  carries the enclosing concept's `text` with that concept's own `sha256`,
+  placed AFTER the cut from the room it left, in rank order, so the delivered
+  set is the same with it as without it; a text that does not fit is cut to
+  the longest prefix that does and marked `truncated`. Whether either default
+  moves is K3-21 B's measurement, not this line's.
   **The index renders every such parent
   UNRESOLVED** (`parent: p1?`): `structure` reads `parent` as a document
   NUMBER, and a segment id answers to none -- the one key with two meanings

README.md

@@ -174,8 +174,10 @@ was off because `okf consume` did not read `parent`. Since K3-21 it does: an
 excerpt carries `parent` as the enclosing concept's `concept_id` and `title`
 (resolved inside the concept's own document, never the raw segment id), and a
 heading-only body gains one line, `Enclosing section: [title](/path)`, in the
-bundle-relative form SPEC § 6.1 recommends. Whether that moves the default is
+bundle-relative form SPEC § 6.1 recommends. `okf consume --follow-parent`
-measured separately. One cost is known and not repaired: the index projects
+carries the enclosing section's text inside `parent` as well, with that
+concept's own `sha256`, and only from the room the cut left — so it never
+displaces an excerpt. Whether either default moves is measured separately. One cost is known and not repaired: the index projects
 `parent` as a document number, so a segment id always renders there as
 unresolved (`parent: p1?`), even though the concept it names is in the bundle.
 

docs/consumption-contract.md

@@ -220,7 +220,10 @@ are permitted; the checker reads only the members this section names.
    (`parent_unfollowable`) a `parent` that is not both members, that names its
    own excerpt, or whose `concept_id` is in neither `excerpts` nor `withheld` —
    together those name every concept the pre-pass considered (§ 5.2), so a
-   pointer outside them names nothing in the bundle.
+   pointer outside them names nothing in the bundle. A pre-pass MAY also carry
+   the enclosing concept's text inside `parent` — `text`, with that concept's
+   own `sha256`, and `truncated` where it was cut to fit — and then only from
+   the room its cut left, so that inherited text never displaces an excerpt.
 
 ## 9. Prohibitions
 

skills/okf-consume-template/SKILL.md

@@ -129,9 +129,11 @@ what that section inherits stands in the enclosing concept, whose file is its
 `concept_id` plus `.md` under the bundle root. The text links it too, on a line
 `Enclosing section: [title](/path)`, where `/` is the bundle root. Cite what you
 take from it by that concept's own `(bundle_id, concept_id)`, never by the
-excerpt that pointed to it. An excerpt carrying `parent_unresolved` names a
+excerpt that pointed to it. When `parent` also carries `text`, the pre-pass
-parent this reader could not find in the bundle: say so rather than reporting
+followed the pointer for you: that is the enclosing concept's text, `sha256` is
-that it has none.
+that concept's own, and `truncated` means it was cut to the budget. An excerpt
+carrying `parent_unresolved` names a parent this reader could not find in the
+bundle: say so rather than reporting that it has none.
 
 | Marking | Use when |
 |---|---|

skills/okf-consume/SKILL.md

@@ -140,9 +140,11 @@ what that section inherits stands in the enclosing concept, whose file is its
 `concept_id` plus `.md` under the bundle root. The text links it too, on a line
 `Enclosing section: [title](/path)`, where `/` is the bundle root. Cite what you
 take from it by that concept's own `(bundle_id, concept_id)`, never by the
-excerpt that pointed to it. An excerpt carrying `parent_unresolved` names a
+excerpt that pointed to it. When `parent` also carries `text`, the pre-pass
-parent this reader could not find in the bundle: say so rather than reporting
+followed the pointer for you: that is the enclosing concept's text, `sha256` is
-that it has none.
+that concept's own, and `truncated` means it was cut to the budget. An excerpt
+carrying `parent_unresolved` names a parent this reader could not find in the
+bundle: say so rather than reporting that it has none.
 
 | Marking | Use when |
 |---|---|
@@ -208,7 +210,7 @@ carries its denominator.
 | Limit | `120000` |
 | Unit | `utf-8 bytes of emitted JSON` |
 | Instrument | `okf_consume.measure (len of the ensure_ascii=False JSON encoding, utf-8)` |
-| Known-positive | `docs/consumption-contract.md, encoded as a JSON string` at `14455` |
+| Known-positive | `docs/consumption-contract.md, encoded as a JSON string` at `14721` |
 
 The instrument reproduces the known-positive figure before any of its own
 numbers are believed. Report what the run actually spent.

skills/okf-consume/references/example-payload.json

@@ -11,10 +11,10 @@
     "spent": 2289,
     "known_positive": {
       "case": "docs/consumption-contract.md, encoded as a JSON string",
-      "expected": 14455,
+      "expected": 14721,
-      "measured": 14455,
+      "measured": 14721,
-      "raw_bytes": 14083,
+      "raw_bytes": 14346,
-      "encoding_delta": 372
+      "encoding_delta": 375
     }
   },
   "denominators": {

src/llm_ingestion_okf/consume.py

@@ -669,14 +669,14 @@ KNOWN_POSITIVE_CASE = "docs/consumption-contract.md, encoded as a JSON string"
 
 #: `measure()`'s own answer for that file. Vacuous ALONE -- which is why the
 #: delta below exists.
-KNOWN_POSITIVE_EXPECTED = 14_455
+KNOWN_POSITIVE_EXPECTED = 14_721
 
-#: The second, independent route. `wc -c` reports 14 083 raw bytes for the same
+#: The second, independent route. `wc -c` reports 14 346 raw bytes for the same
 #: file; the difference is this file's JSON quoting and escaping overhead. A
 #: reader can derive it without running `measure()` at all, and it moves the
 #: moment `measure()` changes what it counts -- which is what stops
 #: `expected == measured` from proving nothing.
-KNOWN_POSITIVE_ENCODING_DELTA = 372
+KNOWN_POSITIVE_ENCODING_DELTA = 375
 
 #: The two places that file can be, resolved in this order.
 #:
@@ -1613,6 +1613,12 @@ DEFAULT_SOURCE_QUOTA: int | None = 2
 WEIGHT_BUCKET = 500
 
 
+def delivered_text(body: str) -> str:
+    """A concept body as a payload carries it: NFC, trailing whitespace
+    stripped per line. One rule for an excerpt's `text` and a `parent`'s."""
+    return "\n".join(line.rstrip() for line in unicodedata.normalize("NFC", body).split("\n"))
+
+
 def excerpt_for(concept: Concept) -> dict[str, object] | None:
     """One concept as a payload excerpt, or `None` when it cannot be tiered.
 
@@ -1651,9 +1657,7 @@ def excerpt_for(concept: Concept) -> dict[str, object] | None:
     tier = trust_tier(concept.frontmatter.get("verified"))
     if tier is None:
         return None
-    text = "\n".join(
+    text = delivered_text(concept.body)
-        line.rstrip() for line in unicodedata.normalize("NFC", concept.body).split("\n")
-    )
     excerpt: dict[str, object] = {
         "bundle_id": concept.bundle_id,
         "concept_id": concept.concept_id,
@@ -1691,6 +1695,79 @@ def excerpt_weight(excerpt: Mapping[str, object]) -> int:
     return len(json.dumps(excerpt, ensure_ascii=False).encode("utf-8"))
 
 
+#: `--follow-parent` (K3-21 B): carry the enclosing concept's TEXT inside an
+#: excerpt's `parent`, rather than the pointer alone.
+DEFAULT_FOLLOW_PARENT = False
+
+
+def _carrying(
+    excerpt: Mapping[str, object], source: Concept, text: str, *, truncated: bool
+) -> dict[str, object]:
+    parent = excerpt["parent"]
+    assert isinstance(parent, dict)
+    carried: dict[str, object] = {**parent, "sha256": source.sha256, "text": text}
+    if truncated:
+        carried["truncated"] = True
+    return {**excerpt, "parent": carried}
+
+
+def attach_parent_text(
+    delivered: Sequence[dict[str, object]],
+    concepts: Mapping[str, Concept],
+    *,
+    limit: int,
+) -> tuple[dict[str, object], ...]:
+    """The delivered excerpts, each `parent` carrying its concept's text where room allows.
+
+    Runs AFTER the cut, on the room the cut left, in rank order -- so the
+    delivered set, its order and the withheld list are what they were without
+    it, and inherited text cannot displace an excerpt. That is the mechanism a
+    consumer measured when it copied ancestors' text into every heading-only
+    section of one standard: the excerpts grew until the budget held 4-8, and
+    the right section fell to withheld place 504 and 1 069.
+
+    A parent the payload already holds, or one a higher-ranked excerpt already
+    carried, is not delivered again. A text the room cannot hold whole is cut
+    to the longest prefix that fits and marked `truncated`; with no room for
+    any of it, the pointer stays a pointer. `sha256` is the enclosing concept's
+    own, so a claim resting on that text is cited as that concept (SS 3.2).
+    """
+    room = limit - sum(excerpt_weight(excerpt) for excerpt in delivered)
+    given = {str(excerpt["concept_id"]) for excerpt in delivered}
+    out: list[dict[str, object]] = []
+    for excerpt in delivered:
+        parent = excerpt.get("parent")
+        source = concepts.get(str(parent.get("concept_id"))) if isinstance(parent, dict) else None
+        if source is None or source.concept_id in given:
+            out.append(excerpt)
+            continue
+        text = delivered_text(source.body)
+        base = excerpt_weight(excerpt)
+        chosen = _carrying(excerpt, source, text, truncated=False)
+        if excerpt_weight(chosen) - base > room:
+            # The longest prefix that fits. The encoded weight never falls as
+            # a prefix grows, so the search is exact.
+            low, high = 0, len(text)
+            while low < high:
+                middle = (low + high + 1) // 2
+                trial = _carrying(excerpt, source, text[:middle], truncated=True)
+                if excerpt_weight(trial) - base <= room:
+                    low = middle
+                else:
+                    high = middle - 1
+            fits = _carrying(excerpt, source, text[:low], truncated=True)
+            chosen = fits if low and excerpt_weight(fits) - base <= room else dict(excerpt)
+        if chosen is not excerpt and "text" in _mapping_of(chosen.get("parent")):
+            given.add(source.concept_id)
+            room -= excerpt_weight(chosen) - base
+        out.append(chosen)
+    return tuple(out)
+
+
+def _mapping_of(value: object) -> Mapping[str, object]:
+    return value if isinstance(value, Mapping) else {}
+
+
 def knapsack(items: Sequence[tuple[float, int]], *, capacity: int) -> tuple[int, ...]:
     """The exact 0/1 knapsack: indices of the highest-value subset that fits.
 
@@ -1889,6 +1966,7 @@ def build_payload(
     withheld_titles: bool = False,
     stem_prefix: bool = DEFAULT_STEM_PREFIX,
     source_quota: int | None = DEFAULT_SOURCE_QUOTA,
+    follow_parent: bool = DEFAULT_FOLLOW_PARENT,
 ) -> dict[str, object]:
     """One bundle plus one question, cut to one contract-conformant payload.
 
@@ -1977,6 +2055,11 @@ def build_payload(
         reserve_top_rank=reserve_top_rank,
         source_quota=source_quota,
     )
+    if follow_parent:
+        # After the cut and never inside it: see `attach_parent_text`.
+        delivered = attach_parent_text(
+            delivered, {concept.concept_id: concept for concept in concepts}, limit=limit
+        )
     spent = sum(excerpt_weight(excerpt) for excerpt in delivered)
     if matched and not delivered:
         # SS 7.3: a finding requiring a decision, never something to retry
@@ -2191,6 +2274,23 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
         dest="source_quota",
         help="The rule's explicit opt-out, reproducing the pre-round-11 excerpt order",
     )
+    parser.add_argument(
+        "--follow-parent",
+        action="store_true",
+        default=DEFAULT_FOLLOW_PARENT,
+        help=(
+            "carry the enclosing concept's text inside an excerpt's `parent`, "
+            "with that concept's sha256, from the room the cut LEFT and in rank "
+            "order -- so it never displaces an excerpt; a text that does not fit "
+            "is clipped and marked `truncated` (K3-21)"
+        ),
+    )
+    parser.add_argument(
+        "--no-follow-parent",
+        action="store_false",
+        dest="follow_parent",
+        help="The rule's explicit opt-out: `parent` is the pointer alone",
+    )
     parser.add_argument(
         "--withheld-titles",
         action="store_true",
@@ -2240,6 +2340,7 @@ def main(argv: list[str] | None = None) -> int:
             stem_prefix=args.stem_prefix,
             source_quota=args.source_quota,
             withheld_titles=args.withheld_titles,
+            follow_parent=args.follow_parent,
         )
     except ConsumeError as error:
         print(f"okf_consume: FAILED - {error}", file=sys.stderr)

tests/test_parent_text.py

@@ -94,12 +94,30 @@ def test_a_text_that_does_not_fit_is_clipped_and_says_so(tmp_path: Path) -> None
 
 
 def test_no_room_means_no_text_and_no_lost_excerpt(tmp_path: Path) -> None:
+    """Held at the function, not through `build_payload`: the knapsack buckets
+    weights up and capacity down to 500 B, so a payload limit equal to what was
+    spent admits nothing at all (`budget_admits_nothing`). The red version of
+    this test asked through that and met the bucket instead of the rule."""
     bundle = _flagged(tmp_path)
     plain = consume.build_payload(bundle, question=SHELL, k=1)
-    exact = plain["budget"]["spent"]
+    concepts = {
-    followed = consume.build_payload(bundle, question=SHELL, k=1, follow_parent=True, limit=exact)
+        concept.concept_id: concept
-    assert "text" not in _shell(followed)["parent"]
+        for concept in consume.link_parents(
-    assert _ids(followed) == _ids(plain)
+            [
+                consume.read_concept(
+                    bundle / f"{concept_id}.md",
+                    bundle_root=bundle,
+                    root_bundle_id="inherit-fixture",
+                )
+                for concept_id in consume.enumerate_concepts(bundle)
+            ]
+        )
+    }
+    spent = plain["budget"]["spent"]
+    # No room, and room smaller than the members a text needs around it.
+    for limit in (spent, spent + 30):
+        followed = consume.attach_parent_text(plain["excerpts"], concepts, limit=limit)
+        assert list(followed) == plain["excerpts"]
 
 
 def test_a_parent_the_payload_already_holds_is_not_delivered_twice(tmp_path: Path) -> None: