fix(consume): every read path into a bundle is contained, not just okf_fetch

`okf_fetch` resolved a concept through `connectors.safe_resolve` from the day
the server was written. The other three ways into the same bytes did not.
`okf consume` and `okf_ask` reach `consume.build_payload`, `okf_describe`
reaches `mcp_server.card`, and both built the concept path by joining the
index's own name onto the bundle root. `consume._join` refuses a `..` segment
and an absolute target, but it is a STRING rule over the index text, and a
symlink is a fact about the filesystem that reading that text cannot see: the
index could name `lekkasje.md`, that name could be a link to a file outside the
bundle, and the file came back in the answer.

Measured before the fix, on a bundle carrying one honest concept and one
escaping link: 8 of 11 new rows red, the 3 green ones being `okf_fetch` on the
same two links and the known-positive that the clean bundle still answers. So
the suite was not red for an unrelated reason, and the fix is not "refuse every
bundle holding a link".

One place, not three copies: `consume.resolve_in_bundle` makes the check and
`consume.read_path_in_bundle` adds the file's presence. Every reader here goes
through them -- the index walk, the ref, the document prior, the payload, the
card, `okf_fetch`, and the three outside `consume` (`skill`, `quality`,
`project`) that joined the same way.

Two more failure modes in the same check, because they are the same question:

* A NAMED PIPE is not a regular file. `read_text` on one blocks for as long as
  nobody writes to it, which on a server is the whole process; the red row for
  it ran 60 s to a subprocess deadline and now returns in under a second.
* A DEAD INDEX LINK raised `FileNotFoundError`, and the broad handler in
  `handle` wrote `{error}` into the refusal -- the SERVER's absolute path,
  handed to whoever asked, over one index entry naming a file nobody wrote.
  It is `concept_unreadable` now, naming the concept and not the machine.

The returned path is the JOINED one, never the resolved one: `read_concept`
derives a concept id by taking the read path relative to the bundle root, and
once containment holds the two are the same bytes.

2334 passed, 2 skipped (was 2323 + 2). `mypy --strict src/` clean over 25 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

src/llm_ingestion_okf/consume.py

@@ -54,7 +54,9 @@ from dataclasses import dataclass, replace
 from pathlib import Path
 from typing import Literal
 
+from .connectors import safe_resolve
 from .corpus import LOG_NAME
+from .errors import SourceError
 from .inbox import (
     ADJUDICATION_ADJUDICATED,
     ADJUDICATION_PROPOSED,
@@ -106,7 +108,7 @@ def _walk_index_tree(
         if relative in indexes:
             continue
         indexes.add(relative)
-        index_file = bundle_root / relative
+        index_file = resolve_in_bundle(bundle_root, relative)
         if not index_file.is_file():
             continue
         parent = _parent_dir(relative)
@@ -201,6 +203,60 @@ def _join(parent: str, target: str) -> str | None:
     return "/".join(parts) if parts else None
 
 
+def resolve_in_bundle(bundle_root: Path, relative: str) -> Path:
+    """The ONE containment check every read path into a bundle goes through.
+
+    `_join` above refuses a `..` segment and an absolute target, but it is a
+    STRING rule over the index text, and a symlink is a fact about the
+    filesystem that no amount of reading that text can see. So this resolves
+    the canonical path (`connectors.safe_resolve`) and refuses anything that
+    lands outside the root -- the check `okf_fetch` has always made, now made
+    in the one place every reader here passes through instead of in the one
+    tool that happened to have it.
+
+    The same check also refuses a target that exists and is NOT a regular
+    file, because that is the other way a name reaches something the bundle
+    does not hold: a named pipe is opened without complaint and blocks the
+    reader for as long as nobody writes to it, which on a server is the whole
+    process.
+
+    Returns the path as JOINED, never as resolved: the resolved form is the
+    symlink's target, and `read_concept` derives a concept id by taking the
+    read path relative to the bundle root. Once containment holds, reading
+    through the link and reading the file are the same bytes.
+    """
+    try:
+        safe_resolve(bundle_root, relative)
+    except SourceError as error:
+        raise ConsumeError(
+            f"`{relative}` resolves outside the bundle that names it; refused rather than read",
+            code="path_escape",
+        ) from error
+    joined = bundle_root / relative
+    if joined.exists() and not joined.is_file():
+        raise ConsumeError(
+            f"`{relative}` names something that is not a regular file; refused rather than opened",
+            code="path_escape",
+        )
+    return joined
+
+
+def read_path_in_bundle(bundle_root: Path, relative: str) -> Path:
+    """`resolve_in_bundle`, and the file has to be there.
+
+    Refuses by NAME. A `FileNotFoundError` escaping from here carried the
+    reader's own absolute path into an answer, over one index entry naming a
+    file nobody wrote -- a fact about the machine, handed to whoever asked.
+    """
+    path = resolve_in_bundle(bundle_root, relative)
+    if not path.is_file():
+        raise ConsumeError(
+            f"the index names `{relative}`, and there is no readable file there",
+            code="concept_unreadable",
+        )
+    return path
+
+
 def bundle_ref(bundle_root: Path, *, profile: BundleProfile = DEFAULT_PROFILE) -> str:
     """A content identity for the bundle: `sha256-tree:<hex>`.
 
@@ -223,7 +279,7 @@ def bundle_ref(bundle_root: Path, *, profile: BundleProfile = DEFAULT_PROFILE) -
     suffix = profile.paths.concept_suffix
     lines: list[bytes] = []
     for relative in (*indexes, *(f"{concept}{suffix}" for concept in concepts)):
-        path = bundle_root / relative
+        path = resolve_in_bundle(bundle_root, relative)
         if not path.is_file():
             continue
         digest = hashlib.sha256(path.read_bytes()).hexdigest()
@@ -1206,7 +1262,10 @@ def document_scores(
         document = relative.split("/", 1)[0]
         if document == profile.index.name:
             continue
-        for line in (bundle_root / relative).read_text(encoding="utf-8").splitlines():
+        index_file = resolve_in_bundle(bundle_root, relative)
+        if not index_file.is_file():
+            continue
+        for line in index_file.read_text(encoding="utf-8").splitlines():
             entry = profile.index.parse_entry(line)
             if entry is None:
                 continue
@@ -2073,7 +2132,7 @@ def root_bundle_id_of(bundle_root: Path, *, profile: BundleProfile = DEFAULT_PRO
     """The `bundle_id` the root index declares, or a refusal naming which half
     of SS 3.1's identity tuple is missing. Shared with the skill generator, so
     the two agree on what makes a directory a readable bundle."""
-    root_index = bundle_root / profile.index.name
+    root_index = resolve_in_bundle(bundle_root, profile.index.name)
     if not root_index.is_file():
         raise ConsumeError(
             f"{bundle_root} carries no {profile.index.name}, so there is no index "
@@ -2210,7 +2269,7 @@ def build_payload(
     concepts = link_parents(
         [
             read_concept(
-                bundle_root / f"{concept_id}{profile.paths.concept_suffix}",
+                read_path_in_bundle(bundle_root, f"{concept_id}{profile.paths.concept_suffix}"),
                 bundle_root=bundle_root,
                 root_bundle_id=root_bundle_id,
             )

src/llm_ingestion_okf/mcp_server.py

@@ -39,6 +39,13 @@ if the bundle's own index names it (`consume.enumerate_concepts`, which
 refuses a target climbing above the root) AND its resolved path is inside the
 bundle (`connectors.safe_resolve`, on canonical paths). Either alone would be
 defensible; the pair is what makes a defect in one of them survivable.
+
+AND IT IS EVERY READ PATH, not the one tool that happened to have it. Until
+`consume.resolve_in_bundle` existed, the second check was made by `okf_fetch`
+alone: `okf_ask` and `okf_describe` joined the index's own name onto the root
+and opened whatever was there, so a link out of the bundle was read and
+delivered. The index rule is a STRING rule -- it cannot see a symlink -- which
+is exactly why one of the two checks is not enough.
 """
 
 from __future__ import annotations
@@ -53,7 +60,6 @@ from typing import Any, TextIO
 
 from . import consume as okf_consume
 from . import materialize
-from .connectors import safe_resolve
 from .errors import SourceError
 from .profiles import BundleProfile
 
@@ -276,7 +282,9 @@ def card(bundle_root: Path, *, profile: BundleProfile, concept_sample: int = 50)
     concepts = okf_consume.link_parents(
         [
             okf_consume.read_concept(
-                bundle_root / f"{concept_id}{profile.paths.concept_suffix}",
+                okf_consume.read_path_in_bundle(
+                    bundle_root, f"{concept_id}{profile.paths.concept_suffix}"
+                ),
                 bundle_root=bundle_root,
                 root_bundle_id=bundle_id,
             )
@@ -489,10 +497,11 @@ def call_fetch(surface: Surface, arguments: Mapping[str, Any]) -> dict[str, Any]
             code="concept_unknown",
         )
     suffix = surface.profile.paths.concept_suffix
-    try:
+    # The SECOND of the two independent checks, and since the read paths were
-        path = safe_resolve(served.root, f"{concept_id}{suffix}")
+    # unified it is the same one `okf_ask` and `okf_describe` make. Left as its
-    except SourceError as error:
+    # own call rather than folded into the index check above: a defect in one
-        raise ToolError(str(error), code="path_escape") from error
+    # of the two is survivable only while the other is still asked.
+    path = okf_consume.read_path_in_bundle(served.root, f"{concept_id}{suffix}")
     size = path.stat().st_size
     if size > MAX_CONCEPT_BYTES:
         raise ToolError(

src/llm_ingestion_okf/project.py

@@ -98,7 +98,9 @@ def inventory(folder: Path, bundle: Path) -> tuple[tuple[str, ...], tuple[str, .
     whole: set[str] = set()
     root_bundle_id = consume.root_bundle_id_of(bundle, profile=SEGMENTED_OKF_V0_2)
     for concept_id in consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2):
-        path = bundle / f"{concept_id}{SEGMENTED_OKF_V0_2.paths.concept_suffix}"
+        path = consume.read_path_in_bundle(
+            bundle, f"{concept_id}{SEGMENTED_OKF_V0_2.paths.concept_suffix}"
+        )
         concept = consume.read_concept(path, bundle_root=bundle, root_bundle_id=root_bundle_id)
         represented.add(concept.source_file)
         # A concept id with no `/` sits at the bundle root rather than under a

src/llm_ingestion_okf/quality.py

@@ -41,7 +41,13 @@ from collections import Counter
 from dataclasses import dataclass
 from pathlib import Path
 
-from .consume import ConsumeError, enumerate_concepts, read_concept, root_bundle_id_of
+from .consume import (
+    ConsumeError,
+    enumerate_concepts,
+    read_concept,
+    read_path_in_bundle,
+    root_bundle_id_of,
+)
 from .corpus import LOG_NAME
 from .profiles import SEGMENTED_OKF_V0_2, BundleProfile
 
@@ -487,7 +493,7 @@ def measure_bundle(
     pairs: set[tuple[str, str]] = set()
     for concept_id in enumerate_concepts(bundle_root, profile=profile):
         concept = read_concept(
-            bundle_root / f"{concept_id}{profile.paths.concept_suffix}",
+            read_path_in_bundle(bundle_root, f"{concept_id}{profile.paths.concept_suffix}"),
             bundle_root=bundle_root,
             root_bundle_id=root_bundle_id,
         )

src/llm_ingestion_okf/skill.py

@@ -347,7 +347,9 @@ def render(
     concepts = okf_consume.link_parents(
         [
             okf_consume.read_concept(
-                bundle_root / f"{concept_id}{profile.paths.concept_suffix}",
+                okf_consume.read_path_in_bundle(
+                    bundle_root, f"{concept_id}{profile.paths.concept_suffix}"
+                ),
                 bundle_root=bundle_root,
                 root_bundle_id=bundle_id,
             )

tests/test_read_path_containment.py

@@ -0,0 +1,217 @@
+"""Every read path into a bundle is contained, not just the one that fetches.
+
+`okf_fetch` has resolved a concept through `connectors.safe_resolve` since the
+server was written. The other three ways into the same bytes did not: `okf
+consume` and `okf_ask` reach `consume.build_payload`, and `okf_describe`
+reaches `mcp_server.card`, and both built the concept path by joining the
+bundle-relative name onto the root. `consume._join` refuses a `..` segment and
+an absolute target, but it is a STRING rule -- a symlink is a fact about the
+filesystem, and no amount of reading the index text can see one.
+
+So the index could name `leak.md`, that name could be a link to a file outside
+the bundle, and the file's contents came back in the answer. The pair of checks
+the module docstring promises held for one tool out of three.
+
+The named pipe is the same check and the other failure mode: a path that is
+neither a regular file nor a refusal is a path that blocks the server for as
+long as nobody writes to it. Those cases run in a SUBPROCESS with a deadline,
+because a test that hangs is not a red test.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import subprocess
+import sys
+import textwrap
+from pathlib import Path
+
+import pytest
+
+from llm_ingestion_okf import consume, mcp_server
+
+TOOLS = Path(__file__).resolve().parents[1] / "tools"
+sys.path.insert(0, str(TOOLS))
+
+import okf_mcp_gate as gate  # noqa: E402
+
+#: A token that occurs NOWHERE in the bundle, only in the file outside it. An
+#: answer carrying it read something the bundle never held.
+SECRET = "SEKRETMARKOER"
+
+QUESTION = f"hva sier notatet om {SECRET.lower()} og krav"
+
+
+def _outside_concept(path: Path) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(
+        gate.CONCEPT.format(
+            title=f"Hemmelig notat {SECRET}",
+            source="hemmelig.txt",
+            bundle_id="not-this-bundle",
+            segment="s9",
+            body=f"Dette notatet ligger UTENFOR samlingen og inneholder {SECRET}.",
+        ),
+        encoding="utf-8",
+    )
+
+
+def _bundle_with_escape(tmp_path: Path, *, link: str) -> Path:
+    """A bundle whose index names one honest concept and one escaping link.
+
+    `link` is `"file"` (a link to a file outside) or `"directory"` (a link to a
+    directory outside, with the index naming a file through it).
+    """
+    outside = tmp_path / "outside"
+    _outside_concept(outside / "hemmelig.md")
+    root = tmp_path / "root"
+    gate.write_bundle(root, "escape-bundle", [("krav", "Krav", "Et krav om krav og notat.")])
+    if link == "file":
+        (root / "lekkasje.md").symlink_to(outside / "hemmelig.md")
+        entry = "- [Lekkasje](lekkasje.md) — adjudication: proposed"
+    else:
+        (root / "lenket").symlink_to(outside, target_is_directory=True)
+        entry = "- [Lekkasje](lenket/hemmelig.md) — adjudication: proposed"
+    index = root / "index.md"
+    index.write_text(index.read_text(encoding="utf-8") + entry + "\n", encoding="utf-8")
+    return root
+
+
+def _surface(root: Path) -> mcp_server.Surface:
+    return mcp_server.build_surface(bundle=root, roots=())
+
+
+def _refusal(root: Path, tool: str, arguments: dict[str, object]) -> str:
+    """The envelope a client actually sees: `refused (<code>): <message>`."""
+    result = mcp_server.handle(_surface(root), "tools/call", {"name": tool, "arguments": arguments})
+    text = result["content"][0]["text"]
+    assert result["isError"], f"{tool} answered instead of refusing: {text[:400]}"
+    return text
+
+
+@pytest.mark.parametrize("link", ["file", "directory"])
+def test_okf_consume_refuses_a_symlink_out_of_the_bundle(tmp_path: Path, link: str) -> None:
+    root = _bundle_with_escape(tmp_path, link=link)
+    with pytest.raises(consume.ConsumeError) as raised:
+        consume.build_payload(root, question=QUESTION, k=8, limit=120_000)
+    assert raised.value.code == "path_escape"
+    assert SECRET not in str(raised.value)
+
+
+@pytest.mark.parametrize("link", ["file", "directory"])
+def test_okf_ask_refuses_a_symlink_out_of_the_bundle(tmp_path: Path, link: str) -> None:
+    root = _bundle_with_escape(tmp_path, link=link)
+    text = _refusal(root, "okf_ask", {"question": QUESTION})
+    assert "path_escape" in text
+    assert SECRET not in text
+
+
+@pytest.mark.parametrize("link", ["file", "directory"])
+def test_okf_describe_refuses_a_symlink_out_of_the_bundle(tmp_path: Path, link: str) -> None:
+    root = _bundle_with_escape(tmp_path, link=link)
+    text = _refusal(root, "okf_describe", {})
+    assert "path_escape" in text
+    assert SECRET not in text
+
+
+@pytest.mark.parametrize("link", ["file", "directory"])
+def test_okf_fetch_refuses_the_same_link_and_is_the_control(tmp_path: Path, link: str) -> None:
+    """The arm that was already contained. Without it, a suite that refused
+    everything for some unrelated reason would look like a fix."""
+    root = _bundle_with_escape(tmp_path, link=link)
+    concept = "lekkasje" if link == "file" else "lenket/hemmelig"
+    text = _refusal(root, "okf_fetch", {"concept_id": concept})
+    assert "path_escape" in text
+    assert SECRET not in text
+
+
+def test_the_honest_concept_of_that_same_bundle_is_still_delivered(tmp_path: Path) -> None:
+    """The known-positive for all four rows above: the refusal is the LINK's,
+    never the bundle's. A rule that refused every bundle holding a link would
+    pass every assertion above and destroy the tool."""
+    root = tmp_path / "root"
+    gate.write_bundle(root, "clean-bundle", [("krav", "Krav", "Et krav om krav og notat.")])
+    payload = consume.build_payload(root, question=QUESTION, k=8, limit=120_000)
+    assert payload["excerpts"], "the control question delivered nothing"
+    card = mcp_server.card(root, profile=consume.DEFAULT_PROFILE)
+    assert card["concept_count"] == 1
+
+
+# --------------------------------------------------------------------------
+# The named pipe, and the dead index link, both in a subprocess with a deadline
+# --------------------------------------------------------------------------
+
+_DRIVER = """
+import json, sys
+from pathlib import Path
+from llm_ingestion_okf import consume, mcp_server
+
+root = Path(sys.argv[1])
+what = sys.argv[2]
+codes = {}
+try:
+    consume.build_payload(root, question="krav notat", k=8, limit=120_000)
+    codes["consume"] = "ANSWERED"
+except consume.ConsumeError as error:
+    codes["consume"] = error.code
+surface = mcp_server.build_surface(bundle=root, roots=())
+for tool, arguments in (("okf_describe", {}), ("okf_ask", {"question": "krav notat"})):
+    result = mcp_server.handle(surface, "tools/call", {"name": tool, "arguments": arguments})
+    codes[tool] = result["content"][0]["text"] if result.get("isError") else "ANSWERED"
+print(json.dumps(codes))
+"""
+
+
+def _drive(root: Path, what: str) -> dict[str, str]:
+    """Run the three read paths out of process, with a deadline.
+
+    A named pipe with no writer blocks the reader forever, so an in-process
+    assertion would hang the suite instead of failing it. `TimeoutExpired`
+    surfaces here as the failure it is.
+    """
+    run = subprocess.run(
+        [sys.executable, "-c", textwrap.dedent(_DRIVER), str(root), what],
+        capture_output=True,
+        text=True,
+        timeout=60,
+    )
+    assert run.returncode == 0, f"driver crashed: {run.stderr[-2000:]}"
+    return json.loads(run.stdout.strip().splitlines()[-1])
+
+
+def test_a_named_pipe_in_the_index_is_refused_and_never_read(tmp_path: Path) -> None:
+    root = tmp_path / "root"
+    gate.write_bundle(root, "pipe-bundle", [("krav", "Krav", "Et krav om krav og notat.")])
+    os.mkfifo(root / "roer.md")
+    index = root / "index.md"
+    index.write_text(
+        index.read_text(encoding="utf-8") + "- [Roer](roer.md) — adjudication: proposed\n",
+        encoding="utf-8",
+    )
+    codes = _drive(root, "fifo")
+    assert codes["consume"] == "path_escape", codes
+    assert "path_escape" in codes["okf_describe"], codes
+    assert "path_escape" in codes["okf_ask"], codes
+
+
+def test_a_dead_index_link_is_refused_by_code_and_concept_id_not_by_path(tmp_path: Path) -> None:
+    """The refusal names what the client can act on.
+
+    A `FileNotFoundError` escaping into the envelope wrote the SERVER's absolute
+    path into an answer -- the reader's own directory layout, handed to whoever
+    asked, over one index entry naming a file nobody wrote.
+    """
+    root = tmp_path / "root"
+    gate.write_bundle(root, "dead-bundle", [("krav", "Krav", "Et krav om krav og notat.")])
+    index = root / "index.md"
+    index.write_text(
+        index.read_text(encoding="utf-8") + "- [Borte](borte.md) — adjudication: proposed\n",
+        encoding="utf-8",
+    )
+    codes = _drive(root, "dead")
+    for key in ("okf_describe", "okf_ask"):
+        assert "borte" in codes[key], codes[key]
+        assert str(root) not in codes[key], codes[key]
+        assert str(tmp_path) not in codes[key], codes[key]
+    assert codes["consume"] != "ANSWERED", codes