open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

fix(consume): every read path into a bundle is contained, not just okf_fetch

Browse source 
`okf_fetch` resolved a concept through `connectors.safe_resolve` from the day
the server was written. The other three ways into the same bytes did not.
`okf consume` and `okf_ask` reach `consume.build_payload`, `okf_describe`
reaches `mcp_server.card`, and both built the concept path by joining the
index's own name onto the bundle root. `consume._join` refuses a `..` segment
and an absolute target, but it is a STRING rule over the index text, and a
symlink is a fact about the filesystem that reading that text cannot see: the
index could name `lekkasje.md`, that name could be a link to a file outside the
bundle, and the file came back in the answer.

Measured before the fix, on a bundle carrying one honest concept and one
escaping link: 8 of 11 new rows red, the 3 green ones being `okf_fetch` on the
same two links and the known-positive that the clean bundle still answers. So
the suite was not red for an unrelated reason, and the fix is not "refuse every
bundle holding a link".

One place, not three copies: `consume.resolve_in_bundle` makes the check and
`consume.read_path_in_bundle` adds the file's presence. Every reader here goes
through them -- the index walk, the ref, the document prior, the payload, the
card, `okf_fetch`, and the three outside `consume` (`skill`, `quality`,
`project`) that joined the same way.

Two more failure modes in the same check, because they are the same question:

* A NAMED PIPE is not a regular file. `read_text` on one blocks for as long as
  nobody writes to it, which on a server is the whole process; the red row for
  it ran 60 s to a subprocess deadline and now returns in under a second.
* A DEAD INDEX LINK raised `FileNotFoundError`, and the broad handler in
  `handle` wrote `{error}` into the refusal -- the SERVER's absolute path,
  handed to whoever asked, over one index entry naming a file nobody wrote.
  It is `concept_unreadable` now, naming the concept and not the machine.

The returned path is the JOINED one, never the resolved one: `read_concept`
derives a concept id by taking the read path relative to the bundle root, and
once containment holds the two are the same bytes.

2334 passed, 2 skipped (was 2323 + 2). `mypy --strict src/` clean over 25 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-20 15:20:38 +02:00
commit bf697bfcad
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
6 changed files with 310 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

4
src/llm_ingestion_okf/project.py
View file

@ -98,7 +98,9 @@ def inventory(folder: Path, bundle: Path) -> tuple[tuple[str, ...], tuple[str, .
whole: set[str] = set()
root_bundle_id = consume.root_bundle_id_of(bundle, profile=SEGMENTED_OKF_V0_2)
for concept_id in consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2):
path = bundle / f"{concept_id}{SEGMENTED_OKF_V0_2.paths.concept_suffix}"
path = consume.read_path_in_bundle(
bundle, f"{concept_id}{SEGMENTED_OKF_V0_2.paths.concept_suffix}"
)
concept = consume.read_concept(path, bundle_root=bundle, root_bundle_id=root_bundle_id)
represented.add(concept.source_file)
# A concept id with no `/` sits at the bundle root rather than under a