feat(okf): scan reserved index.md/log.md in mode-b import, not path-reject (review MAJOR #2)

A received OKF bundle MAY legitimately carry index.md (directory listing, read
first under progressive disclosure) and log.md (update history) at any level
(spec §3.1/§6/§7). import_bundle previously hard-rejected those basenames in the
T4 path gate, so a conformant third-party bundle was over-blocked in full
(FAIL_SECURE) — and because the reject fired before scan_concept, index.md's
body (the highest-priority injection surface) was never scanned.

import_bundle now defaults allow_reserved=True: reserved basenames are scanned
as structural files (path-safety checks — traversal / absolute / backslash / .md
— still apply). The shadow-reject (an *upload* masquerading as index.md) is
preserved: the front-end passes allow_reserved=False so a materialized upload
landing on a reserved basename is still refused. That front-end opt-in was
required to keep the shadow-reject once the default flipped (not in the plan's
Filer set; traced from the code).

- okf.py: validate_concept_path/_validate_concept/import_bundle gain the
  keyword; validate_concept_path default stays False (strict standalone).
- tests: +3 (legit index/log admit; injection in index.md body caught;
  okf_version frontmatter admits). Per-concept-iteration test switched to a
  traversal vector; mode-b showcase's index.md surface reframed from
  reserved-name-reject to index.md-body-scan.
- README honest-limits + CLAUDE.md context note the mode-b/upload distinction.

Suite: 341 -> 344 passed. Core invariant intact (dependencies=[]).

CLAUDE.md

@@ -13,8 +13,11 @@ Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen.
 
 Repoet er på **v0.2 (alpha)**: stdlib-kjernen er bygget og testet (12 moduler +
 topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
-detektor (EchoLeak-klassen) i output-gaten. Start med `docs/BRIEF.md` for design,
+detektor (EchoLeak-klassen) i output-gaten. Mode-b `import_bundle` skanner
-`README.md` for bruk, `docs/PLAN.md` for byggerekkefølgen.
+reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
+path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
+Start med `docs/BRIEF.md` for design, `README.md` for bruk, `docs/PLAN.md` for
+byggerekkefølgen.
 
 ## Konvensjoner
 

README.md

@@ -116,6 +116,12 @@ that a green scan means safe content:
   the payload is planted later, when that target is written. A scanner that sees
   one document at a time cannot catch it; it needs cross-write graph re-scan
   (tracked for the OKF adapter).
+- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are
+  legitimate structure (directory listing, update log), so mode-b `import_bundle`
+  scans their body and frontmatter — an injection planted in a directory listing
+  is caught — rather than path-rejecting the whole conformant bundle. The upload
+  front-end keeps the opposite rule: an individual upload landing on a reserved
+  basename is a shadow of the listing and is refused (`allow_reserved=False`).
 - **A document that *describes* attacks is a false positive.** Content whose
   legitimate purpose is to document prompt-injection payloads (security notes,
   this project's own corpus) trips carrier-strip / fail-secure. At the text layer

src/llm_ingestion_guard/okf.py

@@ -151,7 +151,7 @@ def _scannable_regions(frontmatter, body):
     return regions
 
 
-def validate_concept_path(path):
+def validate_concept_path(path, *, allow_reserved=False):
     """Validate a bundle-relative concept path and return its concept-ID.
 
     T4 — path / reserved-name gate. The concept-ID is the path with the ``.md``
@@ -164,6 +164,13 @@ def validate_concept_path(path):
       lets ``Index.md`` shadow ``index.md``;
     - non-``.md`` files (not a concept document).
 
+    ``allow_reserved`` (default ``False``) keeps this a strict concept-path
+    validator: a reserved basename is not a concept and is rejected. A mode-b
+    bundle import passes ``allow_reserved=True`` because a *received* bundle MAY
+    legitimately carry ``index.md`` / ``log.md`` as structural files — the caller
+    then scans their body rather than persisting them as concepts. The path-safety
+    checks (traversal / absolute / backslash / ``.md``) still apply either way.
+
     Raises :class:`OKFPathError` on any of these; returns the concept-ID string.
     """
     if not path or not isinstance(path, str):
@@ -181,7 +188,7 @@ def validate_concept_path(path):
             raise OKFPathError("malformed path segment in %r" % path)
 
     basename = segments[-1]
-    if basename.lower() in _RESERVED_BASENAMES:
+    if not allow_reserved and basename.lower() in _RESERVED_BASENAMES:
         raise OKFPathError("reserved filename may not name a concept: %r" % basename)
     if not basename.lower().endswith(".md"):
         raise OKFPathError("a concept document must be a .md file: %r" % path)
@@ -338,7 +345,7 @@ class BundleResult:
         return "\n".join(lines)
 
 
-def import_bundle(bundle, *, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC):
+def import_bundle(bundle, *, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC, allow_reserved=True):
     """Validate a received OKF bundle concept-by-concept before merge (mode b).
 
     ``bundle`` maps concept path (e.g. ``tables/users.md``) to its raw document
@@ -348,18 +355,27 @@ def import_bundle(bundle, *, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC):
     rejected (FAIL_SECURE) and recorded, but iteration continues, so the caller
     sees every issue in the bundle, not only the first. The bundle disposition is
     the most severe across its concepts.
+
+    ``allow_reserved`` (default ``True``) reflects that this is the mode-b
+    *received-bundle* path: ``index.md`` / ``log.md`` are legitimate structural
+    files (OKF spec §3.1/§6/§7) that MAY appear at any level, so they are scanned
+    (their body is the highest-priority injection surface) rather than
+    path-rejected — over-blocking a conformant third-party bundle is itself a
+    failure mode (brief principle 5). A front-end materialising individual
+    *uploads* passes ``allow_reserved=False``: there a reserved basename is a
+    shadow of the directory listing and must be refused.
     """
     results = tuple(
-        _validate_concept(path, bundle[path], origin, channel)
+        _validate_concept(path, bundle[path], origin, channel, allow_reserved=allow_reserved)
         for path in sorted(bundle)
     )
     aggregate = _most_severe(r.disposition for r in results)
     return BundleResult(results, aggregate, link_graph(bundle))
 
 
-def _validate_concept(path, doc, origin, channel):
+def _validate_concept(path, doc, origin, channel, *, allow_reserved=True):
     try:
-        concept_id = validate_concept_path(path)
+        concept_id = validate_concept_path(path, allow_reserved=allow_reserved)
     except OKFPathError as exc:
         return ConceptResult(path, None, Disposition.FAIL_SECURE, None, Report(), str(exc))
     try:

tests/inbox_frontend.py

@@ -351,6 +351,12 @@ def receive(
     REJECT regardless of the guard's aggregate — the guard never saw that drop.
     """
     extracted = extract_inbox(paths, max_entry_bytes=max_entry_bytes, max_total_bytes=max_total_bytes)
-    result = import_bundle(extracted.bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC)
+    # allow_reserved=False: these are individually-materialized *uploads*, so an
+    # upload landing on the reserved index.md/log.md is a shadow of the directory
+    # listing and is refused (T4). A received third-party bundle, by contrast,
+    # carries those as legitimate structural files (import_bundle's default).
+    result = import_bundle(
+        extracted.bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC, allow_reserved=False
+    )
     verdict = "REJECT" if extracted.rejected else _VERDICT[result.disposition]
     return extracted, result, verdict

tests/test_okf.py

@@ -333,11 +333,12 @@ def test_import_bundle_all_clean_warns():
 
 
 def test_import_bundle_iterates_per_concept_not_whole_unit():
-    # a reserved-name concept is rejected, but the good concept is still validated
+    # a hard-rejected concept (path traversal) is FAIL_SECURE, but the good
-    result = import_bundle({"index.md": _CLEAN_A, "tables/users.md": _CLEAN_B})
+    # concept is still validated — iteration does not stop at the first reject.
+    result = import_bundle({"../escape.md": _CLEAN_A, "tables/users.md": _CLEAN_B})
     by_path = {c.path: c for c in result.concepts}
-    assert by_path["index.md"].disposition is Disposition.FAIL_SECURE
+    assert by_path["../escape.md"].disposition is Disposition.FAIL_SECURE
-    assert by_path["index.md"].error is not None
+    assert by_path["../escape.md"].error is not None
     assert by_path["tables/users.md"].error is None
     assert by_path["tables/users.md"].disposition is Disposition.WARN
 
@@ -384,6 +385,58 @@ def test_import_bundle_records_origin_channel_on_stamp():
     assert stamp.channel is Channel.MANUAL
 
 
+# --- A2: reserved structural files (index.md / log.md) in a received bundle ---
+# OKF spec §3.1/§6/§7: index.md (directory listing, read FIRST under progressive
+# disclosure) and log.md (update history) are legitimate structural files a
+# received bundle MAY carry at any level — not concepts, but attacker-controlled
+# text. In a mode-b import (the default), import_bundle scans their body (the
+# highest-priority injection surface) instead of path-rejecting the whole bundle.
+# The shadow-reject — an *upload* masquerading as index.md — stays in the
+# front-end/upload context (allow_reserved=False), tested in
+# test_okf_inbox_uploads.py.
+
+_CLEAN_INDEX = (
+    "---\ntype: table\ndescription: A directory listing.\n---\nA clean listing body.\n"
+)
+_CLEAN_LOG = "---\ntype: table\n---\nA clean change-log entry.\n"
+
+
+def test_legit_index_and_log_admit():
+    result = import_bundle(
+        {"index.md": _CLEAN_INDEX, "log.md": _CLEAN_LOG, "tables/users.md": _CLEAN_A}
+    )
+    by_path = {c.path: c for c in result.concepts}
+    assert by_path["index.md"].error is None      # scanned, not path-rejected
+    assert by_path["log.md"].error is None
+    assert result.disposition is Disposition.WARN  # a clean structural bundle admits
+
+
+def test_injection_in_index_body_is_caught():
+    # The coverage hole A2 closes: index.md's body was never scanned (path-rejected
+    # first). Now an injection planted in the directory listing is caught.
+    poisoned_index = "---\ntype: table\n---\n" + _INJECTION + "\n"
+    result = import_bundle({"index.md": poisoned_index, "tables/users.md": _CLEAN_A})
+    idx = {c.path: c for c in result.concepts}["index.md"]
+    assert idx.error is None                        # scanned, not path-rejected
+    assert any(f.label == "override:ignore-previous" for f in idx.report.findings)
+    assert result.disposition in (Disposition.QUARANTINE_REVIEW, Disposition.FAIL_SECURE)
+
+
+def test_index_with_okf_version_frontmatter_admits():
+    # Risk (review): okf_version frontmatter is legal only in the bundle-root
+    # index.md. Scanning its body must parse the frontmatter without the strict
+    # T2 gate tripping on that legitimate key.
+    result = import_bundle(
+        {
+            "index.md": "---\nokf_version: 0.1\n---\n# Concept listing\n",
+            "tables/users.md": "---\ntype: table\n---\nA clean users table.\n",
+        }
+    )
+    by_path = {c.path: c for c in result.concepts}
+    assert by_path["index.md"].error is None
+    assert result.disposition is Disposition.WARN
+
+
 # --- T5a/A: cross-link extraction, target validation, in-import resolution ----
 # OKF links are markdown `.md` paths, bundle-absolute (`/x.md`, recommended) or
 # relative (`./x.md`); verified against SPEC.md. In-import graph only (A); the

tests/test_okf_showcase.py

@@ -3,8 +3,9 @@
 The OKF analogue of ``tests/test_showcase.py``: one realistic *received external
 OKF bundle* that plants one attack per OKF surface at once — a body injection, a
 frontmatter-``description`` injection, a non-``https`` ``resource:``, a
-path-traversal concept key, a reserved-name shadow, a dangerous frontmatter
+path-traversal concept key, an injection in a reserved ``index.md`` body, a
-value, a dangerous-scheme cross-link, a dangling cross-link, and a
+dangerous frontmatter value, a dangerous-scheme cross-link, a dangling
+cross-link, and a
 homoglyph-obfuscated body injection — run through the public ``okf`` surface
 exactly as an "upload inbox" consumer would compose it. Every planted surface is
 caught or rejected and the aggregate disposition fails secure.
@@ -73,8 +74,10 @@ def _poisoned_bundle() -> dict:
         "dangerous-frontmatter.md": "---\ntype: &anchor table\n---\nbody\n",
         # T4 — a path-traversal concept key escapes the bundle root.
         "../escape.md": "---\ntype: table\n---\nbody\n",
-        # T4 — a reserved basename shadows the directory listing.
+        # A2 — index.md is a legitimate structural file in a *received* bundle
-        "index.md": "---\ntype: table\ndescription: Listing.\n---\nbody\n",
+        # (not path-rejected in mode-b), but its body IS scanned: an injection
+        # planted in the directory listing is caught, not silently admitted.
+        "index.md": "---\ntype: table\ndescription: Listing.\n---\n" + _INJECTION + "\n",
         # LLM05 — the zero-click EchoLeak primitive: an auto-fetched markdown
         # image URL that exfiltrates the moment the concept is rendered.
         "echoleak.md": (
@@ -124,7 +127,7 @@ def _surface_checks(result: BundleResult) -> dict:
     desc = by_path.get("frontmatter-desc.md")
     return {
         "T4:path-traversal": rejected("../escape.md"),
-        "T4:reserved-name": rejected("index.md"),
+        "A2:index-body-scanned": caught("index.md", "override:ignore-previous"),
         "T3:resource": rejected("bad-resource.md"),
         "T2:frontmatter": rejected("dangerous-frontmatter.md"),
         "T1:body-injection": caught("body-injection.md", "override:ignore-previous"),