llm-ingestion-pipeline-security/docs/OKF-INGESTION-BRIEF.md

Security brief: OKF ingestion with llm-ingestion-guard

Status: Draft v0.1 — captured 2026-07-06. Applies to: write-time ingestion pipelines for LLM wikis based on Google's Open Knowledge Format (OKF v0.1). Inputs: internal and external data, ingested automatically or manually.

This document is a future-work brief, not shipped scope. The v0.1 core (docs/PLAN.md, all modules built + hardened at commit 5397ba1) is format-agnostic text (prepare_input(text) / screen_output(text, policy)). None of the OKF-specific controls below are implemented yet; §8 is the v0.2 task list. The coverage claims in §4 were verified against the code on 2026-07-06 — see the verification log in §9.

---

1. Summary

OKF is, in practice, exactly the "LLM-wiki" pattern llm-ingestion-guard was built to protect: untrusted content is enriched/persisted into a markdown corpus that a downstream agent later reads as trusted context. The guard therefore belongs at the persist gate, and the existing contract (carrier-strip → fence → tool-less transform → capability isolation → output scan → fail-secure) covers the markdown body directly.

OKF, however, introduces five attack surfaces beyond the body that must be covered explicitly: YAML frontmatter, resource URLs, the cross-link graph, file path / reserved names, and the absence of format-level authenticity. The largest residual risk is semantic poisoning, which is out of scope by design (§7).

2. Scope

In scope: everything written into or merged into an OKF bundle that the pipeline controls — both its own enrichment output and received external bundles.

Out of scope: query-time / runtime guardrails, the vector/embedding layer, and semantic truth-assessment of concepts (see §7). These match the existing out-of-scope boundary in README.md and docs/PLAN.md.

3. Why OKF is the right place to enforce

• OKF formalizes the LLM-wiki pattern into a portable markdown format; concepts are meant to be read directly by agents as curated, authoritative context.
• The format has no schema registry, no central authority, and no signing/authenticity. A received bundle's claimed origin is not verifiable at the format level. (Verified at the format level — OKF v0.1 is a directory of markdown files with YAML frontmatter, minimal by design; §9.)
• Consequence: your ingestion pipeline is the trust boundary. Provenance and disposition must be stamped by you at ingestion, not assumed from the format.

4. Attack surfaces and required controls

Status column verified against the code at commit 5397ba1 on 2026-07-06 (§9).

Surface	Vector	Required control	Status today
Markdown body	Prompt injection, hidden carriers (zero-width, BIDI, Unicode-tag, HTML comments, data: URIs)	Carrier-strip → fence → tool-less transform → output scan → fail-secure	✅ Covered (core contract)
YAML frontmatter	Injection in title/description/tags + arbitrary unknown keys; description propagates into index.md (read first under progressive disclosure); YAML anchor/alias DoS + dangerous type coercion	Same sanitize/scan on frontmatter values; parse YAML with a safe loader	⚠️ Partial — values are scanned iff the caller passes the whole document (frontmatter included) as text; the core never parses YAML, so the safe-loader is a genuinely new gate at an OKF-adapter boundary (§9)
resource URL	data:/javascript:/file:/SSRF target that a consumer or visualizer fetches	Scheme allowlist (https only), validate before commit	❌ New control — neutralize defangs schemes for human audit but there is no reject-gate and no resource-field concept (§9)
Cross-link graph	"Dead links are valid" → dormant injection: plant a link to a non-existent concept-ID now, write the poisoned target later	Resolve in-bundle .md links to concept-IDs; flag dangling links (the dormant-injection signal); reject dangerous-scheme / bundle-escaping targets. Absolute external https URLs and references/ paths are spec-permitted link targets, not rejected. Re-scan on write of a link target	❌ New control (graph level)
File path / reserved names	Concept-ID = file path minus .md; path traversal (../) and shadowing of reserved index.md/log.md	Sanitize/normalize paths; reject .. and reserved filenames as concept names	❌ New control — no path validation in the core (§9)
log.md / provenance	No authenticity at the format level	Stamp disposition + trust tier per concept	↔️ Machinery exists — Trust × Provenance × Disposition types are built; emission-to-log.md + an origin/channel stamp is new wiring on top (§9)

5. Trust-tier model

Trust follows the data's origin, not the insertion channel. A manual paste of external material is still external. This maps onto the existing Trust enum (TRUSTED / UNTRUSTED) plus a channel dimension the pipeline records.

Origin	Channel	Tier	Treatment
External	Automatic	Lowest	Full contract + fail-secure
External	Manual	Low	Full contract (channel grants no discount)
Internal	Automatic	Medium	Full contract; may relax the lexicon threshold with logging
Internal	Manual (trusted author)	Highest	Keep carrier-strip + scan; note residual risk §7

Nuance: "manual internal" is not safe in itself. In-place edits by a trusted author are outside the guard's scope — the channel does not authenticate the content's origin. This is the same insider-edit limitation already in README.md honest-limitations.

6. Two ingestion modes

(a) Own enrichment output — your agent writes concepts. The guard runs on your own output before commit. One transform per concept. This is the existing prepare_input → tool-less transform → screen_output bookend flow, applied per concept.

(b) Received external bundle — you merge a whole third-party OKF bundle (the format is explicitly built for cross-organization exchange). The guard must iterate over each concept and validate body + frontmatter + links + resource before merge — not treat the bundle as one unit. This is a new iterator layer (§8 T7); the core has no bundle concept.

7. Residual risk (explicitly out of scope)

Cross-referenced to README.md honest-limitations. Items 1, 3, 5 are already documented there; items 2 and 4 are OKF-specific and not yet surfaced (→ §8 T8).

1. Semantic / factual poisoning — a plausible-but-wrong concept (wrong join-path, wrong metric definition, wrong runbook step) passes entropy + lexicon cleanly and is treated as ground truth. Highest impact for a wiki. Needs human review or source verification, not token analysis. (Already in README; the grounding module ships the SourceGroundingCheck seam for it.)
2. Dormant broken-link injection — see §4; a per-concept write-time scan does not see it over time. Needs graph / re-scan logic. (Not yet in README.)
3. Trusted-author in-place edits — normal OKF workflow (git-native editing) is not gated by the guard. (Already in README as the insider-edit limitation.)
4. Own security content as a false positive — concepts that document prompt-injection payloads will trip the carrier-strip / fail-secure path. Needs a deliberate escaped path for "this concept is about attacks." (Not yet in README; also tracked as the corpus false-positive tension.)
5. Multimodal stego, embedding layer, query-time — refer to the existing threat-model docs. (Already in README / out-of-scope boundary.)

8. Recommendations → v0.2 task list

The unifying design move: add an OKF adapter on top of the format-agnostic core. The core stays text -> findings (design principle 3); the adapter knows OKF structure (frontmatter, paths, links, resource, bundles) and feeds scannable text regions into the existing sanitize / scan_output / disposition machinery. No YAML/format awareness leaks into the core.

#	Task	Builds on	New?
T1	Whole-concept scan surface — route frontmatter values + resource strings through the same sanitize/scan_output path as the body	sanitize, output	adapter layer
T2	YAML parse-safety gate — safe loader (block anchor/alias DoS + dangerous type coercion) at the adapter boundary	—	new
T3	resource-URL allowlist gate — hard-reject non-https (and data:/javascript:/file:) before commit	neutralize scheme detection	new (reject, not defang)
T4	Path / reserved-name validation — reject .. traversal + index.md/log.md shadowing; normalize concept-ID → path	—	new
T5	Cross-link graph re-scan — constrain targets to relative in-bundle paths + scheme check; re-scan on write of a link target (dormant-injection, §7.2)	—	new (graph state)
T6	Provenance stamping — origin + channel → tier + disposition per concept, emitted to log.md	Trust/Provenance/Disposition	new emission on existing types
T7	Bundle-import iterator (mode b) — iterate concepts; validate body + frontmatter + links + resource per concept	§6 bookends	new
T8	Docs — surface §7.2 (dormant broken-link) + §7.4 (own security content) in README honest-limitations; §7.1/§7.3/§7.5 already there	README.md	doc

Sequencing note: T1–T4 + T6 are per-concept and composable with the existing bookends; T7 is the mode-(b) wrapper around them; T5 is the hardest (needs graph state across writes) and the one the residual risk §7.2 explicitly warns a per-concept scan cannot cover. T8 is cheap and can land independently.

Naming caveat: the illustrative stamp vocabulary in an earlier draft (SANITIZED/FENCED/QUARANTINED/FAIL_SECURE) is not the actual Disposition enum (which is WARN / QUARANTINE_REVIEW / FAIL_SECURE). A provenance stamp (T6) would compose from Trust × Provenance × Disposition, not add new disposition values.

9. Verification log (2026-07-06, commit 5397ba1)

Per the operator's verification duty — what was checked against ground truth, and what remains the operator's premise.

Verified against the code:

• The guard is format-agnostic text. Public API is prepare_input(text) / screen_output(text, policy) (src/llm_ingestion_guard/__init__.py). A grep of src/ finds no YAML import and no frontmatter parsing anywhere. Implication for the §4 "YAML frontmatter" row: frontmatter values are scanned only if the caller includes them in the text passed to the scanner; the safe-loader is a genuinely new gate, because the core never parses YAML.
• resource URL — no reject-gate today. neutralize.py recognizes dangerous schemes (javascript|data|vbscript|file|blob defanged; http(s)/ftp → hxxp) but this is neutralization for human audit, not an allowlist that rejects non-https before commit, and there is no resource-field concept. §4 "New control" confirmed.
• Cross-link graph, file path / reserved names — absent. No graph logic, no path normalization / traversal check / reserved-name (index.md/log.md) check in src/. §4 "New control" confirmed for both.
• Provenance machinery exists. disposition.py defines Disposition (WARN/QUARANTINE_REVIEW/FAIL_SECURE), Trust (TRUSTED/UNTRUSTED), Provenance (PROSE/CODE_FENCE/LOCALIZED) + presets. Per-concept stamping to log.md is new emission on top; the types are ready. §4 "Machinery exists" confirmed.
• README already covers residual risks 1, 3, 5 (semantic poisoning + grounding seam; insider in-place edits; multimodal/query-time). Residual risks 2 and 4 are not yet in README → T8.

Verified externally:

• OKF is real, format level. Google Cloud's Open Knowledge Format v0.1, announced 2026-06-12: a directory of markdown files with YAML frontmatter, minimal by design (only a type field required), vendor-neutral, producer/consumer-independent — which confirms the LLM-wiki framing, the cross-org exchange mode (b), and the "no central authority / no signing" premise in §1/§3. Spec: GoogleCloudPlatform/knowledge-catalog/okf/SPEC.md.

Operator premise — not independently verified this session:

• OKF concept-level specifics used above (concept-ID = file path minus .md; reserved index.md/log.md; a resource field; description propagating into index.md under progressive disclosure) are taken from the operator's brief citing the OKF spec. They are consistent with the format-level facts but were not each checked against SPEC.md. A focused pass over the OKF spec should confirm the reserved-filename and resource-field details before T3/T4/T5 are built.

Sources

• Google Cloud — How the Open Knowledge Format can improve data sharing
• GoogleCloudPlatform/knowledge-catalog — okf/SPEC.md
• MarkTechPost — Google Cloud Introduces Open Knowledge Format (OKF)