llm-ingestion-pipeline-security/docs/PLAN.md

Implementation Plan — llm-ingestion-guard v1

Derived from docs/BRIEF.md plus a 2026-07-04 research pass (competitive landscape, novelty/gap refutation, security-coverage gaps). This plan folds the verified gaps into the v1 scope. The BRIEF is the design rationale; this is the actionable build.

Resolved decisions (2026-07-04)

• Name: distribution llm-ingestion-guard, import llm_ingestion_guard. Repo directory stays llm-ingestion-pipeline-security.
• License: MIT.
• Lexicon storage: shared JSON data + thin Python loader (single source of truth, polyglot-ready for a later TS port).
• Publish target: Forgejo open/ only (public). No PyPI, no GitHub. Install via pip install git+ssh://…/llm-ingestion-guard.git.
• Language: Python-first, stdlib-only core; ML/judge detectors behind extras ([ml], [judge]), never in the core path.

Positioning — A: "Guard the artifact"

Query-time guardrails guard the answer. llm-ingestion-guard guards the artifact.

Lead with the contract + placement (write-path, pre-persist) + failure-semantics (fail-secure toward the artifact, not fail-open toward a user). Detection is the weakest, most-evadable layer — defense-in-depth, not the pitch.

Defensible claim, every qualifier load-bearing: a dependency-light, framework-agnostic library that packages the full write-time injection-containment contract (quarantine → per-stage capability isolation → scan-before-persist → fail-secure disposition) as composable code — the part query-time tooling and single-stage detectors leave to the integrator. Not "the first" (an unverifiable temporal claim); the load-bearing qualifier is the full four-part contract, verified against the 2026-07-15 survey (docs/BRIEF.md §11). Cite OWASP LLM08:2025 / RAG Security Cheat Sheet for legitimacy; reference Dual-LLM (Willison 2023) and CaMeL (DeepMind 2025) as architecture lineage — inspiration, not equivalence.

Claims we will NOT make (verified overclaim risks)

• NOT "prevents RAG/knowledge poisoning" — cannot catch factual poisoning (PoisonedRAG: plausible false facts, no markers, low entropy).
• NOT "blocks prompt injection" — pattern/entropy detection is bypassable (arXiv 2504.11168, up to 100% evasion). Detection is a signal feeding disposition, never a sole gate.
• NOT "invented" tool-less quarantine / capability isolation — prior art (Dual-LLM, CaMeL).
• NOT "the only" ingestion-security tool — PII-scoped libs (Philter, Presidio) and services (AWS, HiddenLayer) exist; we are the minimal-dep library for the injection-containment contract.
• Positioned as complementary to query-time guardrails (LLM Guard, Lakera, LlamaFirewall), not a replacement.

Module set (v1)

Core (BRIEF §5): report, sanitize, fence, lexicon, entropy, contract, output, disposition.

Gap additions folded into v1 (from the security-coverage research):

• neutralize (new) — opt-in pure defang helpers for model OUTPUT: markdown images, autolinks, [ref] links, data:/external URIs, raw HTML. Mutation is opt-in and kept separate from the report-only gate, so design principle 3 (pure detection) and 4 (disposition belongs to the caller) hold. Closes EchoLeak-class (CVE-2025-32711) persisted zero-click exfil — these are neither injection strings nor high-entropy, so lexicon+entropy alone miss them.
• output extended — secret/PII egress patterns (OWASP LLM02); decode-and-rescan (run the lexicon over decoded base64/hex, not just flag blob presence).
• fence hardened — randomized, unspoofable per-call delimiter; strip attacker fence markers from the payload first.
• Scanner self-safety (OWASP LLM10) — input-size cap, ReDoS-safe patterns (bounded quantifiers + per-scan guard), decompression guard. A scanner that hangs on crafted input is the DoS.
• grounding (interface now, impl later) — a SourceGroundingCheck protocol in core; ML/judge implementation behind the [judge] extra. The only structural handle on semantic poisoning that lexicon+entropy cannot see. Ship the seam + honest scope note.
• Chunk-aware / sliding-window scan — option for split-payload evasion across chunk boundaries.
• disposition presets — named source-trust tiers, including a high-untrust user-upload / open-source preset: QUARANTINE_REVIEW as the default and hard-fail on CRITICAL, not WARN. An automatic, unattended inbox that ingests arbitrary user uploads is the canonical high-untrust consumer (§4.7) — over-blocking one upload is far cheaper than persisting a poisoned one.

Build order (TDD — a failing test FIRST for each)

1. report — findings / Report dataclass (shared return type)
2. sanitize — carrier stripping (zero-width, BIDI, Unicode-tag, HTML comment, data:); byte-identical invariant on clean input, removes only
3. entropy — shannon / base64-like / hex-blob; decode-and-rescan support
4. lexicon — JSON data + loader + scan; ReDoS-safe; size cap; port from the llm-security JS seed (CRITICAL/HIGH/MEDIUM/HYBRID + normalization/homoglyph/rot13/ unicode-tag)
5. fence — randomized delimiter; marker-strip
6. neutralize — opt-in defang helpers
7. output — compose lexicon+entropy over output; PII/secret egress; decode-and-rescan
8. disposition — WARN | QUARANTINE_REVIEW | FAIL_SECURE; source-trust + intra-document provenance tiering (§4.7); compound-signal fail-secure (§4.6); fail-closed when the scanner itself errors
9. contract — tool-less assert; per-stage credential allowlist; env-scoping helpers
10. grounding — protocol/interface only in core
11. Top-level __init__ wiring the §6 checklist; pyproject.toml; LICENSE; README (positioning + honest limitations); adversarial + false-positive corpora; the end-to-end showcase pipeline test (see Test strategy — built LAST)

Reuse map — llm-security v7.8.0 (MIT, same author)

Maximal reuse: most detection logic is a JS→Python port, not new code.

Our module	Seed in llm-security	Notes
lexicon	scanners/lib/injection-patterns.mjs	CRITICAL/HIGH/MEDIUM/HYBRID tables + scanForInjection (variant-set: raw / normalized / homoglyph-folded / rot13, dedup by label) + checkCognitiveLoadTrap. The load-bearing port.
sanitize	scanners/unicode-scanner.mjs + string-utils.mjs	zero-width / BIDI / Unicode-tag / homoglyph detection + stripBidiOverrides / decodeUnicodeTags / containsUnicodeTags.
normalization	string-utils.mjs normalizeForScan	decode chain (unicode-tags → bidi → HTML-entities → unicode/hex/url escapes → base64), foldHomoglyphs, rot13, collapseLetterSpacing. Feeds decode-and-rescan.
entropy	scanners/entropy-scanner.mjs + string-utils.mjs	shannonEntropy / isBase64Like / isHexBlob / tryDecodeBase64; length-calibrated thresholds; isFalsePositive suppression rules; DATA_URI_PREFIXES.
report	scanners/lib/output.mjs + severity.mjs	finding / scannerResult envelope; SEVERITY; riskScore / verdict / riskBand / owaspCategorize.
output secret/PII egress	knowledge/secrets-patterns.md	Ready secret regex set (AWS/Azure/GCP/GitHub/npm/OpenAI/Anthropic/PEM/DB-conn/password/JWT) + FP-suppression rules. Seeds the LLM02 egress gap directly.
tests (corpora)	examples/prompt-injection-showcase/payloads.json	Labeled adversarial + false-positive corpus (category/severity/expected). Maps ~1:1 to §9. Plus knowledge/{attack-mutations,attack-scenarios,signatures}.json.
wiring reference	hooks/pre-prompt-inject-scan.mjs (input), post-mcp-verify.mjs (output gate)	How the scan is invoked at input vs output — informs our output module + the §6 checklist.

Port caveats (JS→Python, stdlib-only):

• /…/i → re.I; Buffer.from(s,'base64') → base64.b64decode; codePointAt/fromCodePoint → ord/chr.
• TOKEN_RE = /[\p{L}\p{N}_]+/gu — Python re has no \p{L}; use \w under re.UNICODE (Py3 default) or str.isalnum(). No regex dependency.
• ReDoS: some patterns (sub-agent spawn with nested .*?) backtrack; Python re has no timeout → enforce input-size cap + review/bound these patterns (the self-safety must-have).
• Lexicon ships as JSON (regex source + label + severity), compiled by a thin Python loader — decouples data from engine for the future TS port.

Test strategy (BRIEF §9 + additions)

• Adversarial corpus — one payload per lexicon class incl. obfuscated + multi-language; measure and report recall.
• False-positive corpus — content legitimately discussing injection (security docs, changelogs); assert WARN-not-block default; hard-fail is an explicit opt-in.
• Sanitizer invariant — clean input returns byte-identical with an all-zero report.
• Contract asserters — a tool-carrying request and a credential-leaking stage env both raise; the happy path passes.
• Self-safety — pathological/ReDoS-prone and oversize input return within a bound, never hang. Scope, measured 2026-07-31: the lexicon path is covered by test_redos_pathological_subagent_input_returns_fast (crafted against a known-bad nested .*?). The output path is covered by test_crafted_redos_payload_stays_bounded — 15 crafted payloads plus one through the composed gate. Writing them found the defect they were meant to rule out: 19 quadratic runs across 17 patterns in output (4), active_content (5 patterns / 7 runs) and the lexicon JSON (8), worst case ~5.7 hours at the 1_000_000-char cap the gate accepts. Note the shape, since it is not the textbook one: no nested quantifier is involved. A run in front of a required literal, reachable from a short anchor, is enough — crafted input repeats the anchor, never supplies the literal, and every start position rescans the tail. The earlier "output blob is slower than ordinary prose (0.93x/0.96x)" measurement stands and was never wrong; it simply measured throughput on a blob, which is a different question from what a crafted payload asks.
• Neutralize — active-content output is defanged; clean output is byte-identical.
• End-to-end showcase (the FINAL deliverable, built last). One realistic piece of ingested content that carries many vulnerabilities at once — visible
  • zero-width/BIDI/Unicode-tag stego carriers, homoglyph + whole-string-base64 + rot13-hidden injection, HTML-obfuscated and cognitive-load-buried payloads, secret/PII egress in the enriched output, and active-content exfil links — run through a mock ingestion pipeline (sanitize → lexicon+entropy+decode-rescan → output gate → disposition). Assert every planted vulnerability is caught and disposition fails secure. Doubles as the README's worked example. Inspiration: llm-security/examples/{prompt-injection-showcase,poisoned-claude-md,lethal-trifecta-walkthrough,toxic-agent-demo}.
• No network in any test.

README "Honest limitations" (shipped as a control)

Concede plainly (the concession prevents false assurance, which is itself a control): structural unsolvability at the text layer; adversarial-ML evasion survives normalization; tokenizer mismatch; semantic/factual poisoning invisible to lexicon+entropy; latent/dormant memory poisoning not judgeable at write-time; insider in-place edits; text-only (no multimodal).

Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal steganography; query-time/runtime guardrails; semantic factuality verification. The contract tool-less assertion is the write-time analogue of runtime least-privilege.

Text-extraction boundary. For pipelines that ingest arbitrary user uploads (an upload inbox that ingests automatically), the library stays text -> findings: it does not parse files (no pypdf/python-docx/archive deps in the core). The pipeline extracts text first, then scans the extracted text + the enriched output with provenance tagged as the high-untrust upload tier. Binary/multimodal carrier detection (OCR-embedded instructions in images/PDFs, stego) is out of scope beyond the sanitizer's character-layer stripping and the self-safety size/decompression guard — and must be conceded in the honest-limitations section, because a high-untrust, unattended pipeline is exactly where assuming uncovered-coverage is most dangerous.

Threat-model anchors

OWASP LLM Top-10 2025: LLM01, LLM02, LLM04, LLM05, LLM06 (strongest coverage), LLM08 (boundary), LLM09, LLM10. Research anchors: PoisonedRAG (arXiv 2402.07867), guardrail evasion (2504.11168), EchoLeak (CVE-2025-32711), RAGShield (2604.00387), CaMeL (2503.18813), Dual-LLM (Willison), and the litellm supply-chain compromise (corroborates the minimal-dependency thesis).

v0.2+ stream sequencing (revised 2026-07-06)

v0.1.0 and v0.2.0 are tagged: the format-agnostic text core (modules 1–11, 5397ba1 / released df30c7b) and the OKF adapter (stream 1, released 542ac92). The forward order was re-sequenced on 2026-07-06 after a ground-truth pass over the intended flagship consumer.

Finding that drove the change. The named flagship — portfolio-optimiser's "OKF-upload-inbox" — does not exist as a seam. Both optimiser siblings are frozen at their release milestone, carry their own OKF layer (navigate + materialize from trusted manifests), receive no external bundles, and take no dependency on this guard; the operator's own registered future work for them does not include it. Wiring an immature guard into two mature, spec-frozen repos would complicate them for a consumer that is not asking for it. Consumer integration is therefore deferred until the guard is mature; consumers are informed at the operator's timing, not pushed.

Revised streams:

1. OKF v0.2 hardening — shipped (adapter + brief §8 tasks; T5b/B deferred to a consumer that owns corpus storage).
2. Mature the guard here, keeping it Node-port-friendly — the near-term work. The lexicon is already shared JSON (polyglot-ready); keep the text -> findings surface clean and free of Python-only cleverness in the OKF layer, so stream 3 is a translation, not a redesign. The centrepiece is the OKF inbox showcase (below): the guard demonstrating its own flagship use case end-to-end, in-repo.
3. Node/polyglot port — the strategic enabler of painless integration (many OKF consumers are Node/JS; the lexicon seed was .mjs). One polyglot repo over the shared JSON lexicon; never split §13.3. Its API/lexicon contract should be scan-informed (stream 4), not guessed.
4. Pre-adaptation scan (operator-timed) — at the operator's chosen point, scan every repo and plugin that uses or plans Google OKF, then adapt the guard's surface in advance so a later integration is painless. The scan is the input to the "painless" guarantee: it grounds both the Node port's contract and any consumer-specific seams before they are locked.

Re-sequenced toward v1.0 (Fable cross-model review, 2026-07-09)

An independent cross-model review re-sequenced the forward path. Stream 2 is extended with a set of review-injected hardening + coverage sessions (output-gate coverage, OKF adapter hardening, egress decode-rescan, calibration-threshold consolidation, docs/version-sync, and a novelty-claim verification) that land before a v1.0 freeze. The Node/TS port (stream 3) starts only after that freeze — porting a frozen, scan-clean surface, never a moving target — over the shared JSON lexicon (never split). Streams 4 + consumer integration are operator-timed "ambitious extensions", not part of the v1.0 sequence.

The session-by-session plan and the review record live in docs/ on the Forgejo open/ mirror alongside the other design docs — docs/PLAN-v1.md (the v1.0 + Node session plan) and docs/review-2026-07.md (the review) — open/ being the sole sanctioned public surface (never GitHub).

The OKF inbox showcase (next concrete build, TDD)

The flagship artifact we hand a consumer later — an in-repo end-to-end demonstration of the mode-b receive/quarantine gate, mirroring tests/test_showcase.py but for a received external OKF bundle. It composes the public okf surface exactly as an "upload inbox" consumer would, so it doubles as the README's OKF worked example. Every test is authored by us — the point is to prove intent, not to coincidentally pass.

• Composition (tests/test_okf_showcase.py): an _inbox(bundle) helper calling okf.import_bundle(bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC) and mapping the aggregate disposition — WARN → ADMIT, QUARANTINE_REVIEW → HOLD, FAIL_SECURE → REJECT, any error → REJECT (fail-secure default).
• One poisoned bundle planting one attack per OKF surface at once, each with a label proving it was caught/rejected:
  • body injection (T1 scan) and frontmatter description injection (T1 whole-concept scan);
  • a non-https resource: URL (T3 → FAIL_SECURE);
  • a path-traversal concept key ../x.md (T4 → FAIL_SECURE) and a reserved-name shadow index.md / log.md (T4);
  • a dangerous frontmatter value (anchor / alias / explicit tag, T2 → FAIL_SECURE);
  • a dangerous-scheme cross-link [x](javascript:…) (T5a → links.rejected);
  • a dangling cross-link to an absent concept (§7.2 dormant signal → links.dangling);
  • a carrier/obfuscation-hidden body injection (zero-width / homoglyph / rot13 / whole-string base64) routed through scan_concept.
• Assertions: aggregate disposition is FAIL_SECURE; every planted OKF vulnerability appears in the per-concept rejects / findings / link graph; a clean bundle admits (WARN, no rejects, no dangling); BundleResult.log() renders one line per concept with rejects marked. Detach-proof: neutering import_bundle to always-admit makes the poison assertions fail.
• Honest scope: demonstrates the structural + known-pattern OKF surface only — semantic/factual poisoning stays out (README honest-limits), consistent with the core showcase.

Realistic upload formats — the two-stage inbox (extract → materialize → guard)

A human inbox does not receive tidy {path: text} dicts; it receives the files people actually drop: .txt, .md, Word .docx, Excel .xlsx, PowerPoint .pptx, .pdf, .csv, whole folders, and .zip archives. The showcase therefore has two stages, honouring the locked text-extraction boundary (§ "Text-extraction boundary"): the guard core never grows a file parser.

1. Extract & materialize (the inbox front-end). Reads each dropped file, walks folders, and safely unpacks archives; extracts text via format libraries (python-docx, openpyxl, python-pptx; stdlib zipfile/csv); materializes the result into an OKF bundle {concept_path: text} with provenance (origin=EXTERNAL, source filename + type). This stage owns the container and format threats. These libraries are showcase/dev-scoped only — never core dependencies (which stays []). Promotion to an optional [extract] extra is a documented future option if a consumer wants turnkey extraction, not v1.
2. Guard (import_bundle). Scans every extracted concept + the OKF structural gates → aggregate disposition. This stage owns the text/structural threats.

One representative planted vector per format (the point is where the payload hides — the place a human does not look):

Input	Hidden vector planted	Caught at
.txt	raw injection + carrier (zero-width / homoglyph / rot13 / base64)	guard scan
.md	OKF frontmatter attack (T2) + body injection (T1)	guard
.docx	injection in a comment / hidden (vanish) run / core metadata property	guard scan of extracted text
.xlsx	formula injection (=cmd|'…', =HYPERLINK(…)) + hidden sheet / cell comment	front-end + guard
.pptx	injection in speaker notes / off-slide text box / image alt-text	guard scan
.pdf	injection in extracted / white-on-white text	conceded — refused as an unsupported format, not extracted (see honest-scope below)
.csv	formula injection (=, +, -, @ lead)	front-end + guard
folder	the OKF bundle-directory shape directly; one member path trips the path gate (T4)	guard
.zip	zip-slip entry ../../x.md (maps straight onto OKFPathError, T4) + zip-bomb (bounded by the size cap / a safe-extract limit, LLM10) + symlink entry	front-end + guard

Assertions: each planted vector is caught at the correct stage; the front-end refuses zip-slip / zip-bomb / oversize fail-fast; the guard rejects injection/carrier/frontmatter/resource/link/path; a clean file of every accepted format admits (WARN). Detach-proof at both stages.

Honest scope for uploads (must be conceded — README honest-limits). What survives text extraction is out of scope: VBA/macros (.docm/.xlsm/.pptm), OLE / embedded objects, image-embedded instructions that need OCR, font/render steganography, and encrypted/password-protected files. The guard scans extracted text; binary-layer carriers need a separate scanner. .pdf is conceded as a format: a top-level .pdf drop is refused as unsupported rather than extracted, since a PDF parser + reportlab (fixtures only) is disproportionate for a dev-scoped showcase and its OCR / font-render stego carriers are out of scope regardless. A high-untrust, unattended inbox is exactly where assuming uncovered-coverage is most dangerous (§4.7), so the concession is itself a control.

Deferred, unchanged: T5b/B (the persisted cross-run link graph) waits for a consumer that owns corpus storage; §7.2 dormant cross-run links remain a documented residual until then.

Splittable early win (optional): text-only consumers can be wired at v0.1.0 today — for them the guard is already complete. Deferred with stream 2 by default; pull forward only on explicit request.