llm-ingestion-pipeline-security/docs/PLAN.md

Implementation Plan — llm-ingestion-guard v1

Derived from docs/BRIEF.md plus a 2026-07-04 research pass (competitive landscape, novelty/gap refutation, security-coverage gaps). This plan folds the verified gaps into the v1 scope. The BRIEF is the design rationale; this is the actionable build.

Resolved decisions (2026-07-04)

• Name: distribution llm-ingestion-guard, import llm_ingestion_guard. Repo directory stays llm-ingestion-pipeline-security.
• License: MIT.
• Lexicon storage: shared JSON data + thin Python loader (single source of truth, polyglot-ready for a later TS port).
• Publish target: Forgejo open/ only (public). No PyPI, no GitHub. Install via pip install git+ssh://…/llm-ingestion-guard.git.
• Language: Python-first, stdlib-only core; ML/judge detectors behind extras ([ml], [judge]), never in the core path.

Positioning — A: "Guard the artifact"

Query-time guardrails guard the answer. llm-ingestion-guard guards the artifact.

Lead with the contract + placement (write-path, pre-persist) + failure-semantics (fail-secure toward the artifact, not fail-open toward a user). Detection is the weakest, most-evadable layer — defense-in-depth, not the pitch.

Defensible claim, every qualifier load-bearing: the first dependency-light, framework-agnostic library that packages the write-time injection-containment contract with fail-secure disposition, for unattended pipelines. Cite OWASP LLM08:2025 / RAG Security Cheat Sheet for legitimacy; reference Dual-LLM (Willison 2023) and CaMeL (DeepMind 2025) as architecture lineage — inspiration, not equivalence.

Claims we will NOT make (verified overclaim risks)

• NOT "prevents RAG/knowledge poisoning" — cannot catch factual poisoning (PoisonedRAG: plausible false facts, no markers, low entropy).
• NOT "blocks prompt injection" — pattern/entropy detection is bypassable (arXiv 2504.11168, up to 100% evasion). Detection is a signal feeding disposition, never a sole gate.
• NOT "invented" tool-less quarantine / capability isolation — prior art (Dual-LLM, CaMeL).
• NOT "the only" ingestion-security tool — PII-scoped libs (Philter, Presidio) and services (AWS, HiddenLayer) exist; we are the minimal-dep library for the injection-containment contract.
• Positioned as complementary to query-time guardrails (LLM Guard, Lakera, LlamaFirewall), not a replacement.

Module set (v1)

Core (BRIEF §5): report, sanitize, fence, lexicon, entropy, contract, output, disposition.

Gap additions folded into v1 (from the security-coverage research):

• neutralize (new) — opt-in pure defang helpers for model OUTPUT: markdown images, autolinks, [ref] links, data:/external URIs, raw HTML. Mutation is opt-in and kept separate from the report-only gate, so design principle 3 (pure detection) and 4 (disposition belongs to the caller) hold. Closes EchoLeak-class (CVE-2025-32711) persisted zero-click exfil — these are neither injection strings nor high-entropy, so lexicon+entropy alone miss them.
• output extended — secret/PII egress patterns (OWASP LLM02); decode-and-rescan (run the lexicon over decoded base64/hex, not just flag blob presence).
• fence hardened — randomized, unspoofable per-call delimiter; strip attacker fence markers from the payload first.
• Scanner self-safety (OWASP LLM10) — input-size cap, ReDoS-safe patterns (bounded quantifiers + per-scan guard), decompression guard. A scanner that hangs on crafted input is the DoS.
• grounding (interface now, impl later) — a SourceGroundingCheck protocol in core; ML/judge implementation behind the [judge] extra. The only structural handle on semantic poisoning that lexicon+entropy cannot see. Ship the seam + honest scope note.
• Chunk-aware / sliding-window scan — option for split-payload evasion across chunk boundaries.
• disposition presets — named source-trust tiers, including a high-untrust user-upload / open-source preset: QUARANTINE_REVIEW as the default and hard-fail on CRITICAL, not WARN. An automatic, unattended inbox that ingests arbitrary user uploads is the canonical high-untrust consumer (§4.7) — over-blocking one upload is far cheaper than persisting a poisoned one.

Build order (TDD — a failing test FIRST for each)

1. report — findings / Report dataclass (shared return type)
2. sanitize — carrier stripping (zero-width, BIDI, Unicode-tag, HTML comment, data:); byte-identical invariant on clean input, removes only
3. entropy — shannon / base64-like / hex-blob; decode-and-rescan support
4. lexicon — JSON data + loader + scan; ReDoS-safe; size cap; port from the llm-security JS seed (CRITICAL/HIGH/MEDIUM/HYBRID + normalization/homoglyph/rot13/ unicode-tag)
5. fence — randomized delimiter; marker-strip
6. neutralize — opt-in defang helpers
7. output — compose lexicon+entropy over output; PII/secret egress; decode-and-rescan
8. disposition — WARN | QUARANTINE_REVIEW | FAIL_SECURE; source-trust + intra-document provenance tiering (§4.7); compound-signal fail-secure (§4.6); fail-closed when the scanner itself errors
9. contract — tool-less assert; per-stage credential allowlist; env-scoping helpers
10. grounding — protocol/interface only in core
11. Top-level __init__ wiring the §6 checklist; pyproject.toml; LICENSE; README (positioning + honest limitations); adversarial + false-positive corpora; the end-to-end showcase pipeline test (see Test strategy — built LAST)

Reuse map — llm-security v7.8.0 (MIT, same author)

Maximal reuse: most detection logic is a JS→Python port, not new code.

Our module	Seed in llm-security	Notes
lexicon	scanners/lib/injection-patterns.mjs	CRITICAL/HIGH/MEDIUM/HYBRID tables + scanForInjection (variant-set: raw / normalized / homoglyph-folded / rot13, dedup by label) + checkCognitiveLoadTrap. The load-bearing port.
sanitize	scanners/unicode-scanner.mjs + string-utils.mjs	zero-width / BIDI / Unicode-tag / homoglyph detection + stripBidiOverrides / decodeUnicodeTags / containsUnicodeTags.
normalization	string-utils.mjs normalizeForScan	decode chain (unicode-tags → bidi → HTML-entities → unicode/hex/url escapes → base64), foldHomoglyphs, rot13, collapseLetterSpacing. Feeds decode-and-rescan.
entropy	scanners/entropy-scanner.mjs + string-utils.mjs	shannonEntropy / isBase64Like / isHexBlob / tryDecodeBase64; length-calibrated thresholds; isFalsePositive suppression rules; DATA_URI_PREFIXES.
report	scanners/lib/output.mjs + severity.mjs	finding / scannerResult envelope; SEVERITY; riskScore / verdict / riskBand / owaspCategorize.
output secret/PII egress	knowledge/secrets-patterns.md	Ready secret regex set (AWS/Azure/GCP/GitHub/npm/OpenAI/Anthropic/PEM/DB-conn/password/JWT) + FP-suppression rules. Seeds the LLM02 egress gap directly.
tests (corpora)	examples/prompt-injection-showcase/payloads.json	Labeled adversarial + false-positive corpus (category/severity/expected). Maps ~1:1 to §9. Plus knowledge/{attack-mutations,attack-scenarios,signatures}.json.
wiring reference	hooks/pre-prompt-inject-scan.mjs (input), post-mcp-verify.mjs (output gate)	How the scan is invoked at input vs output — informs our output module + the §6 checklist.

Port caveats (JS→Python, stdlib-only):

• /…/i → re.I; Buffer.from(s,'base64') → base64.b64decode; codePointAt/fromCodePoint → ord/chr.
• TOKEN_RE = /[\p{L}\p{N}_]+/gu — Python re has no \p{L}; use \w under re.UNICODE (Py3 default) or str.isalnum(). No regex dependency.
• ReDoS: some patterns (sub-agent spawn with nested .*?) backtrack; Python re has no timeout → enforce input-size cap + review/bound these patterns (the self-safety must-have).
• Lexicon ships as JSON (regex source + label + severity), compiled by a thin Python loader — decouples data from engine for the future TS port.

Test strategy (BRIEF §9 + additions)

• Adversarial corpus — one payload per lexicon class incl. obfuscated + multi-language; measure and report recall.
• False-positive corpus — content legitimately discussing injection (security docs, changelogs); assert WARN-not-block default; hard-fail is an explicit opt-in.
• Sanitizer invariant — clean input returns byte-identical with an all-zero report.
• Contract asserters — a tool-carrying request and a credential-leaking stage env both raise; the happy path passes.
• Self-safety — pathological/ReDoS-prone and oversize input return within a bound, never hang.
• Neutralize — active-content output is defanged; clean output is byte-identical.
• End-to-end showcase (the FINAL deliverable, built last). One realistic piece of ingested content that carries many vulnerabilities at once — visible
  • zero-width/BIDI/Unicode-tag stego carriers, homoglyph + whole-string-base64 + rot13-hidden injection, HTML-obfuscated and cognitive-load-buried payloads, secret/PII egress in the enriched output, and active-content exfil links — run through a mock ingestion pipeline (sanitize → lexicon+entropy+decode-rescan → output gate → disposition). Assert every planted vulnerability is caught and disposition fails secure. Doubles as the README's worked example. Inspiration: llm-security/examples/{prompt-injection-showcase,poisoned-claude-md,lethal-trifecta-walkthrough,toxic-agent-demo}.
• No network in any test.

README "Honest limitations" (shipped as a control)

Concede plainly (the concession prevents false assurance, which is itself a control): structural unsolvability at the text layer; adversarial-ML evasion survives normalization; tokenizer mismatch; semantic/factual poisoning invisible to lexicon+entropy; latent/dormant memory poisoning not judgeable at write-time; insider in-place edits; text-only (no multimodal).

Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal steganography; query-time/runtime guardrails; semantic factuality verification. The contract tool-less assertion is the write-time analogue of runtime least-privilege.

Text-extraction boundary. For pipelines that ingest arbitrary user uploads (an upload inbox that ingests automatically), the library stays text -> findings: it does not parse files (no pypdf/python-docx/archive deps in the core). The pipeline extracts text first, then scans the extracted text + the enriched output with provenance tagged as the high-untrust upload tier. Binary/multimodal carrier detection (OCR-embedded instructions in images/PDFs, stego) is out of scope beyond the sanitizer's character-layer stripping and the self-safety size/decompression guard — and must be conceded in the honest-limitations section, because a high-untrust, unattended pipeline is exactly where assuming uncovered-coverage is most dangerous.

Threat-model anchors

OWASP LLM Top-10 2025: LLM01, LLM02, LLM04, LLM05, LLM06 (strongest coverage), LLM08 (boundary), LLM09, LLM10. Research anchors: PoisonedRAG (arXiv 2402.07867), guardrail evasion (2504.11168), EchoLeak (CVE-2025-32711), RAGShield (2604.00387), CaMeL (2503.18813), Dual-LLM (Willison), and the litellm supply-chain compromise (corroborates the minimal-dependency thesis).