llm-ingestion-pipeline-security/docs/BRIEF.md

Brief: llm-ingestion-pipeline-security

A reusable, minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails.

Status: brief / pre-implementation. This document defines what the repo should contain and why. No code yet.

---

1. One-line purpose

Give any pipeline that runs untrusted content through a large language model and then persists the result into a downstream-consumed artifact (RAG corpus, knowledge base, wiki, embeddings store) a small, framework-agnostic library that packages the architectural contract for doing it safely: sanitize → fence → tool-less quarantined transform → capability isolation → scan output before commit → fail-secure.

2. The problem, stated precisely

The mature tooling for prompt injection is query-time: it sits between a user (or an agent) and a model, at inference, on live traffic. Ingestion is a different shape and is underserved:

• Direction. The danger flows from content (documents, changelogs, scraped web pages, user uploads, tickets) through an LLM enrichment/summarization/ extraction/classification step, into a persisted artifact that other systems and agents later read.
• Two distinct failure modes, only the first of which query-time guardrails address:
  1. Injection steers the enrichment step — untrusted content contains instructions that hijack the summarizer/extractor.
  2. Poisoned content is published — the artifact itself becomes the attack. A verbatim-quoted payload, or an LLM-emitted instruction, lands in the knowledge base and later poisons a downstream agent's context. This is RAG/memory poisoning committed at write time, and a query-time guardrail on the downstream reader never sees where it came from.
• Non-interactive, unattended. Ingestion typically runs headless on a schedule. There is no human in the loop to notice a weird answer. Failure discipline (fail-secure, alerting, no silent verbatim fallback under attack) matters more here than in a chat UI.

3. Prior art and honest positioning

Injection detection is a crowded space. This repo must not pretend otherwise.

Tool	Orientation
LLM Guard, Guardrails AI, NeMo Guardrails	Query-time I/O guardrails / dialogue policy
Rebuff	Self-hardening query-time detector (vector store of past attacks)
Vigil, Lakera Guard, Vijil	Query-time injection classifiers / detection servers
LlamaFirewall (Meta)	Agent guardrail framework (I/O, tool calls)
Resk	Python guardrail lib for LLM API calls
CleanBase, DataFilter (research)	Detecting/cleaning malicious docs in RAG DBs

The gap this repo fills is not "no one detects injection." It is:

A small library (not a hosted service, not a fine-tuned model) that packages the write-time ingestion contract — quarantined tool-less transform, per-stage capability isolation, output-scan-before-persist, fail-secure disposition — as composable, framework-agnostic, minimal-dependency code, with corpus-aware false-positive control.

Necessary honesty (verified, see §11): 2025 research shows character-level and adversarial-ML evasion defeats most pattern- and classifier-based detectors while keeping the payload legible to the target model. Therefore the value of this library is architecture and defense-in-depth, not a detector that "solves" injection. The lexicon is one WARN-class layer; the contract (a tool-less call that cannot act on a successful injection, credentials the hijacked step never holds, output scanned before it is trusted) is the load- bearing part. The brief must lead with the contract, not the regex.

4. Design principles

1. Minimal dependencies. Stdlib-first core. Optional pluggable detectors (an embedding classifier, an LLM-judge) live behind extras, never in the core path. A pipeline should be able to adopt the core with zero new runtime deps.
2. Framework-agnostic. Works with any SDK (Anthropic, OpenAI, local). The library never makes the model call; it hardens the call the pipeline makes.
3. Pure functions. Detection is text -> findings. No network, no telemetry, no global state. Portable, testable, auditable.
4. Disposition belongs to the caller. The library reports; the pipeline decides WARN vs quarantine-review vs fail-secure-halt. Defaults are provided, but blocking is never imposed — see principle 5.
5. Corpus-aware false-positive control. A corpus that legitimately discusses prompt injection (security docs, changelogs, a wiki about AI safety) will trip any broad lexicon. The library must make WARN-not-block the easy default and hard-fail an explicit, calibrated opt-in. Silent over-blocking of legitimate content is a failure mode, not a safe default.
6. Fail-secure under compound signals. Injection-scan hit and a transform failure is treated as a probable forced-fallback attack: halt + alert, never an auto-committed verbatim entry.

5. Proposed module layout

src/llm_ingest_security/
  sanitize/      Carrier stripping: invisible chars (Unicode Tags, zero-width,
                 bidi), HTML comments, data: URIs. Byte-identical on clean input;
                 removes only, never rewrites. Per-class counts for a WARN gate.
  fence/         Spotlighting helpers: wrap untrusted input in markers, and strip
                 the markers FROM the input first so it cannot break out of its
                 own fence.
  lexicon/       Semantic injection patterns as DATA (regex + label + severity),
                 tiered CRITICAL/HIGH/MEDIUM: override, spoofed headers/tags,
                 identity redefinition, config attacks, NL-indirection
                 (fetch-and-execute, send-to-external, read-dotfiles,
                 extract-and-exfiltrate), sub-agent spawn, leetspeak/homoglyph,
                 multi-language. Curated, versioned, re-syncable.
  entropy/       base64/hex blob detection for smuggled encoded instructions or
                 encoded exfil (complements secret-pattern scanning).
  contract/      The quarantine asserters — the differentiator:
                   - assert the model request carries NO tools
                   - assert per-stage credential allowlist (the enrichment step
                     sees exactly the one key it needs, and none of the publish
                     credentials)
                   - capability-isolation helpers (env scoping)
  output/        Output-side scanner: run the lexicon + entropy over the model's
                 OUTPUT (summary, verbatim quotes, extracted fields) BEFORE the
                 artifact is persisted. This is the RAG-poisoning gate.
  disposition/   Policy object: WARN | QUARANTINE_REVIEW | FAIL_SECURE per gate,
                 plus the compound-signal fail-secure rule.
  report/        Structured findings: class, count, severity, offset, source
                 (input|output), disposition.

6. The reusable contract (adopt-this checklist)

The actual product is this checklist, encoded as code a pipeline wires in order:

1. Sanitize before fence. Strip carrier classes from untrusted input first.
2. Fence untrusted input. Spotlight-mark it; strip markers from the payload.
3. Tool-less transform. Call the model with zero tools. A successful injection then has nothing to act with.
4. Per-stage capability isolation. The enrichment stage holds only the model key; the publish stage holds only the publish credential; no stage holds both.
5. Treat output as data. Parse to a frozen schema; reject on structural violation. The output never reaches a shell, git, or a filesystem path.
6. Scan output before persist. Run the lexicon + entropy over the emitted text. Verbatim-carried payloads and model-emitted instructions are caught here.
7. Fail-secure on compound signals. Injection hit + transform failure = halt
   • alert, never a silent verbatim commit.
8. Minimal alert payloads. Alert with a gate code + run ID, never content. Treat the alert channel/topic as a secret.

7. Non-goals

• Not a query-time chatbot / agent guardrail (that space is served).
• Not a fine-tuned model or a required ML classifier — deterministic core; ML detectors are optional, pluggable extras.
• Not a hosted service or SaaS.
• Not a silver bullet — explicitly a defense-in-depth layer whose lexicon is bypassable in isolation; the contract is what carries the security.

8. Language, packaging, licensing

• Python first — most ingestion pipelines are Python. Stdlib-only core; pyproject.toml with optional extras ([ml], [judge]).
• Framework-agnostic public API; no SDK imported by the core.
• A Node/TypeScript port is a plausible follow-on (a shared JSON lexicon as the single source of truth across both).
• Permissive license (MIT or Apache-2.0) so it is droppable into any pipeline.

9. Test strategy

• Adversarial corpus. A seeded set of payloads (one per lexicon class, including obfuscated and multi-language variants); measure and report recall.
• False-positive corpus. Real content that legitimately discusses injection (security docs, changelogs); assert the default disposition is WARN, not block, and that hard-fail is an explicit opt-in.
• Sanitizer invariant. Clean input returns byte-identical with an all-zero report; the sanitizer only removes.
• Contract asserters. A tool-carrying request and a credential-leaking stage env both raise; the happy path passes.
• No network in any test.

10. Threat-model grounding

• OWASP LLM Top 10: LLM01 Prompt Injection, output-handling, and the RAG / knowledge-poisoning vectors.
• Google DeepMind "AI Agent Traps" (Franklin et al., 2025): Latent Memory Poisoning and RAG Knowledge Poisoning map directly onto the write-time artifact.
• Research anchors: CleanBase (malicious docs in RAG DBs), DataFilter, and the 2025 guardrail-evasion literature that motivates the defense-in-depth framing.

11. Verification log (prior-art claims + sources)

Per the operator's verification duty — key claims in this brief and where they are grounded:

• Injection detection is crowded and query-time oriented — LLM Guard, NeMo Guardrails, Guardrails AI, Rebuff, Vigil, Vijil, LlamaFirewall, Resk, verified via the 2026 guardrails comparison and tool docs.
  • https://dev.to/agdex_ai/best-ai-agent-security-guardrails-tools-in-2026-llm-guard-vs-nemo-vs-guardrails-ai-5e5d
• Pattern/classifier detection alone is bypassable (defense-in-depth framing) — "Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks", arXiv 2504.11168 (character injection defeats most guardrails).
  • https://arxiv.org/abs/2504.11168
• Write-time RAG-DB poisoning is a researched but under-tooled vector — CleanBase (detecting malicious documents in RAG knowledge databases).
  • https://arxiv.org/pdf/2605.00460
• Multi-agent output-guard-before-release architecture exists in research — "A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks", arXiv 2509.14285.
  • https://arxiv.org/abs/2509.14285
• LlamaFirewall as an open-source guardrail reference — arXiv 2505.03574.
  • https://arxiv.org/pdf/2505.03574

Marked assumed, not verified: the specific claim that no existing library packages the full write-time contract as minimal-dependency code. The search found no such library, but absence of evidence is not proof; a focused survey of PyPI + GitHub topics should confirm before the README makes a novelty claim.

12. Reference implementation

The contract in §6 is not hypothetical — it is extracted from a working pipeline: the claude-code-llm-wiki Stage B enrichment path (tools/wiki_ingest/), which already implements sanitize + spotlight-fence + tool-less quarantined SDK call + per-stage credential isolation + schema-validated output + fail-secure ForcedFallbackHalt. That pipeline is the reference implementation and the first consumer; this repo generalizes its pattern for reuse. The semantic lexicon seed is the injection-patterns.mjs table from the llm-security plugin (a pure regex+label+severity dataset, portable as data).

13. Open decisions for the operator

1. Name — llm-ingestion-pipeline-security is descriptive but long; a shorter package name (e.g. ingestguard) may be worth it before first publish.
2. License — MIT vs Apache-2.0.
3. Relationship to llm-security — sibling repos sharing one lexicon dataset as the single source of truth, or fully independent? Avoid two drifting copies of the pattern table.
4. Publish target — Forgejo (git.fromaitochitta.com), and whether a public open/ mirror. No GitHub per house policy.
5. Python-only vs polyglot from day one.
6. Maintenance model — solo fork-and-own (as llm-security), or open to PRs.