llm-ingestion-pipeline-security/docs/BRIEF.md

Brief: llm-ingestion-pipeline-security

A reusable, minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails.

Status: brief / pre-implementation. This document defines what the repo should contain and why. No code yet.

---

1. One-line purpose

Give any pipeline that runs untrusted content through a large language model and then persists the result into a downstream-consumed artifact (RAG corpus, knowledge base, wiki, embeddings store) a small, framework-agnostic library that packages the architectural contract for doing it safely: sanitize → fence → tool-less quarantined transform → capability isolation → scan output before commit → fail-secure.

2. The problem, stated precisely

The mature tooling for prompt injection is query-time: it sits between a user (or an agent) and a model, at inference, on live traffic. Ingestion is a different shape and is underserved:

• Direction. The danger flows from content (documents, changelogs, scraped web pages, user uploads, tickets) through an LLM enrichment/summarization/ extraction/classification step, into a persisted artifact that other systems and agents later read.
• Two distinct failure modes, only the first of which query-time guardrails address:
  1. Injection steers the enrichment step — untrusted content contains instructions that hijack the summarizer/extractor.
  2. Poisoned content is published — the artifact itself becomes the attack. A verbatim-quoted payload, or an LLM-emitted instruction, lands in the knowledge base and later poisons a downstream agent's context. This is RAG/memory poisoning committed at write time, and a query-time guardrail on the downstream reader never sees where it came from.
• Non-interactive, unattended. Ingestion typically runs headless on a schedule. There is no human in the loop to notice a weird answer. Failure discipline (fail-secure, alerting, no silent verbatim fallback under attack) matters more here than in a chat UI.

3. Prior art and honest positioning

Injection detection is a crowded space. This repo must not pretend otherwise.

Tool	Orientation
LLM Guard, Guardrails AI, NeMo Guardrails	Query-time I/O guardrails / dialogue policy
Rebuff	Self-hardening query-time detector (vector store of past attacks)
Vigil, Lakera Guard, Vijil	Query-time injection classifiers / detection servers
LlamaFirewall (Meta)	Agent guardrail framework (I/O, tool calls)
Resk	Python guardrail lib for LLM API calls
CleanBase, DataFilter (research)	Detecting/cleaning malicious docs in RAG DBs

The gap this repo fills is not "no one detects injection." It is:

A small library (not a hosted service, not a fine-tuned model) that packages the write-time ingestion contract — quarantined tool-less transform, per-stage capability isolation, output-scan-before-persist, fail-secure disposition — as composable, framework-agnostic, minimal-dependency code, with corpus-aware false-positive control.

Necessary honesty (verified, see §11): 2025 research shows character-level and adversarial-ML evasion defeats most pattern- and classifier-based detectors while keeping the payload legible to the target model. Therefore the value of this library is architecture and defense-in-depth, not a detector that "solves" injection. The lexicon is one WARN-class layer; the contract (a tool-less call that cannot act on a successful injection, credentials the hijacked step never holds, output scanned before it is trusted) is the load- bearing part. The brief must lead with the contract, not the regex.

4. Design principles

1. Minimal dependencies. Stdlib-first core. Optional pluggable detectors (an embedding classifier, an LLM-judge) live behind extras, never in the core path. A pipeline should be able to adopt the core with zero new runtime deps.

2. Framework-agnostic. Works with any SDK (Anthropic, OpenAI, local). The library never makes the model call; it hardens the call the pipeline makes.

3. Pure functions. Detection is text -> findings. No network, no telemetry, no global state. Portable, testable, auditable.

4. Disposition belongs to the caller. The library reports; the pipeline decides WARN vs quarantine-review vs fail-secure-halt. Defaults are provided, but blocking is never imposed — see principle 5.

5. Corpus-aware false-positive control. A corpus that legitimately discusses prompt injection (security docs, changelogs, a wiki about AI safety) will trip any broad lexicon. The library must make WARN-not-block the easy default and hard-fail an explicit, calibrated opt-in. Silent over-blocking of legitimate content is a failure mode, not a safe default.

6. Fail-secure under compound signals. Injection-scan hit and a transform failure is treated as a probable forced-fallback attack: halt + alert, never an auto-committed verbatim entry. The same default applies when the output scanner itself is unavailable or errors (a missing/unresolvable detector dependency, a scanner exception): treat an un-scannable artifact as a BLOCK — never persist it unscanned. Fail closed, not open. (Verified in the first output-side implementation, 2026-07-04: an unresolvable detector is turned into a hard-fail, not a silent pass.)

7. Disposition scales with source trust. The same lexicon should not carry the same disposition for every source. A pinned, reputable, single-author source (an official changelog) warrants WARN — false positives from legitimate security vocabulary dominate the real signal. An open, user-generated source (a community Q&A forum, scraped web) warrants QUARANTINE_REVIEW or hard-fail on high-severity hits — a random contributor emitting a spoofed <system> block is not "legitimately discussing injection," and the cost of over-blocking one forum post is far lower than publishing a poisoned one. Source trust level is therefore a first-class input to the disposition policy, not a global constant.

   Trust also tiers within a document, not only across sources (verified in the first output-side implementation, 2026-07-04). A fenced code sample or a localized (non-English) string inside an otherwise-authoritative doc is the low-trust surface even when the document's source is reputable — a spoofed <system> block or a base64 blob in a code sample is far likelier a real payload than a false positive, whereas the same lexicon hit in authored, en-locale prose is likelier a doc legitimately discussing the pattern. So the same hit hard-fails inside a code block / localized string but WARNs in authored prose. Intra-document provenance (in-code-fence, non-en locale) is a first-class disposition input alongside source-level trust. Invisible carriers (zero-width / bidi / Unicode-tag) and critical injection are the exception: they have no legitimate place in a reference file and block in any tier.

5. Proposed module layout

src/llm_ingest_security/
  sanitize/      Carrier stripping: invisible chars (Unicode Tags, zero-width,
                 bidi), HTML comments, data: URIs. Byte-identical on clean input;
                 removes only, never rewrites. Per-class counts for a WARN gate.
  fence/         Spotlighting helpers: wrap untrusted input in markers, and strip
                 the markers FROM the input first so it cannot break out of its
                 own fence.
  lexicon/       Semantic injection patterns as DATA (regex + label + severity),
                 tiered CRITICAL/HIGH/MEDIUM: override, spoofed headers/tags,
                 identity redefinition, config attacks, NL-indirection
                 (fetch-and-execute, send-to-external, read-dotfiles,
                 extract-and-exfiltrate), sub-agent spawn, leetspeak/homoglyph,
                 multi-language. Curated, versioned, re-syncable.
  entropy/       base64/hex blob detection for smuggled encoded instructions or
                 encoded exfil (complements secret-pattern scanning).
  contract/      The quarantine asserters — the differentiator:
                   - assert the model request carries NO tools
                   - assert per-stage credential allowlist (the enrichment step
                     sees exactly the one key it needs, and none of the publish
                     credentials)
                   - capability-isolation helpers (env scoping)
  output/        Output-side scanner: run the lexicon + entropy over the model's
                 OUTPUT (summary, verbatim quotes, extracted fields) BEFORE the
                 artifact is persisted. This is the RAG-poisoning gate.
  disposition/   Policy object: WARN | QUARANTINE_REVIEW | FAIL_SECURE per gate,
                 plus the compound-signal fail-secure rule.
  report/        Structured findings: class, count, severity, offset, source
                 (input|output), disposition.

6. The reusable contract (adopt-this checklist)

The actual product is this checklist, encoded as code a pipeline wires in order:

1. Sanitize before fence. Strip carrier classes from untrusted input first.
2. Fence untrusted input. Spotlight-mark it; strip markers from the payload.
3. Tool-less transform. Call the model with zero tools. A successful injection then has nothing to act with.
4. Per-stage capability isolation. The enrichment stage holds only the model key; the publish stage holds only the publish credential; no stage holds both.
5. Treat output as data. Parse to a frozen schema; reject on structural violation. The output never reaches a shell, git, or a filesystem path.
6. Scan output before persist. Run the lexicon + entropy over the emitted text. Verbatim-carried payloads and model-emitted instructions are caught here.
7. Fail-secure on compound signals. Injection hit + transform failure = halt
   • alert, never a silent verbatim commit.
8. Minimal alert payloads. Alert with a gate code + run ID, never content. Treat the alert channel/topic as a secret.

7. Non-goals

• Not a query-time chatbot / agent guardrail (that space is served).
• Not a fine-tuned model or a required ML classifier — deterministic core; ML detectors are optional, pluggable extras.
• Not a hosted service or SaaS.
• Not a silver bullet — explicitly a defense-in-depth layer whose lexicon is bypassable in isolation; the contract is what carries the security.

8. Language, packaging, licensing

• Python first — most ingestion pipelines are Python. Stdlib-only core; pyproject.toml with optional extras ([ml], [judge]).
• Framework-agnostic public API; no SDK imported by the core.
• A Node/TypeScript port is a plausible follow-on (a shared JSON lexicon as the single source of truth across both).
• Permissive license (MIT or Apache-2.0) so it is droppable into any pipeline.

9. Test strategy

• Adversarial corpus. A seeded set of payloads (one per lexicon class, including obfuscated and multi-language variants); measure and report recall.
• False-positive corpus. Real content that legitimately discusses injection (security docs, changelogs); assert the default disposition is WARN, not block, and that hard-fail is an explicit opt-in.
• Sanitizer invariant. Clean input returns byte-identical with an all-zero report; the sanitizer only removes.
• Contract asserters. A tool-carrying request and a credential-leaking stage env both raise; the happy path passes.
• No network in any test.

10. Threat-model grounding

• OWASP LLM Top 10: LLM01 Prompt Injection, output-handling, and the RAG / knowledge-poisoning vectors.
• Google DeepMind "AI Agent Traps" (Franklin et al., 2025): Latent Memory Poisoning and RAG Knowledge Poisoning map directly onto the write-time artifact.
• Research anchors: CleanBase (malicious docs in RAG DBs), DataFilter, and the 2025 guardrail-evasion literature that motivates the defense-in-depth framing.

11. Verification log (prior-art claims + sources)

Per the operator's verification duty — key claims in this brief and where they are grounded:

• Injection detection is crowded and query-time oriented — LLM Guard, NeMo Guardrails, Guardrails AI, Rebuff, Vigil, Vijil, LlamaFirewall, Resk, verified via the 2026 guardrails comparison and tool docs.
  • https://dev.to/agdex_ai/best-ai-agent-security-guardrails-tools-in-2026-llm-guard-vs-nemo-vs-guardrails-ai-5e5d
• Pattern/classifier detection alone is bypassable (defense-in-depth framing) — "Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks", arXiv 2504.11168 (character injection defeats most guardrails).
  • https://arxiv.org/abs/2504.11168
• Write-time RAG-DB poisoning is a researched but under-tooled vector — CleanBase (detecting malicious documents in RAG knowledge databases).
  • https://arxiv.org/pdf/2605.00460
• Multi-agent output-guard-before-release architecture exists in research — "A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks", arXiv 2509.14285.
  • https://arxiv.org/abs/2509.14285
• LlamaFirewall as an open-source guardrail reference — arXiv 2505.03574.
  • https://arxiv.org/pdf/2505.03574

Marked assumed, not verified: the specific claim that no existing library packages the full write-time contract as minimal-dependency code. The search found no such library, but absence of evidence is not proof; a focused survey of PyPI + GitHub topics should confirm before the README makes a novelty claim.

12. Reference implementation and target consumers

The contract in §6 is not hypothetical — it is extracted from a working pipeline: the claude-code-llm-wiki Stage B enrichment path (tools/wiki_ingest/), which already implements sanitize + spotlight-fence + tool-less quarantined SDK call + per-stage credential isolation + schema-validated output + fail-secure ForcedFallbackHalt. That pipeline is the reference implementation for the full contract (tool-less transform, credential isolation, fail-secure). The semantic lexicon seed is the injection-patterns.mjs table from the llm-security plugin (a pure regex+label+severity dataset, portable as data).

Integration lesson (verified 2026-07-04) — consume the lexicon as a pure function, not via the scanner CLI. The llm-security orchestrator CLI (llm-security scan) is repo/directory-oriented and, empirically, does not apply the injection lexicon to arbitrary Markdown prose, nor does its entropy-scanner flag a base64 blob inside a fenced code block — a full deep-scan over three seeded fixtures caught only the invisible-Unicode class. The load- bearing reuse seam is therefore the importable pure primitives (scanForInjection(text) from injection-patterns.mjs; isBase64Like / shannonEntropy from string-utils.mjs; the unicode-scanner module), driven per line/region so the consumer controls provenance tiering. This library should ship the lexicon as data and detection as pure text -> findings functions (design principle 3) so no consumer has to shell out to an orchestrator that under-covers its content type.

Target consumers (the reason the library exists):

1. claude-code-llm-wiki — low-untrust source (a pinned, reputable Anthropic changelog). Here the contract is defense-in-depth hygiene; the architectural controls already close the severe outcomes, and the lexicon runs at WARN. This consumer is retrofitting the output-side widening late (at its A13), which is precisely the cost this library exists to avoid for the next one.

2. ms-ai-architect (the plugin in ktg-plugin-marketplace; there is no separate "MS AI Security plugin" — repo/name reconciled 2026-07-04) — an ingestion pipeline over Microsoft Learn content fetched via the microsoft-learn MCP (microsoft_docs_fetch / _search / _code_sample_search): authored, reviewed Learn docs plus code samples and localized strings. Correction (verified against the live command/agent code 2026-07-04): the earlier characterization — that it is the high-untrust end ingesting the open Microsoft Q&A forum (learn.microsoft.com/answers), MSDN Forums and Stack Overflow — is unverified and overstated. The pipeline's fetch surface is authored Learn docs + code samples; it does not ingest the open Q&A forum as part of its flow. Its low-trust surface is therefore intra-document (a fenced code sample or a localized string inside an otherwise-authoritative doc), which is exactly why disposition must tier at the chunk level (§4.7), not treat the whole consumer as globally high-untrust. Even so the contract is load-bearing, not hygiene here: the KB is publicly distributed and re-served as instruction-adjacent context to every downstream agent session, so one poisoned reference file is a supply-chain compromise.

   A working, tested implementation of the output-scan-before-persist gate (§6 step 6) plus provenance-tiered disposition (§4.7) now exists in this plugin — "Layer B" of its ingestion security gate, wired as a sibling to the existing create-guard at the single write chokepoint, with a fail-closed exit-code contract (0 clean / 1 block / 2 warn). It is the second reference implementation for this library — specifically for the output side (the RAG-poisoning gate + disposition), complementing consumer 1's reference for the full contract (tool-less transform + credential isolation). It consumes the llm-security lexicon by in-process import of the pure functions (see the integration lesson above), which is a concrete data point for open decision §13.3.

Why day-1, not retrofit. The load-bearing parts of the contract are architectural — tool-less quarantine, per-stage credential isolation, output-scan-before-persist, fail-secure disposition. These are cheap to design in and expensive to bolt on after a pipeline exists (consumer 1 is proving that). Adopting the library from day 1 means the high-untrust consumer (2) inherits a correct, calibrated contract for free instead of discovering the gap in production. The library's value is not this pipeline; it is every pipeline after it whose input is less trustworthy than a pinned changelog.

13. Open decisions for the operator

1. Name — llm-ingestion-pipeline-security is descriptive but long; a shorter package name (e.g. ingestguard) may be worth it before first publish.
2. License — MIT vs Apache-2.0.
3. Relationship to llm-security — sibling repos sharing one lexicon dataset as the single source of truth, or fully independent? Avoid two drifting copies of the pattern table.
4. Publish target — Forgejo (git.fromaitochitta.com), and whether a public open/ mirror. No GitHub per house policy.
5. Python-only vs polyglot from day one.
6. Maintenance model — solo fork-and-own (as llm-security), or open to PRs.