llm-ingestion-pipeline-security/docs/BRIEF.md

# Brief: `llm-ingestion-pipeline-security`

**A reusable, minimal, dependency-light defensive layer for LLM *ingestion*
pipelines — the write-time siblings of query-time chatbot guardrails.**

Status: brief / pre-implementation. This document defines what the repo should
contain and why. No code yet.

---

## 1. One-line purpose

Give any pipeline that runs untrusted content through a large language model and
then **persists the result into a downstream-consumed artifact** (RAG corpus,
knowledge base, wiki, embeddings store) a small, framework-agnostic library that
packages the *architectural contract* for doing it safely: sanitize → fence →
tool-less quarantined transform → capability isolation → scan output before
commit → fail-secure.

## 2. The problem, stated precisely

The mature tooling for prompt injection is **query-time**: it sits between a user
(or an agent) and a model, at inference, on live traffic. Ingestion is a
different shape and is underserved:

- **Direction.** The danger flows from *content* (documents, changelogs, scraped
  web pages, user uploads, tickets) through an LLM *enrichment/summarization/
  extraction/classification* step, into a **persisted artifact** that other
  systems and agents later read.
- **Two distinct failure modes**, only the first of which query-time guardrails
  address:
  1. **Injection steers the enrichment step** — untrusted content contains
     instructions that hijack the summarizer/extractor.
  2. **Poisoned content is published** — the artifact itself becomes the attack.
     A verbatim-quoted payload, or an LLM-emitted instruction, lands in the
     knowledge base and later poisons a *downstream* agent's context. This is
     RAG/memory poisoning committed at *write* time, and a query-time guardrail
     on the downstream reader never sees where it came from.
- **Non-interactive, unattended.** Ingestion typically runs headless on a
  schedule. There is no human in the loop to notice a weird answer. Failure
  discipline (fail-secure, alerting, no silent verbatim fallback under attack)
  matters more here than in a chat UI.

## 3. Prior art and honest positioning

Injection *detection* is a crowded space. This repo must not pretend otherwise.

| Tool | Orientation |
|------|-------------|
| LLM Guard, Guardrails AI, NeMo Guardrails | Query-time I/O guardrails / dialogue policy |
| Rebuff | Self-hardening query-time detector (vector store of past attacks) |
| Vigil, Lakera Guard, Vijil | Query-time injection classifiers / detection servers |
| LlamaFirewall (Meta) | Agent guardrail framework (I/O, tool calls) |
| Resk | Python guardrail lib for LLM API calls |
| CleanBase, DataFilter (research) | Detecting/cleaning malicious docs in RAG DBs |

The gap this repo fills is **not** "no one detects injection." It is:

> A small **library** (not a hosted service, not a fine-tuned model) that
> packages the **write-time ingestion contract** — quarantined tool-less
> transform, per-stage capability isolation, output-scan-before-persist,
> fail-secure disposition — as composable, framework-agnostic, minimal-dependency
> code, with corpus-aware false-positive control.

**Necessary honesty (verified, see §11):** 2025 research shows character-level
and adversarial-ML evasion defeats most pattern- and classifier-based detectors
while keeping the payload legible to the target model. Therefore the value of
this library is **architecture and defense-in-depth**, not a detector that
"solves" injection. The lexicon is one WARN-class layer; the *contract* (a
tool-less call that cannot act on a successful injection, credentials the
hijacked step never holds, output scanned before it is trusted) is the load-
bearing part. The brief must lead with the contract, not the regex.

## 4. Design principles

1. **Minimal dependencies.** Stdlib-first core. Optional pluggable detectors
   (an embedding classifier, an LLM-judge) live behind extras, never in the core
   path. A pipeline should be able to adopt the core with zero new runtime deps.
2. **Framework-agnostic.** Works with any SDK (Anthropic, OpenAI, local). The
   library never makes the model call; it hardens the call the pipeline makes.
3. **Pure functions.** Detection is `text -> findings`. No network, no telemetry,
   no global state. Portable, testable, auditable.
4. **Disposition belongs to the caller.** The library reports; the pipeline
   decides WARN vs quarantine-review vs fail-secure-halt. Defaults are provided,
   but blocking is never imposed — see principle 5.
5. **Corpus-aware false-positive control.** A corpus that *legitimately discusses*
   prompt injection (security docs, changelogs, a wiki about AI safety) will trip
   any broad lexicon. The library must make WARN-not-block the easy default and
   hard-fail an explicit, calibrated opt-in. Silent over-blocking of legitimate
   content is a failure mode, not a safe default.
6. **Fail-secure under compound signals.** Injection-scan hit *and* a transform
   failure is treated as a probable forced-fallback attack: halt + alert, never
   an auto-committed verbatim entry. The same default applies when the output
   scanner itself is **unavailable or errors** (a missing/unresolvable detector
   dependency, a scanner exception): treat an un-scannable artifact as a BLOCK —
   never persist it unscanned. Fail closed, not open. (Verified in the first
   output-side implementation, 2026-07-04: an unresolvable detector is turned into
   a hard-fail, not a silent pass.)
7. **Disposition scales with source trust.** The *same* lexicon should not carry
   the *same* disposition for every source. A pinned, reputable, single-author
   source (an official changelog) warrants **WARN** — false positives from
   legitimate security vocabulary dominate the real signal. An open,
   user-generated source (a community Q&A forum, scraped web) warrants
   **QUARANTINE_REVIEW or hard-fail on high-severity hits** — a random
   contributor emitting a spoofed `<system>` block is not "legitimately
   discussing injection," and the cost of over-blocking one forum post is far
   lower than publishing a poisoned one. Source trust level is therefore a
   first-class input to the disposition policy, not a global constant.

   **Trust also tiers *within* a document, not only across sources** (verified in
   the first output-side implementation, 2026-07-04). A fenced code sample or a
   localized (non-English) string inside an otherwise-authoritative doc is the
   low-trust surface *even when the document's source is reputable* — a spoofed
   `<system>` block or a base64 blob in a code sample is far likelier a real
   payload than a false positive, whereas the same lexicon hit in authored,
   en-locale prose is likelier a doc legitimately discussing the pattern. So the
   *same* hit hard-fails inside a code block / localized string but WARNs in
   authored prose. Intra-document provenance (in-code-fence, non-en locale) is a
   first-class disposition input alongside source-level trust. Invisible carriers
   (zero-width / bidi / Unicode-tag) and critical injection are the exception:
   they have no legitimate place in a reference file and block in any tier.

## 5. Proposed module layout

```
src/llm_ingest_security/
  sanitize/      Carrier stripping: invisible chars (Unicode Tags, zero-width,
                 bidi), HTML comments, data: URIs. Byte-identical on clean input;
                 removes only, never rewrites. Per-class counts for a WARN gate.
  fence/         Spotlighting helpers: wrap untrusted input in markers, and strip
                 the markers FROM the input first so it cannot break out of its
                 own fence.
  lexicon/       Semantic injection patterns as DATA (regex + label + severity),
                 tiered CRITICAL/HIGH/MEDIUM: override, spoofed headers/tags,
                 identity redefinition, config attacks, NL-indirection
                 (fetch-and-execute, send-to-external, read-dotfiles,
                 extract-and-exfiltrate), sub-agent spawn, leetspeak/homoglyph,
                 multi-language. Curated, versioned, re-syncable.
  entropy/       base64/hex blob detection for smuggled encoded instructions or
                 encoded exfil (complements secret-pattern scanning).
  contract/      The quarantine asserters — the differentiator:
                   - assert the model request carries NO tools
                   - assert per-stage credential allowlist (the enrichment step
                     sees exactly the one key it needs, and none of the publish
                     credentials)
                   - capability-isolation helpers (env scoping)
  output/        Output-side scanner: run the lexicon + entropy over the model's
                 OUTPUT (summary, verbatim quotes, extracted fields) BEFORE the
                 artifact is persisted. This is the RAG-poisoning gate.
  disposition/   Policy object: WARN | QUARANTINE_REVIEW | FAIL_SECURE per gate,
                 plus the compound-signal fail-secure rule.
  report/        Structured findings: class, count, severity, offset, source
                 (input|output), disposition.
```

## 6. The reusable contract (adopt-this checklist)

The actual product is this checklist, encoded as code a pipeline wires in order:

1. **Sanitize before fence.** Strip carrier classes from untrusted input first.
2. **Fence untrusted input.** Spotlight-mark it; strip markers from the payload.
3. **Tool-less transform.** Call the model with zero tools. A successful
   injection then has nothing to act with.
4. **Per-stage capability isolation.** The enrichment stage holds only the model
   key; the publish stage holds only the publish credential; no stage holds both.
5. **Treat output as data.** Parse to a frozen schema; reject on structural
   violation. The output never reaches a shell, git, or a filesystem path.
6. **Scan output before persist.** Run the lexicon + entropy over the emitted
   text. Verbatim-carried payloads and model-emitted instructions are caught here.
7. **Fail-secure on compound signals.** Injection hit + transform failure = halt
   + alert, never a silent verbatim commit.
8. **Minimal alert payloads.** Alert with a gate code + run ID, never content.
   Treat the alert channel/topic as a secret.

## 7. Non-goals

- **Not** a query-time chatbot / agent guardrail (that space is served).
- **Not** a fine-tuned model or a required ML classifier — deterministic core;
  ML detectors are optional, pluggable extras.
- **Not** a hosted service or SaaS.
- **Not** a silver bullet — explicitly a defense-in-depth layer whose lexicon is
  bypassable in isolation; the contract is what carries the security.

## 8. Language, packaging, licensing

- **Python first** — most ingestion pipelines are Python. Stdlib-only core;
  `pyproject.toml` with optional extras (`[ml]`, `[judge]`).
- Framework-agnostic public API; no SDK imported by the core.
- A Node/TypeScript port is a plausible follow-on (a shared JSON lexicon as the
  single source of truth across both).
- Permissive license (MIT or Apache-2.0) so it is droppable into any pipeline.

## 9. Test strategy

- **Adversarial corpus.** A seeded set of payloads (one per lexicon class,
  including obfuscated and multi-language variants); measure and report *recall*.
- **False-positive corpus.** Real content that legitimately discusses injection
  (security docs, changelogs); assert the default disposition is WARN, not block,
  and that hard-fail is an explicit opt-in.
- **Sanitizer invariant.** Clean input returns byte-identical with an all-zero
  report; the sanitizer only removes.
- **Contract asserters.** A tool-carrying request and a credential-leaking stage
  env both raise; the happy path passes.
- **No network in any test.**

## 10. Threat-model grounding

- OWASP LLM Top 10: **LLM01 Prompt Injection**, output-handling, and the RAG /
  knowledge-poisoning vectors.
- Google DeepMind "AI Agent Traps" (Franklin et al., 2025): Latent Memory
  Poisoning and RAG Knowledge Poisoning map directly onto the write-time artifact.
- Research anchors: CleanBase (malicious docs in RAG DBs), DataFilter, and the
  2025 guardrail-evasion literature that motivates the defense-in-depth framing.

## 11. Verification log (prior-art claims + sources)

Per the operator's verification duty — key claims in this brief and where they
are grounded:

- *Injection detection is crowded and query-time oriented* — LLM Guard, NeMo
  Guardrails, Guardrails AI, Rebuff, Vigil, Vijil, LlamaFirewall, Resk, verified
  via the 2026 guardrails comparison and tool docs.
  - https://dev.to/agdex_ai/best-ai-agent-security-guardrails-tools-in-2026-llm-guard-vs-nemo-vs-guardrails-ai-5e5d
- *Pattern/classifier detection alone is bypassable (defense-in-depth framing)* —
  "Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks", arXiv
  2504.11168 (character injection defeats most guardrails).
  - https://arxiv.org/abs/2504.11168
- *Write-time RAG-DB poisoning is a researched but under-tooled vector* —
  CleanBase (detecting malicious documents in RAG knowledge databases).
  - https://arxiv.org/pdf/2605.00460
- *Multi-agent output-guard-before-release architecture exists in research* —
  "A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks",
  arXiv 2509.14285.
  - https://arxiv.org/abs/2509.14285
- *LlamaFirewall as an open-source guardrail reference* — arXiv 2505.03574.
  - https://arxiv.org/pdf/2505.03574

Marked **assumed, not verified**: the specific claim that no existing *library*
packages the full write-time contract as minimal-dependency code. The search
found no such library, but absence of evidence is not proof; a focused survey of
PyPI + GitHub topics should confirm before the README makes a novelty claim.

## 12. Reference implementation and target consumers

The contract in §6 is not hypothetical — it is extracted from a working pipeline:
the `claude-code-llm-wiki` Stage B enrichment path (`tools/wiki_ingest/`),
which already implements sanitize + spotlight-fence + tool-less quarantined SDK
call + per-stage credential isolation + schema-validated output + fail-secure
`ForcedFallbackHalt`. That pipeline is the reference implementation for the *full
contract* (tool-less transform, credential isolation, fail-secure). The semantic
lexicon seed is the `injection-patterns.mjs` table from the `llm-security` plugin
(a pure regex+label+severity dataset, portable as data).

**Integration lesson (verified 2026-07-04) — consume the lexicon as a pure
function, not via the scanner CLI.** The `llm-security` orchestrator CLI
(`llm-security scan`) is repo/directory-oriented and, empirically, does **not**
apply the injection lexicon to arbitrary Markdown prose, nor does its
entropy-scanner flag a base64 blob inside a fenced code block — a full deep-scan
over three seeded fixtures caught only the invisible-Unicode class. The load-
bearing reuse seam is therefore the **importable pure primitives**
(`scanForInjection(text)` from `injection-patterns.mjs`; `isBase64Like` /
`shannonEntropy` from `string-utils.mjs`; the `unicode-scanner` module), driven
per line/region so the consumer controls provenance tiering. This library should
ship the lexicon **as data** and detection **as pure `text -> findings`
functions** (design principle 3) so no consumer has to shell out to an
orchestrator that under-covers its content type.

**Target consumers (the reason the library exists):**

1. **`claude-code-llm-wiki`** — low-untrust source (a pinned, reputable
   Anthropic changelog). Here the contract is *defense-in-depth hygiene*; the
   architectural controls already close the severe outcomes, and the lexicon runs
   at WARN. This consumer is *retrofitting* the output-side widening late (at its
   A13), which is precisely the cost this library exists to avoid for the next one.
2. **`ms-ai-architect`** (the plugin in `ktg-plugin-marketplace`; there is **no**
   separate "MS AI Security plugin" — repo/name reconciled 2026-07-04) — an
   ingestion pipeline over **Microsoft Learn** content fetched via the
   `microsoft-learn` MCP (`microsoft_docs_fetch` / `_search` /
   `_code_sample_search`): authored, reviewed Learn docs **plus code samples and
   localized strings**. **Correction (verified against the live command/agent code
   2026-07-04):** the earlier characterization — that it is the *high-untrust* end
   ingesting the open Microsoft Q&A forum (`learn.microsoft.com/answers`), MSDN
   Forums and Stack Overflow — is **unverified and overstated**. The pipeline's
   fetch surface is authored Learn docs + code samples; it does not ingest the open
   Q&A forum as part of its flow. Its low-trust surface is therefore
   **intra-document** (a fenced code sample or a localized string inside an
   otherwise-authoritative doc), which is exactly why disposition must tier at the
   *chunk* level (§4.7), not treat the whole consumer as globally high-untrust.
   Even so the contract is **load-bearing, not hygiene** here: the KB is publicly
   distributed and re-served as instruction-adjacent context to every downstream
   agent session, so one poisoned reference file is a supply-chain compromise.

   A working, tested implementation of the **output-scan-before-persist** gate (§6
   step 6) plus **provenance-tiered disposition** (§4.7) now exists in this plugin
   — "Layer B" of its ingestion security gate, wired as a sibling to the existing
   create-guard at the single write chokepoint, with a fail-closed exit-code
   contract (0 clean / 1 block / 2 warn). It is the **second reference
   implementation** for this library — specifically for the *output side* (the
   RAG-poisoning gate + disposition), complementing consumer 1's reference for the
   *full contract* (tool-less transform + credential isolation). It consumes the
   `llm-security` lexicon by **in-process import of the pure functions** (see the
   integration lesson above), which is a concrete data point for open decision §13.3.

**Why day-1, not retrofit.** The load-bearing parts of the contract are
architectural — tool-less quarantine, per-stage credential isolation,
output-scan-before-persist, fail-secure disposition. These are cheap to design in
and expensive to bolt on after a pipeline exists (consumer 1 is proving that).
Adopting the library from day 1 means the *high-untrust* consumer (2) inherits a
correct, calibrated contract for free instead of discovering the gap in
production. The library's value is not this pipeline; it is every pipeline after
it whose input is less trustworthy than a pinned changelog.

## 13. Open decisions for the operator

1. **Name** — `llm-ingestion-pipeline-security` is descriptive but long; a
   shorter package name (e.g. `ingestguard`) may be worth it before first publish.
2. **License** — MIT vs Apache-2.0.
3. **Relationship to `llm-security`** — sibling repos sharing one lexicon dataset
   as the single source of truth, or fully independent? Avoid two drifting copies
   of the pattern table.
4. **Publish target** — Forgejo (`git.fromaitochitta.com`), and whether a public
   `open/` mirror. No GitHub per house policy.
5. **Python-only vs polyglot** from day one.
6. **Maintenance model** — solo fork-and-own (as `llm-security`), or open to PRs.