llm-ingestion-pipeline-security/README.md

# llm-ingestion-guard

![Version](https://img.shields.io/badge/version-0.2.0-blue)
![Status](https://img.shields.io/badge/status-alpha-orange)
![Python](https://img.shields.io/badge/python-3.10%2B-purple)
![Tests](https://img.shields.io/badge/tests-357_passing-green)
![License](https://img.shields.io/badge/license-MIT-lightgrey)

A reusable, minimal, dependency-light defensive layer for **LLM ingestion
pipelines** — the write-time siblings of query-time chatbot guardrails.

Where mature guardrails (LLM Guard, NeMo Guardrails, Rebuff, Vigil, …) sit
between a user and a model at query time, this library hardens the other shape:
untrusted content flowing through an LLM enrichment/summarization/extraction step
into a **persisted, downstream-consumed artifact** (RAG corpus, knowledge base,
wiki). It packages the architectural contract — sanitize → fence → tool-less
quarantined transform → per-stage capability isolation → scan output before
commit → fail-secure — as composable, stdlib-first, framework-agnostic code.

The gap it fills is **not** "no one detects injection." Existing OSS tools are
either single-stage *detectors* — they emit a risk verdict and leave quarantine,
capability isolation, scan-before-persist, and fail-secure disposition to the
integrator — or runtime content-hardening that guards material as it *enters* the
model. This library packages the full **write-time ingestion contract** as
composable code: the part query-time tooling structurally cannot see, because a
poisoned artifact committed at write time is read by a *downstream* agent whose
guardrail never sees where it came from. (Nearest neighbours surveyed in
`docs/BRIEF.md` §11.)

**Status:** `v0.2`, alpha. The stdlib-only core is built and tested — ten
detector/contract modules and the top-level wiring, exercised by an end-to-end
showcase and adversarial + false-positive corpora. The public API may still
change. There are real limitations, stated plainly below; read them.

## Install

```bash
pip install llm-ingestion-guard          # stdlib-only core, zero dependencies
```

Optional ML/judge detectors live behind extras (`[ml]`, `[judge]`) and are not
required — the core is deterministic and dependency-free.

## Quickstart — the two bookends

The library never makes the model call itself. It gives you the two library-side
halves around your own **tool-less** transform:

```python
from llm_ingestion_guard import (
    prepare_input, screen_output, Disposition, PRESET_USER_UPLOAD,
)

prepared = prepare_input(untrusted_content)              # §6 1-2: sanitize + fence
enriched = your_model(prepared.fenced)                   # §6 3: tool-less — YOUR call
decision = screen_output(enriched, PRESET_USER_UPLOAD)   # §6 6-7: scan + dispose

if decision.disposition is Disposition.FAIL_SECURE:
    alert(gate_code=decision.reasons)                    # §6 8: minimal payload, no content
    raise SystemExit                                     # §6 7: halt — never persist
```

`screen_output` fails **closed**: if the scanner itself errors on crafted input,
the disposition is `FAIL_SECURE`, never a silent persist. Pass
`transform_failed=True` when your model call raised or fell back — a scan hit
together with a transform failure is treated as a probable forced-fallback attack
and halts regardless of trust tier.

Every primitive is also exported for pipelines that compose the checklist
themselves — `sanitize`, `scan_lexicon`, `scan_entropy`, `scan_output`,
`scan_active_content`, `neutralize`, the `decide` / `guard` disposition
machinery, and the contract asserters `assert_tool_less` /
`assert_credential_allowlist` / `scoped_env`. See
[the end-to-end showcase](tests/test_showcase.py) for a full worked pipeline.

## Verify what it stops (coverage matrix)

Before wiring the guard into your pipeline, see — in one place — every
vulnerability class it stops, and the ones it deliberately does not:

```bash
python -m llm_ingestion_guard.coverage    # prints the matrix; exit 0 = all as documented
```

Each row drives the real guard with a live payload and prints
`class -> OWASP anchor -> expected -> observed -> verdict`. The manifest
([`coverage.py`](src/llm_ingestion_guard/coverage.py)) is the single source of
truth for [`tests/test_coverage_matrix.py`](tests/test_coverage_matrix.py), which
asserts **total recall** across every defended class, that **every documented gap
still holds** (a closed gap fails the test, forcing a doc update), and — the
honesty guard — that **every lexicon pattern has a case** so the matrix cannot
fall behind the lexicon. The full LLM02 secret-egress set and the container-layer
front-end (CSV formula-injection, zip-slip/bomb, symlink) are asserted there too.

## The reusable contract (adopt-this checklist)

The actual product is this checklist, encoded as code you wire in order:

1. **Sanitize before fence.** Strip carrier classes (zero-width, BIDI,
   Unicode-tag, HTML comment, `data:`) from untrusted input first.
2. **Fence untrusted input.** Spotlight-mark it in a randomized per-call
   delimiter; strip attacker fence markers from the payload.
3. **Tool-less transform.** Call the model with zero tools. A successful
   injection then has nothing to act with.
4. **Per-stage capability isolation.** The enrichment stage holds only the model
   key; the publish stage holds only the publish credential; no stage holds both.
5. **Treat output as data.** Parse to a frozen schema; reject on structural
   violation. The output never reaches a shell, git, or a filesystem path.
6. **Scan output before persist.** Run the lexicon + entropy + active-content
   scan over the emitted text. Verbatim-carried payloads, model-emitted
   instructions, and zero-click exfil carriers (EchoLeak-class markdown
   images/links, raw active HTML, `data:` URIs) are caught here; `neutralize`
   additionally defangs them, opt-in.
7. **Fail-secure on compound signals.** Injection hit + transform failure = halt
   + alert, never a silent verbatim commit.
8. **Minimal alert payloads.** Alert with a gate code + run ID, never content.

Steps 1-2 are `prepare_input`; steps 6-7 are `screen_output`; steps 3-5 are
yours; the contract asserters harden step 3-4.

## Honest limitations (shipped as a control)

Conceding these plainly is itself a control — it prevents the false assurance
that a green scan means safe content:

- **Structural unsolvability at the text layer.** Pattern/lexicon detection is
  bypassable in isolation; character-injection and novel phrasings evade it. The
  *contract* (tool-less transform, capability isolation, fail-secure) is what
  carries the security — the lexicon is defense-in-depth, not a wall.
- **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.**
  Under `PRESET_TRUSTED_SOURCE`, §4.7 trust-scaling downgrades a single HIGH to
  WARN (a reputable single-author source is expected to carry security
  vocabulary), and one HIGH is not "compound" — compound escalation needs ≥2
  findings at MEDIUM+. So a HIGH-severity injection reproduced verbatim in output
  under a *trusted* policy persists with only a WARN. This is the §4.7 design, not
  a miss: if your "trusted" sources can carry attacker-influenced text, run them as
  untrusted (or add a quarantine floor) instead.
- **The upload preset's quarantine floor is currently vacuous.**
  `PRESET_USER_UPLOAD` sets `quarantine_default` ("any finding → at least
  QUARANTINE_REVIEW"), but every detector emits CRITICAL/HIGH/MEDIUM only (no
  LOW/INFO), and under untrusted trust a MEDIUM already escalates to
  QUARANTINE_REVIEW. The floor therefore never changes an outcome in today's
  severity set — it is defensive headroom for a future LOW/INFO finding (e.g. a
  grounding "unchecked" marker), not an active control. Documented so the preset's
  guarantee is not over-read.
- **Semantic / factual poisoning is invisible** to lexicon + entropy: a
  factually false claim in clean prose carries no suspicious token. The
  `grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic
  core does not judge semantics; a `[judge]` implementation must be plugged in.
- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch**
  between scanner and model leaves gaps.
- **Latent / dormant memory poisoning** is not judgeable at write time.
- **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a
  link to a not-yet-existing target passes a per-concept write-time scan clean —
  the payload is planted later, when that target is written. A scanner that sees
  one document at a time cannot catch it; it needs cross-write graph re-scan
  (tracked for the OKF adapter).
- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are
  legitimate structure (directory listing, update log), so mode-b `import_bundle`
  scans their body and frontmatter — an injection planted in a directory listing
  is caught — rather than path-rejecting the whole conformant bundle. The upload
  front-end keeps the opposite rule: an individual upload landing on a reserved
  basename is a shadow of the listing and is refused (`allow_reserved=False`).
- **A document that *describes* attacks is a false positive.** Content whose
  legitimate purpose is to document prompt-injection payloads (security notes,
  this project's own corpus) trips carrier-strip / fail-secure. At the text layer
  there is no way to distinguish "*about* an attack" from "*carrying* an attack";
  such content needs a deliberate, explicitly-marked escaped path, never a silent
  allow.
- **Bilingual text trips the Cyrillic/Latin homoglyph rule.**
  `homoglyph:cyrillic-latin-mix` (MEDIUM) flags any Latin letter adjacent to a
  Cyrillic look-alike, so genuine bilingual prose (e.g. Russian, or mixed
  Norwegian/Cyrillic) trips MEDIUM → under untrusted → QUARANTINE_REVIEW — a real
  false positive for an inbox that *expects* multilingual content. A calibration
  fix is pending (require ≥N mixed pairs, or only flag when a folded variant also
  hits another pattern); until then, multilingual corpora over-quarantine on this
  rule.
- **Insider in-place edits** by a trusted author are out of the untrusted-content
  threat model.
- **Text-only.** The core is `text -> findings`: it parses no files (no
  `pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the
  high-untrust upload provenance. OCR-embedded instructions and multimodal stego
  in images/PDFs are out of scope beyond the sanitizer's character-layer stripping.
- **Uploaded files: only the *extracted text* is scanned.** The two-stage OKF
  inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`, a
  dev-scoped demonstration whose `python-docx`/`python-pptx` parsers live in the
  `[dev]` extra, never core `dependencies`) reads `.txt`/`.md`/`.csv`/`.docx`/
  `.pptx`/`.xlsx`, folders and `.zip`, materializes them into an OKF bundle, then
  guards it. What survives text extraction is **out of scope**: VBA/macros
  (`.docm`/`.xlsm`/`.pptm`), OLE / embedded objects, image-embedded instructions
  needing OCR, font/render steganography, and encrypted / password-protected files
  — the binary layer needs a separate scanner. The front-end owns the container
  threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink
  refusal, CSV/XLSX formula-lead cells). **`.pdf` is a deliberate concession, not a
  TODO:** a top-level `.pdf` drop is *refused as an unsupported format* rather than
  half-scanned, because a PDF parser (plus `reportlab` solely to author white-on-white
  test fixtures) is disproportionate for a dev-scoped showcase, and the OCR /
  font-render stego carriers a PDF would smuggle are already out of scope above. The
  one remaining known gap is the numeric `-`/`+` CSV false positive (an XLSX numeric
  cell is typed, so it does not trip the gate).
- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first
  offset are reported, so the same class matched across several channels/variants
  collapses to one finding at its first location. This keeps reports readable, but
  a caller that counts occurrences or needs every offset of a repeated pattern sees
  only the first: a deliberate readability tradeoff, not full positional coverage.
- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate
  decodes base64 blobs and re-scans the plaintext through the credential/egress set,
  so a base64-*wrapped* secret surfaces as `decoded:egress:*` rather than vanishing.
  A *hex*-wrapped secret does not: `entropy` exposes decoded plaintext for base64
  only, so hex (and other encodings, or nested wraps) is a deliberate boundary, not
  a silent miss — decode the transport layer first if you need it scanned.

## Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
steganography; query-time / runtime guardrails; semantic factuality verification.

## Design & threat model

- [Design brief](docs/BRIEF.md) — what this repo contains and why.
- [Build plan](docs/PLAN.md) — module build order and the reuse map.

The contract is extracted from a working reference implementation (the
`claude-code-llm-wiki` Stage B enrichment pipeline). Threat-model anchors: OWASP
LLM Top-10 2025 (LLM01/02/04/05/06 strongest, LLM08 boundary, LLM09/10),
PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak (CVE-2025-32711).

## License

MIT — see [LICENSE](LICENSE).