Kjell Tore Guttormsen
f4e89d2885
Output gate step 3 now runs scan_secret_egress over every decoded base64 blob's plaintext, not only scan_lexicon. A base64-wrapped credential that formerly vanished (decode fed the lexicon, which has no secret patterns) now surfaces as decoded:egress:* carrying the blob offset. Evidence stays length-only, so the decoded finding never leaks the secret value. Hex-wrapped secrets remain a documented honest-limit (entropy exposes decoded plaintext for base64 only). README honest-limits + CLAUDE.md Kontekst updated; 3 tests added (347 passed, was 344).
179 lines
10 KiB
Markdown
179 lines
10 KiB
Markdown
# llm-ingestion-guard
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A reusable, minimal, dependency-light defensive layer for **LLM ingestion
|
|
pipelines** — the write-time siblings of query-time chatbot guardrails.
|
|
|
|
Where mature guardrails (LLM Guard, NeMo Guardrails, Rebuff, Vigil, …) sit
|
|
between a user and a model at query time, this library hardens the other shape:
|
|
untrusted content flowing through an LLM enrichment/summarization/extraction step
|
|
into a **persisted, downstream-consumed artifact** (RAG corpus, knowledge base,
|
|
wiki). It packages the architectural contract — sanitize → fence → tool-less
|
|
quarantined transform → per-stage capability isolation → scan output before
|
|
commit → fail-secure — as composable, stdlib-first, framework-agnostic code.
|
|
|
|
The gap it fills is **not** "no one detects injection." It is a small *library*
|
|
(not a hosted service, not a fine-tuned model) that packages the **write-time
|
|
ingestion contract** — the part query-time tooling structurally cannot see,
|
|
because a poisoned artifact committed at write time is read by a *downstream*
|
|
agent whose guardrail never sees where it came from.
|
|
|
|
**Status:** `v0.1`, alpha. The stdlib-only core is built and tested — ten
|
|
detector/contract modules and the top-level wiring, exercised by an end-to-end
|
|
showcase and adversarial + false-positive corpora. The public API may still
|
|
change. There are real limitations, stated plainly below; read them.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
pip install llm-ingestion-guard # stdlib-only core, zero dependencies
|
|
```
|
|
|
|
Optional ML/judge detectors live behind extras (`[ml]`, `[judge]`) and are not
|
|
required — the core is deterministic and dependency-free.
|
|
|
|
## Quickstart — the two bookends
|
|
|
|
The library never makes the model call itself. It gives you the two library-side
|
|
halves around your own **tool-less** transform:
|
|
|
|
```python
|
|
from llm_ingestion_guard import (
|
|
prepare_input, screen_output, Disposition, PRESET_USER_UPLOAD,
|
|
)
|
|
|
|
prepared = prepare_input(untrusted_content) # §6 1-2: sanitize + fence
|
|
enriched = your_model(prepared.fenced) # §6 3: tool-less — YOUR call
|
|
decision = screen_output(enriched, PRESET_USER_UPLOAD) # §6 6-7: scan + dispose
|
|
|
|
if decision.disposition is Disposition.FAIL_SECURE:
|
|
alert(gate_code=decision.reasons) # §6 8: minimal payload, no content
|
|
raise SystemExit # §6 7: halt — never persist
|
|
```
|
|
|
|
`screen_output` fails **closed**: if the scanner itself errors on crafted input,
|
|
the disposition is `FAIL_SECURE`, never a silent persist. Pass
|
|
`transform_failed=True` when your model call raised or fell back — a scan hit
|
|
together with a transform failure is treated as a probable forced-fallback attack
|
|
and halts regardless of trust tier.
|
|
|
|
Every primitive is also exported for pipelines that compose the checklist
|
|
themselves — `sanitize`, `scan_lexicon`, `scan_entropy`, `scan_output`,
|
|
`scan_active_content`, `neutralize`, the `decide` / `guard` disposition
|
|
machinery, and the contract asserters `assert_tool_less` /
|
|
`assert_credential_allowlist` / `scoped_env`. See
|
|
[the end-to-end showcase](tests/test_showcase.py) for a full worked pipeline.
|
|
|
|
## The reusable contract (adopt-this checklist)
|
|
|
|
The actual product is this checklist, encoded as code you wire in order:
|
|
|
|
1. **Sanitize before fence.** Strip carrier classes (zero-width, BIDI,
|
|
Unicode-tag, HTML comment, `data:`) from untrusted input first.
|
|
2. **Fence untrusted input.** Spotlight-mark it in a randomized per-call
|
|
delimiter; strip attacker fence markers from the payload.
|
|
3. **Tool-less transform.** Call the model with zero tools. A successful
|
|
injection then has nothing to act with.
|
|
4. **Per-stage capability isolation.** The enrichment stage holds only the model
|
|
key; the publish stage holds only the publish credential; no stage holds both.
|
|
5. **Treat output as data.** Parse to a frozen schema; reject on structural
|
|
violation. The output never reaches a shell, git, or a filesystem path.
|
|
6. **Scan output before persist.** Run the lexicon + entropy + active-content
|
|
scan over the emitted text. Verbatim-carried payloads, model-emitted
|
|
instructions, and zero-click exfil carriers (EchoLeak-class markdown
|
|
images/links, raw active HTML, `data:` URIs) are caught here; `neutralize`
|
|
additionally defangs them, opt-in.
|
|
7. **Fail-secure on compound signals.** Injection hit + transform failure = halt
|
|
+ alert, never a silent verbatim commit.
|
|
8. **Minimal alert payloads.** Alert with a gate code + run ID, never content.
|
|
|
|
Steps 1-2 are `prepare_input`; steps 6-7 are `screen_output`; steps 3-5 are
|
|
yours; the contract asserters harden step 3-4.
|
|
|
|
## Honest limitations (shipped as a control)
|
|
|
|
Conceding these plainly is itself a control — it prevents the false assurance
|
|
that a green scan means safe content:
|
|
|
|
- **Structural unsolvability at the text layer.** Pattern/lexicon detection is
|
|
bypassable in isolation; character-injection and novel phrasings evade it. The
|
|
*contract* (tool-less transform, capability isolation, fail-secure) is what
|
|
carries the security — the lexicon is defense-in-depth, not a wall.
|
|
- **Semantic / factual poisoning is invisible** to lexicon + entropy: a
|
|
factually false claim in clean prose carries no suspicious token. The
|
|
`grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic
|
|
core does not judge semantics; a `[judge]` implementation must be plugged in.
|
|
- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch**
|
|
between scanner and model leaves gaps.
|
|
- **Latent / dormant memory poisoning** is not judgeable at write time.
|
|
- **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a
|
|
link to a not-yet-existing target passes a per-concept write-time scan clean —
|
|
the payload is planted later, when that target is written. A scanner that sees
|
|
one document at a time cannot catch it; it needs cross-write graph re-scan
|
|
(tracked for the OKF adapter).
|
|
- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are
|
|
legitimate structure (directory listing, update log), so mode-b `import_bundle`
|
|
scans their body and frontmatter — an injection planted in a directory listing
|
|
is caught — rather than path-rejecting the whole conformant bundle. The upload
|
|
front-end keeps the opposite rule: an individual upload landing on a reserved
|
|
basename is a shadow of the listing and is refused (`allow_reserved=False`).
|
|
- **A document that *describes* attacks is a false positive.** Content whose
|
|
legitimate purpose is to document prompt-injection payloads (security notes,
|
|
this project's own corpus) trips carrier-strip / fail-secure. At the text layer
|
|
there is no way to distinguish "*about* an attack" from "*carrying* an attack";
|
|
such content needs a deliberate, explicitly-marked escaped path, never a silent
|
|
allow.
|
|
- **Insider in-place edits** by a trusted author are out of the untrusted-content
|
|
threat model.
|
|
- **Text-only.** The core is `text -> findings`: it parses no files (no
|
|
`pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the
|
|
high-untrust upload provenance. OCR-embedded instructions and multimodal stego
|
|
in images/PDFs are out of scope beyond the sanitizer's character-layer stripping.
|
|
- **Uploaded files: only the *extracted text* is scanned.** The two-stage OKF
|
|
inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`, a
|
|
dev-scoped demonstration whose `python-docx`/`python-pptx` parsers live in the
|
|
`[dev]` extra, never core `dependencies`) reads `.txt`/`.md`/`.csv`/`.docx`/
|
|
`.pptx`/`.xlsx`, folders and `.zip`, materializes them into an OKF bundle, then
|
|
guards it. What survives text extraction is **out of scope**: VBA/macros
|
|
(`.docm`/`.xlsm`/`.pptm`), OLE / embedded objects, image-embedded instructions
|
|
needing OCR, font/render steganography, and encrypted / password-protected files
|
|
— the binary layer needs a separate scanner. The front-end owns the container
|
|
threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink
|
|
refusal, CSV/XLSX formula-lead cells). Known gaps: `.pdf` extraction, and the
|
|
numeric `-`/`+` CSV false positive (an XLSX numeric cell is typed, so it does not
|
|
trip the gate).
|
|
- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first
|
|
offset are reported, so the same class matched across several channels/variants
|
|
collapses to one finding at its first location. This keeps reports readable, but
|
|
a caller that counts occurrences or needs every offset of a repeated pattern sees
|
|
only the first: a deliberate readability tradeoff, not full positional coverage.
|
|
- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate
|
|
decodes base64 blobs and re-scans the plaintext through the credential/egress set,
|
|
so a base64-*wrapped* secret surfaces as `decoded:egress:*` rather than vanishing.
|
|
A *hex*-wrapped secret does not: `entropy` exposes decoded plaintext for base64
|
|
only, so hex (and other encodings, or nested wraps) is a deliberate boundary, not
|
|
a silent miss — decode the transport layer first if you need it scanned.
|
|
|
|
## Out-of-scope (documented boundary)
|
|
|
|
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
|
steganography; query-time / runtime guardrails; semantic factuality verification.
|
|
|
|
## Design & threat model
|
|
|
|
- [Design brief](docs/BRIEF.md) — what this repo contains and why.
|
|
- [Build plan](docs/PLAN.md) — module build order and the reuse map.
|
|
|
|
The contract is extracted from a working reference implementation (the
|
|
`claude-code-llm-wiki` Stage B enrichment pipeline). Threat-model anchors: OWASP
|
|
LLM Top-10 2025 (LLM01/02/04/05/06 strongest, LLM08 boundary, LLM09/10),
|
|
PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak (CVE-2025-32711).
|
|
|
|
## License
|
|
|
|
MIT — see [LICENSE](LICENSE).
|