From b505f704858baeab6f70cf895391c529385f94da Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Sat, 4 Jul 2026 06:18:32 +0200 Subject: [PATCH] docs: design brief for reusable LLM ingestion-pipeline security library Brief-stage repo. docs/BRIEF.md defines a minimal, framework-agnostic library that packages the write-time ingestion contract (sanitize -> fence -> tool-less quarantined transform -> per-stage capability isolation -> scan output before persist -> fail-secure) as reusable code. Positioned honestly against query-time guardrails (LLM Guard, NeMo, Rebuff, Vigil) with a prior-art verification log. Reference implementation: claude-code-llm-wiki Stage B. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HPAmFyEVWbwvmSNVdXTu4d --- CLAUDE.md | 21 +++++ README.md | 19 ++++ docs/BRIEF.md | 237 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 277 insertions(+) create mode 100644 CLAUDE.md create mode 100644 README.md create mode 100644 docs/BRIEF.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d59bc71 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,21 @@ +# llm-ingestion-pipeline-security + +## Kontekst + +Gjenbrukbar, minimal defensiv layer for LLM **ingestion**-pipelines (write-time), +til forskjell fra query-time chatbot-guardrails. Pakker det arkitektoniske +kontraktet — sanitize → fence → tool-less karantenert transform → per-stadium +capability-isolasjon → scan output før commit → fail-secure — som komponerbar, +framework-agnostisk kode. + +Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`). +Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen. + +Repoet er på **brief-stadiet**. Start med `docs/BRIEF.md`. + +## Konvensjoner + +- Norsk for dialog og planer, engelsk for kode og innhold (repoet er ment publisert). +- Ingen GitHub — kun Forgejo (`git.fromaitochitta.com`) hvis/når publisert. +- Ingen remote satt ennå; ingen push før operatøren bestemmer publisering. +- Minimal-dependency: stdlib-first kjerne; ML/judge-detektorer bak extras. diff --git a/README.md b/README.md new file mode 100644 index 0000000..a5b0262 --- /dev/null +++ b/README.md @@ -0,0 +1,19 @@ +# llm-ingestion-pipeline-security + +A reusable, minimal, dependency-light defensive layer for **LLM ingestion +pipelines** — the write-time siblings of query-time chatbot guardrails. + +Where mature guardrails (LLM Guard, NeMo Guardrails, Rebuff, Vigil, …) sit +between a user and a model at query time, this library hardens the other shape: +untrusted content flowing through an LLM enrichment/summarization/extraction step +into a **persisted, downstream-consumed artifact** (RAG corpus, knowledge base, +wiki). It packages the architectural contract — sanitize → fence → tool-less +quarantined transform → per-stage capability isolation → scan output before +commit → fail-secure — as composable, framework-agnostic code. + +**Status:** brief / pre-implementation. Start with the design brief: + +- [Design brief](docs/BRIEF.md) — what this repo should contain and why. + +The contract is extracted from a working reference implementation (the +`claude-code-llm-wiki` Stage B enrichment pipeline). diff --git a/docs/BRIEF.md b/docs/BRIEF.md new file mode 100644 index 0000000..64b8951 --- /dev/null +++ b/docs/BRIEF.md @@ -0,0 +1,237 @@ +# Brief: `llm-ingestion-pipeline-security` + +**A reusable, minimal, dependency-light defensive layer for LLM *ingestion* +pipelines — the write-time siblings of query-time chatbot guardrails.** + +Status: brief / pre-implementation. This document defines what the repo should +contain and why. No code yet. + +--- + +## 1. One-line purpose + +Give any pipeline that runs untrusted content through a large language model and +then **persists the result into a downstream-consumed artifact** (RAG corpus, +knowledge base, wiki, embeddings store) a small, framework-agnostic library that +packages the *architectural contract* for doing it safely: sanitize → fence → +tool-less quarantined transform → capability isolation → scan output before +commit → fail-secure. + +## 2. The problem, stated precisely + +The mature tooling for prompt injection is **query-time**: it sits between a user +(or an agent) and a model, at inference, on live traffic. Ingestion is a +different shape and is underserved: + +- **Direction.** The danger flows from *content* (documents, changelogs, scraped + web pages, user uploads, tickets) through an LLM *enrichment/summarization/ + extraction/classification* step, into a **persisted artifact** that other + systems and agents later read. +- **Two distinct failure modes**, only the first of which query-time guardrails + address: + 1. **Injection steers the enrichment step** — untrusted content contains + instructions that hijack the summarizer/extractor. + 2. **Poisoned content is published** — the artifact itself becomes the attack. + A verbatim-quoted payload, or an LLM-emitted instruction, lands in the + knowledge base and later poisons a *downstream* agent's context. This is + RAG/memory poisoning committed at *write* time, and a query-time guardrail + on the downstream reader never sees where it came from. +- **Non-interactive, unattended.** Ingestion typically runs headless on a + schedule. There is no human in the loop to notice a weird answer. Failure + discipline (fail-secure, alerting, no silent verbatim fallback under attack) + matters more here than in a chat UI. + +## 3. Prior art and honest positioning + +Injection *detection* is a crowded space. This repo must not pretend otherwise. + +| Tool | Orientation | +|------|-------------| +| LLM Guard, Guardrails AI, NeMo Guardrails | Query-time I/O guardrails / dialogue policy | +| Rebuff | Self-hardening query-time detector (vector store of past attacks) | +| Vigil, Lakera Guard, Vijil | Query-time injection classifiers / detection servers | +| LlamaFirewall (Meta) | Agent guardrail framework (I/O, tool calls) | +| Resk | Python guardrail lib for LLM API calls | +| CleanBase, DataFilter (research) | Detecting/cleaning malicious docs in RAG DBs | + +The gap this repo fills is **not** "no one detects injection." It is: + +> A small **library** (not a hosted service, not a fine-tuned model) that +> packages the **write-time ingestion contract** — quarantined tool-less +> transform, per-stage capability isolation, output-scan-before-persist, +> fail-secure disposition — as composable, framework-agnostic, minimal-dependency +> code, with corpus-aware false-positive control. + +**Necessary honesty (verified, see §11):** 2025 research shows character-level +and adversarial-ML evasion defeats most pattern- and classifier-based detectors +while keeping the payload legible to the target model. Therefore the value of +this library is **architecture and defense-in-depth**, not a detector that +"solves" injection. The lexicon is one WARN-class layer; the *contract* (a +tool-less call that cannot act on a successful injection, credentials the +hijacked step never holds, output scanned before it is trusted) is the load- +bearing part. The brief must lead with the contract, not the regex. + +## 4. Design principles + +1. **Minimal dependencies.** Stdlib-first core. Optional pluggable detectors + (an embedding classifier, an LLM-judge) live behind extras, never in the core + path. A pipeline should be able to adopt the core with zero new runtime deps. +2. **Framework-agnostic.** Works with any SDK (Anthropic, OpenAI, local). The + library never makes the model call; it hardens the call the pipeline makes. +3. **Pure functions.** Detection is `text -> findings`. No network, no telemetry, + no global state. Portable, testable, auditable. +4. **Disposition belongs to the caller.** The library reports; the pipeline + decides WARN vs quarantine-review vs fail-secure-halt. Defaults are provided, + but blocking is never imposed — see principle 5. +5. **Corpus-aware false-positive control.** A corpus that *legitimately discusses* + prompt injection (security docs, changelogs, a wiki about AI safety) will trip + any broad lexicon. The library must make WARN-not-block the easy default and + hard-fail an explicit, calibrated opt-in. Silent over-blocking of legitimate + content is a failure mode, not a safe default. +6. **Fail-secure under compound signals.** Injection-scan hit *and* a transform + failure is treated as a probable forced-fallback attack: halt + alert, never + an auto-committed verbatim entry. + +## 5. Proposed module layout + +``` +src/llm_ingest_security/ + sanitize/ Carrier stripping: invisible chars (Unicode Tags, zero-width, + bidi), HTML comments, data: URIs. Byte-identical on clean input; + removes only, never rewrites. Per-class counts for a WARN gate. + fence/ Spotlighting helpers: wrap untrusted input in markers, and strip + the markers FROM the input first so it cannot break out of its + own fence. + lexicon/ Semantic injection patterns as DATA (regex + label + severity), + tiered CRITICAL/HIGH/MEDIUM: override, spoofed headers/tags, + identity redefinition, config attacks, NL-indirection + (fetch-and-execute, send-to-external, read-dotfiles, + extract-and-exfiltrate), sub-agent spawn, leetspeak/homoglyph, + multi-language. Curated, versioned, re-syncable. + entropy/ base64/hex blob detection for smuggled encoded instructions or + encoded exfil (complements secret-pattern scanning). + contract/ The quarantine asserters — the differentiator: + - assert the model request carries NO tools + - assert per-stage credential allowlist (the enrichment step + sees exactly the one key it needs, and none of the publish + credentials) + - capability-isolation helpers (env scoping) + output/ Output-side scanner: run the lexicon + entropy over the model's + OUTPUT (summary, verbatim quotes, extracted fields) BEFORE the + artifact is persisted. This is the RAG-poisoning gate. + disposition/ Policy object: WARN | QUARANTINE_REVIEW | FAIL_SECURE per gate, + plus the compound-signal fail-secure rule. + report/ Structured findings: class, count, severity, offset, source + (input|output), disposition. +``` + +## 6. The reusable contract (adopt-this checklist) + +The actual product is this checklist, encoded as code a pipeline wires in order: + +1. **Sanitize before fence.** Strip carrier classes from untrusted input first. +2. **Fence untrusted input.** Spotlight-mark it; strip markers from the payload. +3. **Tool-less transform.** Call the model with zero tools. A successful + injection then has nothing to act with. +4. **Per-stage capability isolation.** The enrichment stage holds only the model + key; the publish stage holds only the publish credential; no stage holds both. +5. **Treat output as data.** Parse to a frozen schema; reject on structural + violation. The output never reaches a shell, git, or a filesystem path. +6. **Scan output before persist.** Run the lexicon + entropy over the emitted + text. Verbatim-carried payloads and model-emitted instructions are caught here. +7. **Fail-secure on compound signals.** Injection hit + transform failure = halt + + alert, never a silent verbatim commit. +8. **Minimal alert payloads.** Alert with a gate code + run ID, never content. + Treat the alert channel/topic as a secret. + +## 7. Non-goals + +- **Not** a query-time chatbot / agent guardrail (that space is served). +- **Not** a fine-tuned model or a required ML classifier — deterministic core; + ML detectors are optional, pluggable extras. +- **Not** a hosted service or SaaS. +- **Not** a silver bullet — explicitly a defense-in-depth layer whose lexicon is + bypassable in isolation; the contract is what carries the security. + +## 8. Language, packaging, licensing + +- **Python first** — most ingestion pipelines are Python. Stdlib-only core; + `pyproject.toml` with optional extras (`[ml]`, `[judge]`). +- Framework-agnostic public API; no SDK imported by the core. +- A Node/TypeScript port is a plausible follow-on (a shared JSON lexicon as the + single source of truth across both). +- Permissive license (MIT or Apache-2.0) so it is droppable into any pipeline. + +## 9. Test strategy + +- **Adversarial corpus.** A seeded set of payloads (one per lexicon class, + including obfuscated and multi-language variants); measure and report *recall*. +- **False-positive corpus.** Real content that legitimately discusses injection + (security docs, changelogs); assert the default disposition is WARN, not block, + and that hard-fail is an explicit opt-in. +- **Sanitizer invariant.** Clean input returns byte-identical with an all-zero + report; the sanitizer only removes. +- **Contract asserters.** A tool-carrying request and a credential-leaking stage + env both raise; the happy path passes. +- **No network in any test.** + +## 10. Threat-model grounding + +- OWASP LLM Top 10: **LLM01 Prompt Injection**, output-handling, and the RAG / + knowledge-poisoning vectors. +- Google DeepMind "AI Agent Traps" (Franklin et al., 2025): Latent Memory + Poisoning and RAG Knowledge Poisoning map directly onto the write-time artifact. +- Research anchors: CleanBase (malicious docs in RAG DBs), DataFilter, and the + 2025 guardrail-evasion literature that motivates the defense-in-depth framing. + +## 11. Verification log (prior-art claims + sources) + +Per the operator's verification duty — key claims in this brief and where they +are grounded: + +- *Injection detection is crowded and query-time oriented* — LLM Guard, NeMo + Guardrails, Guardrails AI, Rebuff, Vigil, Vijil, LlamaFirewall, Resk, verified + via the 2026 guardrails comparison and tool docs. + - https://dev.to/agdex_ai/best-ai-agent-security-guardrails-tools-in-2026-llm-guard-vs-nemo-vs-guardrails-ai-5e5d +- *Pattern/classifier detection alone is bypassable (defense-in-depth framing)* — + "Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks", arXiv + 2504.11168 (character injection defeats most guardrails). + - https://arxiv.org/abs/2504.11168 +- *Write-time RAG-DB poisoning is a researched but under-tooled vector* — + CleanBase (detecting malicious documents in RAG knowledge databases). + - https://arxiv.org/pdf/2605.00460 +- *Multi-agent output-guard-before-release architecture exists in research* — + "A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks", + arXiv 2509.14285. + - https://arxiv.org/abs/2509.14285 +- *LlamaFirewall as an open-source guardrail reference* — arXiv 2505.03574. + - https://arxiv.org/pdf/2505.03574 + +Marked **assumed, not verified**: the specific claim that no existing *library* +packages the full write-time contract as minimal-dependency code. The search +found no such library, but absence of evidence is not proof; a focused survey of +PyPI + GitHub topics should confirm before the README makes a novelty claim. + +## 12. Reference implementation + +The contract in §6 is not hypothetical — it is extracted from a working pipeline: +the `claude-code-llm-wiki` Stage B enrichment path (`tools/wiki_ingest/`), +which already implements sanitize + spotlight-fence + tool-less quarantined SDK +call + per-stage credential isolation + schema-validated output + fail-secure +`ForcedFallbackHalt`. That pipeline is the reference implementation and the first +consumer; this repo generalizes its pattern for reuse. The semantic lexicon seed +is the `injection-patterns.mjs` table from the `llm-security` plugin (a pure +regex+label+severity dataset, portable as data). + +## 13. Open decisions for the operator + +1. **Name** — `llm-ingestion-pipeline-security` is descriptive but long; a + shorter package name (e.g. `ingestguard`) may be worth it before first publish. +2. **License** — MIT vs Apache-2.0. +3. **Relationship to `llm-security`** — sibling repos sharing one lexicon dataset + as the single source of truth, or fully independent? Avoid two drifting copies + of the pattern table. +4. **Publish target** — Forgejo (`git.fromaitochitta.com`), and whether a public + `open/` mirror. No GitHub per house policy. +5. **Python-only vs polyglot** from day one. +6. **Maintenance model** — solo fork-and-own (as `llm-security`), or open to PRs.