open/llm-ingestion-pipeline-security
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions
Reusable, minimal, dependency-light defensive layer for LLM ingestion (write-time) pipelines — sanitize, fence, tool-less quarantined transform, capability isolation, output-scan-before-persist, fail-secure.
8 commits 1 branch 2 tags 443 KiB
  • Python 100%
Find a file
Kjell Tore Guttormsen a55404460a docs(plan): add end-to-end showcase pipeline test as the final deliverable 
One realistic content sample carrying many vulnerabilities at once, run through
a mock ingestion pipeline (sanitize -> lexicon+entropy+decode-rescan -> output
gate -> disposition); assert every planted vuln is caught and disposition fails
secure. Doubles as the README worked example. Added to build-order step 11 and
the test strategy; tracked as task #11. Inspiration: llm-security/examples/*.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K8GmKRCdsPjWYAKWsNgeQS
2026-07-04 17:23:17 +02:00
docs docs(plan): add end-to-end showcase pipeline test as the final deliverable 2026-07-04 17:23:17 +02:00
src/llm_ingestion_guard feat(lexicon): JSON injection lexicon + variant-set scan (TDD) [skip-docs] 2026-07-04 17:20:21 +02:00
tests feat(lexicon): JSON injection lexicon + variant-set scan (TDD) [skip-docs] 2026-07-04 17:20:21 +02:00
.gitignore chore: repo scaffolding via /repo-init 2026-07-04 08:39:23 +02:00
CHANGELOG.md chore: repo scaffolding via /repo-init 2026-07-04 08:39:23 +02:00
CLAUDE.md chore: repo scaffolding via /repo-init 2026-07-04 08:39:23 +02:00
LICENSE feat: scaffold package + report and sanitize modules (TDD) 2026-07-04 09:24:20 +02:00
pyproject.toml feat: scaffold package + report and sanitize modules (TDD) 2026-07-04 09:24:20 +02:00
README.md docs: design brief for reusable LLM ingestion-pipeline security library 2026-07-04 06:18:32 +02:00

README.md

llm-ingestion-pipeline-security

A reusable, minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails.

Where mature guardrails (LLM Guard, NeMo Guardrails, Rebuff, Vigil, …) sit between a user and a model at query time, this library hardens the other shape: untrusted content flowing through an LLM enrichment/summarization/extraction step into a persisted, downstream-consumed artifact (RAG corpus, knowledge base, wiki). It packages the architectural contract — sanitize → fence → tool-less quarantined transform → per-stage capability isolation → scan output before commit → fail-secure — as composable, framework-agnostic code.

Status: brief / pre-implementation. Start with the design brief:

The contract is extracted from a working reference implementation (the claude-code-llm-wiki Stage B enrichment pipeline).