- Python 100%
Kjell Tore Guttormsen
19981623f5
Module 7 of the build order: the last gate before model output is persisted.
Composes scan_lexicon + scan_entropy over the emitted text, feeds each base64
blob entropy decoded back through the lexicon (decode-and-rescan → decoded:*
findings with blob offset), and adds the LLM02 secret/credential egress layer
(cloud/provider keys, PEM headers, DB conn-strings, JWT, labelled
password/secret/api-key assignments) with placeholder/varref FP-suppression.
Report-only (never mutates; neutralize is the separate opt-in mutator). Evidence
carries only a description + match length — never the secret value. Self-safe:
single input-size cap, linear egress patterns. PEM patterns use -{5} form so the
module itself never trips a secret scanner. 26 new tests; 108 green total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
|
||
|---|---|---|
| docs | ||
| src/llm_ingestion_guard | ||
| tests | ||
| .gitignore | ||
| CHANGELOG.md | ||
| CLAUDE.md | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
llm-ingestion-pipeline-security
A reusable, minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails.
Where mature guardrails (LLM Guard, NeMo Guardrails, Rebuff, Vigil, …) sit between a user and a model at query time, this library hardens the other shape: untrusted content flowing through an LLM enrichment/summarization/extraction step into a persisted, downstream-consumed artifact (RAG corpus, knowledge base, wiki). It packages the architectural contract — sanitize → fence → tool-less quarantined transform → per-stage capability isolation → scan output before commit → fail-secure — as composable, framework-agnostic code.
Status: brief / pre-implementation. Start with the design brief:
- Design brief — what this repo should contain and why.
The contract is extracted from a working reference implementation (the
claude-code-llm-wiki Stage B enrichment pipeline).