llm-ingestion-pipeline-security/docs/PLAN.md

# Implementation Plan — `llm-ingestion-guard` v1

Derived from `docs/BRIEF.md` plus a 2026-07-04 research pass (competitive landscape,
novelty/gap refutation, security-coverage gaps). This plan folds the verified gaps
into the v1 scope. The BRIEF is the design rationale; this is the actionable build.

## Resolved decisions (2026-07-04)

- **Name:** distribution `llm-ingestion-guard`, import `llm_ingestion_guard`. Repo
  directory stays `llm-ingestion-pipeline-security`.
- **License:** MIT.
- **Lexicon storage:** shared JSON data + thin Python loader (single source of truth,
  polyglot-ready for a later TS port).
- **Publish target:** Forgejo `open/` only (public). No PyPI, no GitHub. Install via
  `pip install git+ssh://…/llm-ingestion-guard.git`.
- **Language:** Python-first, stdlib-only core; ML/judge detectors behind extras
  (`[ml]`, `[judge]`), never in the core path.

## Positioning — A: "Guard the artifact"

> Query-time guardrails guard the answer. `llm-ingestion-guard` guards the **artifact**.

Lead with the **contract + placement** (write-path, pre-persist) + **failure-semantics**
(fail-secure toward the artifact, not fail-open toward a user). Detection is the weakest,
most-evadable layer — defense-in-depth, not the pitch.

Defensible claim, every qualifier load-bearing: *the first dependency-light,
framework-agnostic **library** that packages the write-time injection-**containment**
contract with fail-secure disposition, for unattended pipelines.* Cite OWASP LLM08:2025 /
RAG Security Cheat Sheet for legitimacy; reference Dual-LLM (Willison 2023) and CaMeL
(DeepMind 2025) as architecture lineage — inspiration, **not** equivalence.

### Claims we will NOT make (verified overclaim risks)

- NOT "prevents RAG/knowledge poisoning" — cannot catch **factual** poisoning
  (PoisonedRAG: plausible false facts, no markers, low entropy).
- NOT "blocks prompt injection" — pattern/entropy detection is bypassable (arXiv
  2504.11168, up to 100% evasion). Detection is a signal feeding disposition, never a
  sole gate.
- NOT "invented" tool-less quarantine / capability isolation — prior art (Dual-LLM, CaMeL).
- NOT "the only" ingestion-security tool — PII-scoped libs (Philter, Presidio) and
  services (AWS, HiddenLayer) exist; we are the minimal-dep *library* for the
  injection-**containment** contract.
- Positioned as **complementary** to query-time guardrails (LLM Guard, Lakera,
  LlamaFirewall), not a replacement.

## Module set (v1)

Core (BRIEF §5): `report`, `sanitize`, `fence`, `lexicon`, `entropy`, `contract`,
`output`, `disposition`.

Gap additions folded into v1 (from the security-coverage research):

- **`neutralize`** (new) — opt-in **pure** defang helpers for model OUTPUT: markdown
  images, autolinks, `[ref]` links, `data:`/external URIs, raw HTML. Mutation is opt-in
  and kept separate from the report-only gate, so design principle 3 (pure detection) and
  4 (disposition belongs to the caller) hold. Closes EchoLeak-class (CVE-2025-32711)
  persisted zero-click exfil — these are neither injection strings nor high-entropy, so
  lexicon+entropy alone miss them.
- **`output`** extended — secret/PII egress patterns (OWASP LLM02); **decode-and-rescan**
  (run the lexicon over decoded base64/hex, not just flag blob presence).
- **`fence`** hardened — randomized, unspoofable per-call delimiter; strip attacker fence
  markers from the payload first.
- **Scanner self-safety** (OWASP LLM10) — input-size cap, ReDoS-safe patterns (bounded
  quantifiers + per-scan guard), decompression guard. A scanner that hangs on crafted
  input *is* the DoS.
- **`grounding`** (interface now, impl later) — a `SourceGroundingCheck` protocol in
  core; ML/judge implementation behind the `[judge]` extra. The only structural handle on
  semantic poisoning that lexicon+entropy cannot see. Ship the seam + honest scope note.
- **Chunk-aware / sliding-window scan** — option for split-payload evasion across chunk
  boundaries.
- **`disposition` presets** — named source-trust tiers, including a **high-untrust
  user-upload / open-source preset**: QUARANTINE_REVIEW as the default and **hard-fail on
  CRITICAL**, not WARN. An automatic, unattended inbox that ingests arbitrary user uploads
  is the canonical high-untrust consumer (§4.7) — over-blocking one upload is far cheaper
  than persisting a poisoned one.

## Build order (TDD — a failing test FIRST for each)

1. `report` — findings / Report dataclass (shared return type)
2. `sanitize` — carrier stripping (zero-width, BIDI, Unicode-tag, HTML comment, `data:`);
   byte-identical invariant on clean input, removes only
3. `entropy` — shannon / base64-like / hex-blob; decode-and-rescan support
4. `lexicon` — JSON data + loader + `scan`; ReDoS-safe; size cap; port from the
   `llm-security` JS seed (CRITICAL/HIGH/MEDIUM/HYBRID + normalization/homoglyph/rot13/
   unicode-tag)
5. `fence` — randomized delimiter; marker-strip
6. `neutralize` — opt-in defang helpers
7. `output` — compose lexicon+entropy over output; PII/secret egress; decode-and-rescan
8. `disposition` — WARN | QUARANTINE_REVIEW | FAIL_SECURE; source-trust + intra-document
   provenance tiering (§4.7); compound-signal fail-secure (§4.6); fail-**closed** when the
   scanner itself errors
9. `contract` — tool-less assert; per-stage credential allowlist; env-scoping helpers
10. `grounding` — protocol/interface only in core
11. Top-level `__init__` wiring the §6 checklist; `pyproject.toml`; `LICENSE`; README
    (positioning + honest limitations); adversarial + false-positive corpora; the
    end-to-end showcase pipeline test (see Test strategy — built LAST)

## Reuse map — `llm-security` v7.8.0 (MIT, same author)

Maximal reuse: most detection logic is a JS→Python **port**, not new code.

| Our module | Seed in `llm-security` | Notes |
|---|---|---|
| `lexicon` | `scanners/lib/injection-patterns.mjs` | CRITICAL/HIGH/MEDIUM/HYBRID tables + `scanForInjection` (variant-set: raw / normalized / homoglyph-folded / rot13, dedup by label) + `checkCognitiveLoadTrap`. The load-bearing port. |
| `sanitize` | `scanners/unicode-scanner.mjs` + `string-utils.mjs` | zero-width / BIDI / Unicode-tag / homoglyph detection + `stripBidiOverrides` / `decodeUnicodeTags` / `containsUnicodeTags`. |
| normalization | `string-utils.mjs` `normalizeForScan` | decode chain (unicode-tags → bidi → HTML-entities → unicode/hex/url escapes → base64), `foldHomoglyphs`, `rot13`, `collapseLetterSpacing`. Feeds decode-and-rescan. |
| `entropy` | `scanners/entropy-scanner.mjs` + `string-utils.mjs` | `shannonEntropy` / `isBase64Like` / `isHexBlob` / `tryDecodeBase64`; length-calibrated thresholds; `isFalsePositive` suppression rules; `DATA_URI_PREFIXES`. |
| `report` | `scanners/lib/output.mjs` + `severity.mjs` | `finding` / `scannerResult` envelope; `SEVERITY`; `riskScore` / `verdict` / `riskBand` / `owaspCategorize`. |
| `output` secret/PII egress | `knowledge/secrets-patterns.md` | Ready secret regex set (AWS/Azure/GCP/GitHub/npm/OpenAI/Anthropic/PEM/DB-conn/password/JWT) + FP-suppression rules. Seeds the LLM02 egress gap directly. |
| tests (corpora) | `examples/prompt-injection-showcase/payloads.json` | Labeled adversarial + false-positive corpus (category/severity/expected). Maps ~1:1 to §9. Plus `knowledge/{attack-mutations,attack-scenarios,signatures}.json`. |
| wiring reference | `hooks/pre-prompt-inject-scan.mjs` (input), `post-mcp-verify.mjs` (output gate) | How the scan is invoked at input vs output — informs our `output` module + the §6 checklist. |

**Port caveats (JS→Python, stdlib-only):**
- `/…/i` → `re.I`; `Buffer.from(s,'base64')` → `base64.b64decode`; `codePointAt`/`fromCodePoint` → `ord`/`chr`.
- `TOKEN_RE = /[\p{L}\p{N}_]+/gu` — Python `re` has no `\p{L}`; use `\w` under `re.UNICODE` (Py3 default) or `str.isalnum()`. No `regex` dependency.
- **ReDoS**: some patterns (sub-agent spawn with nested `.*?`) backtrack; Python `re` has no timeout → enforce input-size cap + review/bound these patterns (the self-safety must-have).
- Lexicon ships as **JSON** (regex source + label + severity), compiled by a thin Python loader — decouples data from engine for the future TS port.

## Test strategy (BRIEF §9 + additions)

- **Adversarial corpus** — one payload per lexicon class incl. obfuscated + multi-language;
  measure and report *recall*.
- **False-positive corpus** — content legitimately discussing injection (security docs,
  changelogs); assert WARN-not-block default; hard-fail is an explicit opt-in.
- **Sanitizer invariant** — clean input returns byte-identical with an all-zero report.
- **Contract asserters** — a tool-carrying request and a credential-leaking stage env both
  raise; the happy path passes.
- **Self-safety** — pathological/ReDoS-prone and oversize input return within a bound,
  never hang.
- **Neutralize** — active-content output is defanged; clean output is byte-identical.
- **End-to-end showcase (the FINAL deliverable, built last).** One realistic
  piece of ingested content that carries *many* vulnerabilities at once — visible
  + zero-width/BIDI/Unicode-tag stego carriers, homoglyph + whole-string-base64 +
  rot13-hidden injection, HTML-obfuscated and cognitive-load-buried payloads,
  secret/PII egress in the enriched output, and active-content exfil links — run
  through a mock ingestion pipeline (sanitize → lexicon+entropy+decode-rescan →
  output gate → disposition). Assert every planted vulnerability is caught and
  disposition fails secure. Doubles as the README's worked example. Inspiration:
  `llm-security/examples/{prompt-injection-showcase,poisoned-claude-md,lethal-trifecta-walkthrough,toxic-agent-demo}`.
- **No network in any test.**

## README "Honest limitations" (shipped as a control)

Concede plainly (the concession prevents false assurance, which is itself a control):
structural unsolvability at the text layer; adversarial-ML evasion survives normalization;
tokenizer mismatch; semantic/factual poisoning invisible to lexicon+entropy; latent/dormant
memory poisoning not judgeable at write-time; insider in-place edits; text-only (no
multimodal).

## Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
steganography; query-time/runtime guardrails; semantic factuality verification. The
`contract` tool-less assertion is the write-time analogue of runtime least-privilege.

**Text-extraction boundary.** For pipelines that ingest arbitrary user uploads (an upload
inbox that ingests automatically), the library stays `text -> findings`: it does not parse
files (no `pypdf`/`python-docx`/archive deps in the core). The pipeline extracts text
first, then scans the extracted text + the enriched output with provenance tagged as the
high-untrust upload tier. Binary/multimodal carrier detection (OCR-embedded instructions in
images/PDFs, stego) is out of scope beyond the sanitizer's character-layer stripping and
the self-safety size/decompression guard — and must be conceded in the honest-limitations
section, because a high-untrust, unattended pipeline is exactly where assuming
uncovered-coverage is most dangerous.

## Threat-model anchors

OWASP LLM Top-10 2025: LLM01, LLM02, LLM04, LLM05, LLM06 (strongest coverage), LLM08
(boundary), LLM09, LLM10. Research anchors: PoisonedRAG (arXiv 2402.07867), guardrail
evasion (2504.11168), EchoLeak (CVE-2025-32711), RAGShield (2604.00387), CaMeL
(2503.18813), Dual-LLM (Willison), and the litellm supply-chain compromise (corroborates
the minimal-dependency thesis).

## v0.2+ stream sequencing (post-v0.1.0, decided 2026-07-06)

v0.1.0 is tagged — the format-agnostic text core (modules 1–11, hardened at
`5397ba1`, released at `df30c7b`). Three forward streams, **all to be built**;
the order is dependency-driven, not preference:

1. **OKF v0.2 hardening** — the OKF adapter + the 8 tasks in
   `docs/OKF-INGESTION-BRIEF.md` §8 (internal order T8 → T1–T4+T6 → T7 → T5).
2. **Wire consumers** — `claude-code-llm-wiki`, `ms-ai-architect`,
   `portfolio-optimiser` (its OKF-upload-inbox is the flagship).
3. **Node/polyglot port** — one polyglot repo over the shared JSON lexicon
   (BRIEF §13.5/§8); driver = `ms-ai-architect` convergence.

**Why this order (dependencies, not taste):**

- The flagship consumer (`portfolio-optimiser`'s OKF-upload-inbox) *is* an OKF
  surface. Wiring it before OKF hardening ships a knowingly-incomplete guard
  into the exact case that needs it → **stream 2 depends on stream 1**.
- The Node port must port a *settled* public surface. OKF v0.2 adds the
  OKF-adapter surface; porting before it = porting a moving target and exactly
  the §13.3 split-drift the polyglot rule exists to forbid → **stream 3 after 1**.
- Within stream 1: **T8** (residual-risk docs, brief §7.2/§7.4) is the cheap,
  decision-free first step — it lands independently and makes the live README
  honest about uncovered OKF surfaces before anything else is built. **T3/T4/T5
  are blocked on an OKF-spec confirmation pass** (brief §9 "operator premise":
  concept-ID=path-minus-`.md`, reserved `index.md`/`log.md`, the `resource`
  field, `description`→`index.md` were taken from the brief, not checked against
  `SPEC.md`).

**Splittable early win (optional):** text-only consumers can be wired at v0.1.0
today — for them the guard is already complete. Folded behind stream 1 by
default (editing other repos = context switch + weaker feedback than the
flagship). Pull forward only on explicit request.