llm-ingestion-pipeline-security/docs/PLAN.md

# Implementation Plan — `llm-ingestion-guard` v1

Derived from `docs/BRIEF.md` plus a 2026-07-04 research pass (competitive landscape,
novelty/gap refutation, security-coverage gaps). This plan folds the verified gaps
into the v1 scope. The BRIEF is the design rationale; this is the actionable build.

## Resolved decisions (2026-07-04)

- **Name:** distribution `llm-ingestion-guard`, import `llm_ingestion_guard`. Repo
  directory stays `llm-ingestion-pipeline-security`.
- **License:** MIT.
- **Lexicon storage:** shared JSON data + thin Python loader (single source of truth,
  polyglot-ready for a later TS port).
- **Publish target:** Forgejo `open/` only (public). No PyPI, no GitHub. Install via
  `pip install git+ssh://…/llm-ingestion-guard.git`.
- **Language:** Python-first, stdlib-only core; ML/judge detectors behind extras
  (`[ml]`, `[judge]`), never in the core path.

## Positioning — A: "Guard the artifact"

> Query-time guardrails guard the answer. `llm-ingestion-guard` guards the **artifact**.

Lead with the **contract + placement** (write-path, pre-persist) + **failure-semantics**
(fail-secure toward the artifact, not fail-open toward a user). Detection is the weakest,
most-evadable layer — defense-in-depth, not the pitch.

Defensible claim, every qualifier load-bearing: *the first dependency-light,
framework-agnostic **library** that packages the write-time injection-**containment**
contract with fail-secure disposition, for unattended pipelines.* Cite OWASP LLM08:2025 /
RAG Security Cheat Sheet for legitimacy; reference Dual-LLM (Willison 2023) and CaMeL
(DeepMind 2025) as architecture lineage — inspiration, **not** equivalence.

### Claims we will NOT make (verified overclaim risks)

- NOT "prevents RAG/knowledge poisoning" — cannot catch **factual** poisoning
  (PoisonedRAG: plausible false facts, no markers, low entropy).
- NOT "blocks prompt injection" — pattern/entropy detection is bypassable (arXiv
  2504.11168, up to 100% evasion). Detection is a signal feeding disposition, never a
  sole gate.
- NOT "invented" tool-less quarantine / capability isolation — prior art (Dual-LLM, CaMeL).
- NOT "the only" ingestion-security tool — PII-scoped libs (Philter, Presidio) and
  services (AWS, HiddenLayer) exist; we are the minimal-dep *library* for the
  injection-**containment** contract.
- Positioned as **complementary** to query-time guardrails (LLM Guard, Lakera,
  LlamaFirewall), not a replacement.

## Module set (v1)

Core (BRIEF §5): `report`, `sanitize`, `fence`, `lexicon`, `entropy`, `contract`,
`output`, `disposition`.

Gap additions folded into v1 (from the security-coverage research):

- **`neutralize`** (new) — opt-in **pure** defang helpers for model OUTPUT: markdown
  images, autolinks, `[ref]` links, `data:`/external URIs, raw HTML. Mutation is opt-in
  and kept separate from the report-only gate, so design principle 3 (pure detection) and
  4 (disposition belongs to the caller) hold. Closes EchoLeak-class (CVE-2025-32711)
  persisted zero-click exfil — these are neither injection strings nor high-entropy, so
  lexicon+entropy alone miss them.
- **`output`** extended — secret/PII egress patterns (OWASP LLM02); **decode-and-rescan**
  (run the lexicon over decoded base64/hex, not just flag blob presence).
- **`fence`** hardened — randomized, unspoofable per-call delimiter; strip attacker fence
  markers from the payload first.
- **Scanner self-safety** (OWASP LLM10) — input-size cap, ReDoS-safe patterns (bounded
  quantifiers + per-scan guard), decompression guard. A scanner that hangs on crafted
  input *is* the DoS.
- **`grounding`** (interface now, impl later) — a `SourceGroundingCheck` protocol in
  core; ML/judge implementation behind the `[judge]` extra. The only structural handle on
  semantic poisoning that lexicon+entropy cannot see. Ship the seam + honest scope note.
- **Chunk-aware / sliding-window scan** — option for split-payload evasion across chunk
  boundaries.
- **`disposition` presets** — named source-trust tiers, including a **high-untrust
  user-upload / open-source preset**: QUARANTINE_REVIEW as the default and **hard-fail on
  CRITICAL**, not WARN. An automatic, unattended inbox that ingests arbitrary user uploads
  is the canonical high-untrust consumer (§4.7) — over-blocking one upload is far cheaper
  than persisting a poisoned one.

## Build order (TDD — a failing test FIRST for each)

1. `report` — findings / Report dataclass (shared return type)
2. `sanitize` — carrier stripping (zero-width, BIDI, Unicode-tag, HTML comment, `data:`);
   byte-identical invariant on clean input, removes only
3. `entropy` — shannon / base64-like / hex-blob; decode-and-rescan support
4. `lexicon` — JSON data + loader + `scan`; ReDoS-safe; size cap; port from the
   `llm-security` JS seed (CRITICAL/HIGH/MEDIUM/HYBRID + normalization/homoglyph/rot13/
   unicode-tag)
5. `fence` — randomized delimiter; marker-strip
6. `neutralize` — opt-in defang helpers
7. `output` — compose lexicon+entropy over output; PII/secret egress; decode-and-rescan
8. `disposition` — WARN | QUARANTINE_REVIEW | FAIL_SECURE; source-trust + intra-document
   provenance tiering (§4.7); compound-signal fail-secure (§4.6); fail-**closed** when the
   scanner itself errors
9. `contract` — tool-less assert; per-stage credential allowlist; env-scoping helpers
10. `grounding` — protocol/interface only in core
11. Top-level `__init__` wiring the §6 checklist; `pyproject.toml`; `LICENSE`; README
    (positioning + honest limitations); adversarial + false-positive corpora; the
    end-to-end showcase pipeline test (see Test strategy — built LAST)

## Reuse map — `llm-security` v7.8.0 (MIT, same author)

Maximal reuse: most detection logic is a JS→Python **port**, not new code.

| Our module | Seed in `llm-security` | Notes |
|---|---|---|
| `lexicon` | `scanners/lib/injection-patterns.mjs` | CRITICAL/HIGH/MEDIUM/HYBRID tables + `scanForInjection` (variant-set: raw / normalized / homoglyph-folded / rot13, dedup by label) + `checkCognitiveLoadTrap`. The load-bearing port. |
| `sanitize` | `scanners/unicode-scanner.mjs` + `string-utils.mjs` | zero-width / BIDI / Unicode-tag / homoglyph detection + `stripBidiOverrides` / `decodeUnicodeTags` / `containsUnicodeTags`. |
| normalization | `string-utils.mjs` `normalizeForScan` | decode chain (unicode-tags → bidi → HTML-entities → unicode/hex/url escapes → base64), `foldHomoglyphs`, `rot13`, `collapseLetterSpacing`. Feeds decode-and-rescan. |
| `entropy` | `scanners/entropy-scanner.mjs` + `string-utils.mjs` | `shannonEntropy` / `isBase64Like` / `isHexBlob` / `tryDecodeBase64`; length-calibrated thresholds; `isFalsePositive` suppression rules; `DATA_URI_PREFIXES`. |
| `report` | `scanners/lib/output.mjs` + `severity.mjs` | `finding` / `scannerResult` envelope; `SEVERITY`; `riskScore` / `verdict` / `riskBand` / `owaspCategorize`. |
| `output` secret/PII egress | `knowledge/secrets-patterns.md` | Ready secret regex set (AWS/Azure/GCP/GitHub/npm/OpenAI/Anthropic/PEM/DB-conn/password/JWT) + FP-suppression rules. Seeds the LLM02 egress gap directly. |
| tests (corpora) | `examples/prompt-injection-showcase/payloads.json` | Labeled adversarial + false-positive corpus (category/severity/expected). Maps ~1:1 to §9. Plus `knowledge/{attack-mutations,attack-scenarios,signatures}.json`. |
| wiring reference | `hooks/pre-prompt-inject-scan.mjs` (input), `post-mcp-verify.mjs` (output gate) | How the scan is invoked at input vs output — informs our `output` module + the §6 checklist. |

**Port caveats (JS→Python, stdlib-only):**
- `/…/i` → `re.I`; `Buffer.from(s,'base64')` → `base64.b64decode`; `codePointAt`/`fromCodePoint` → `ord`/`chr`.
- `TOKEN_RE = /[\p{L}\p{N}_]+/gu` — Python `re` has no `\p{L}`; use `\w` under `re.UNICODE` (Py3 default) or `str.isalnum()`. No `regex` dependency.
- **ReDoS**: some patterns (sub-agent spawn with nested `.*?`) backtrack; Python `re` has no timeout → enforce input-size cap + review/bound these patterns (the self-safety must-have).
- Lexicon ships as **JSON** (regex source + label + severity), compiled by a thin Python loader — decouples data from engine for the future TS port.

## Test strategy (BRIEF §9 + additions)

- **Adversarial corpus** — one payload per lexicon class incl. obfuscated + multi-language;
  measure and report *recall*.
- **False-positive corpus** — content legitimately discussing injection (security docs,
  changelogs); assert WARN-not-block default; hard-fail is an explicit opt-in.
- **Sanitizer invariant** — clean input returns byte-identical with an all-zero report.
- **Contract asserters** — a tool-carrying request and a credential-leaking stage env both
  raise; the happy path passes.
- **Self-safety** — pathological/ReDoS-prone and oversize input return within a bound,
  never hang.
- **Neutralize** — active-content output is defanged; clean output is byte-identical.
- **End-to-end showcase (the FINAL deliverable, built last).** One realistic
  piece of ingested content that carries *many* vulnerabilities at once — visible
  + zero-width/BIDI/Unicode-tag stego carriers, homoglyph + whole-string-base64 +
  rot13-hidden injection, HTML-obfuscated and cognitive-load-buried payloads,
  secret/PII egress in the enriched output, and active-content exfil links — run
  through a mock ingestion pipeline (sanitize → lexicon+entropy+decode-rescan →
  output gate → disposition). Assert every planted vulnerability is caught and
  disposition fails secure. Doubles as the README's worked example. Inspiration:
  `llm-security/examples/{prompt-injection-showcase,poisoned-claude-md,lethal-trifecta-walkthrough,toxic-agent-demo}`.
- **No network in any test.**

## README "Honest limitations" (shipped as a control)

Concede plainly (the concession prevents false assurance, which is itself a control):
structural unsolvability at the text layer; adversarial-ML evasion survives normalization;
tokenizer mismatch; semantic/factual poisoning invisible to lexicon+entropy; latent/dormant
memory poisoning not judgeable at write-time; insider in-place edits; text-only (no
multimodal).

## Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
steganography; query-time/runtime guardrails; semantic factuality verification. The
`contract` tool-less assertion is the write-time analogue of runtime least-privilege.

**Text-extraction boundary.** For pipelines that ingest arbitrary user uploads (an upload
inbox that ingests automatically), the library stays `text -> findings`: it does not parse
files (no `pypdf`/`python-docx`/archive deps in the core). The pipeline extracts text
first, then scans the extracted text + the enriched output with provenance tagged as the
high-untrust upload tier. Binary/multimodal carrier detection (OCR-embedded instructions in
images/PDFs, stego) is out of scope beyond the sanitizer's character-layer stripping and
the self-safety size/decompression guard — and must be conceded in the honest-limitations
section, because a high-untrust, unattended pipeline is exactly where assuming
uncovered-coverage is most dangerous.

## Threat-model anchors

OWASP LLM Top-10 2025: LLM01, LLM02, LLM04, LLM05, LLM06 (strongest coverage), LLM08
(boundary), LLM09, LLM10. Research anchors: PoisonedRAG (arXiv 2402.07867), guardrail
evasion (2504.11168), EchoLeak (CVE-2025-32711), RAGShield (2604.00387), CaMeL
(2503.18813), Dual-LLM (Willison), and the litellm supply-chain compromise (corroborates
the minimal-dependency thesis).

## v0.2+ stream sequencing (revised 2026-07-06)

v0.1.0 and v0.2.0 are tagged: the format-agnostic text core (modules 1–11,
`5397ba1` / released `df30c7b`) and the OKF adapter (stream 1, released
`542ac92`). The forward order was **re-sequenced on 2026-07-06** after a
ground-truth pass over the intended flagship consumer.

**Finding that drove the change.** The named flagship — `portfolio-optimiser`'s
"OKF-upload-inbox" — does not exist as a seam. Both optimiser siblings are frozen
at their release milestone, carry their *own* OKF layer (navigate + materialize
from trusted manifests), receive no *external* bundles, and take no dependency on
this guard; the operator's own registered future work for them does not include
it. Wiring an immature guard into two mature, spec-frozen repos would complicate
them for a consumer that is not asking for it. **Consumer integration is therefore
deferred until the guard is mature**; consumers are informed at the operator's
timing, not pushed.

Revised streams:

1. **OKF v0.2 hardening** — shipped (adapter + brief §8 tasks; T5b/B deferred to a
   consumer that owns corpus storage).
2. **Mature the guard here, keeping it Node-port-friendly** — the near-term work.
   The lexicon is already shared JSON (polyglot-ready); keep the `text -> findings`
   surface clean and free of Python-only cleverness in the OKF layer, so stream 3
   is a *translation*, not a redesign. The centrepiece is the **OKF inbox
   showcase** (below): the guard demonstrating its own flagship use case
   end-to-end, in-repo.
3. **Node/polyglot port** — the strategic enabler of painless integration (many
   OKF consumers are Node/JS; the lexicon seed was `.mjs`). One polyglot repo over
   the shared JSON lexicon; never split §13.3. Its API/lexicon contract should be
   **scan-informed** (stream 4), not guessed.
4. **Pre-adaptation scan (operator-timed)** — at the operator's chosen point, scan
   every repo and plugin that uses or plans Google OKF, then adapt the guard's
   surface in advance so a later integration is painless. The scan is the input to
   the "painless" guarantee: it grounds both the Node port's contract and any
   consumer-specific seams before they are locked.

### The OKF inbox showcase (next concrete build, TDD)

The flagship artifact we hand a consumer later — an in-repo end-to-end
demonstration of the mode-b receive/quarantine gate, mirroring
`tests/test_showcase.py` but for a *received external OKF bundle*. It composes the
public `okf` surface exactly as an "upload inbox" consumer would, so it doubles as
the README's OKF worked example. Every test is authored by us — the point is to
prove intent, not to coincidentally pass.

- **Composition** (`tests/test_okf_showcase.py`): an `_inbox(bundle)` helper
  calling `okf.import_bundle(bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC)`
  and mapping the aggregate disposition — `WARN` → ADMIT, `QUARANTINE_REVIEW` →
  HOLD, `FAIL_SECURE` → REJECT, any error → REJECT (fail-secure default).
- **One poisoned bundle** planting one attack per OKF surface at once, each with a
  label proving it was caught/rejected:
  - body injection (T1 scan) and frontmatter `description` injection (T1
    whole-concept scan);
  - a non-`https` `resource:` URL (T3 → FAIL_SECURE);
  - a path-traversal concept key `../x.md` (T4 → FAIL_SECURE) and a reserved-name
    shadow `index.md` / `log.md` (T4);
  - a dangerous frontmatter value (anchor / alias / explicit tag, T2 → FAIL_SECURE);
  - a dangerous-scheme cross-link `[x](javascript:…)` (T5a → `links.rejected`);
  - a dangling cross-link to an absent concept (§7.2 dormant signal →
    `links.dangling`);
  - a carrier/obfuscation-hidden body injection (zero-width / homoglyph / rot13 /
    whole-string base64) routed through `scan_concept`.
- **Assertions:** aggregate disposition is FAIL_SECURE; every planted OKF
  vulnerability appears in the per-concept rejects / findings / link graph; a
  **clean** bundle admits (WARN, no rejects, no dangling); `BundleResult.log()`
  renders one line per concept with rejects marked. **Detach-proof:** neutering
  `import_bundle` to always-admit makes the poison assertions fail.
- **Honest scope:** demonstrates the *structural + known-pattern* OKF surface only
  — semantic/factual poisoning stays out (README honest-limits), consistent with
  the core showcase.

#### Realistic upload formats — the two-stage inbox (extract → materialize → guard)

A human inbox does not receive tidy `{path: text}` dicts; it receives the files
people actually drop: `.txt`, `.md`, Word `.docx`, Excel `.xlsx`, PowerPoint
`.pptx`, `.pdf`, `.csv`, whole **folders**, and **`.zip`** archives. The showcase
therefore has two stages, honouring the locked text-extraction boundary (§
"Text-extraction boundary"): the guard core never grows a file parser.

1. **Extract & materialize (the inbox front-end).** Reads each dropped file, walks
   folders, and *safely* unpacks archives; extracts text via format libraries
   (`python-docx`, `openpyxl`, `python-pptx`; stdlib `zipfile`/`csv`); materializes
   the result into an OKF bundle `{concept_path: text}` with provenance
   (`origin=EXTERNAL`, source filename + type). **This stage owns the container and
   format threats.** These libraries are **showcase/dev-scoped only** — never core
   `dependencies` (which stays `[]`). Promotion to an optional `[extract]` extra is
   a documented future option if a consumer wants turnkey extraction, not v1.
2. **Guard (`import_bundle`).** Scans every extracted concept + the OKF structural
   gates → aggregate disposition. **This stage owns the text/structural threats.**

**One representative planted vector per format** (the point is where the payload
hides — the place a human does not look):

| Input | Hidden vector planted | Caught at |
|---|---|---|
| `.txt` | raw injection + carrier (zero-width / homoglyph / rot13 / base64) | guard scan |
| `.md` | OKF frontmatter attack (T2) + body injection (T1) | guard |
| `.docx` | injection in a **comment / hidden (vanish) run / core metadata property** | guard scan of extracted text |
| `.xlsx` | **formula injection** (`=cmd\|'…'`, `=HYPERLINK(…)`) + **hidden sheet / cell comment** | front-end + guard |
| `.pptx` | injection in **speaker notes / off-slide text box / image alt-text** | guard scan |
| `.pdf` | injection in extracted / **white-on-white** text | guard scan |
| `.csv` | formula injection (`=`, `+`, `-`, `@` lead) | front-end + guard |
| **folder** | the OKF bundle-directory shape directly; one member path trips the path gate (T4) | guard |
| **`.zip`** | **zip-slip** entry `../../x.md` (maps straight onto OKFPathError, T4) + **zip-bomb** (bounded by the size cap / a safe-extract limit, LLM10) + symlink entry | front-end + guard |

**Assertions:** each planted vector is caught at the correct stage; the front-end
refuses zip-slip / zip-bomb / oversize fail-fast; the guard rejects
injection/carrier/frontmatter/resource/link/path; a **clean** file of every format
admits (WARN). Detach-proof at **both** stages.

**Honest scope for uploads (must be conceded — README honest-limits).** What
survives text extraction is **out of scope**: VBA/macros (`.docm`/`.xlsm`/`.pptm`),
OLE / embedded objects, image-embedded instructions that need OCR, font/render
steganography, and encrypted/password-protected files. The guard scans *extracted
text*; binary-layer carriers need a separate scanner. A high-untrust, unattended
inbox is exactly where assuming uncovered-coverage is most dangerous (§4.7), so the
concession is itself a control.

**Deferred, unchanged:** T5b/B (the persisted cross-run link graph) waits for a
consumer that owns corpus storage; §7.2 dormant cross-run links remain a
documented residual until then.

**Splittable early win (optional):** text-only consumers can be wired at v0.1.0
today — for them the guard is already complete. Deferred with stream 2 by default;
pull forward only on explicit request.