llm-ingestion-pipeline-security/docs/LIMITATIONS.md

Honest limitations

Conceding these plainly is itself a control — it prevents the false assurance that a green scan means safe content. The README carries a summary of the highest-impact items; this is the full list, each with the mechanism.

• Structural unsolvability at the text layer. Pattern/lexicon detection is bypassable in isolation; character-injection and novel phrasings evade it. The contract (tool-less transform, capability isolation, fail-secure) carries the security — the lexicon is defense-in-depth, not a wall.
• A lone HIGH finding in trusted prose disposes to WARN, not quarantine. Under PRESET_TRUSTED_SOURCE, trust-scaling downgrades a single HIGH to WARN, and one HIGH is not "compound" (escalation needs ≥2 findings at MEDIUM+). So a HIGH injection reproduced verbatim under a trusted policy persists with a WARN. This is by design: if your "trusted" sources can carry attacker-influenced text, run them as untrusted (or add a quarantine floor).
• The quarantine floor fires at MEDIUM+, and is a no-op under the shipped upload preset. Through 0.3.0 quarantine_default floored any finding to QUARANTINE_REVIEW. That premise ("a finding is the exception") broke when the active-content detector made every ordinary markdown link a finding, so 0.3.1 raised the floor to MEDIUM+. Under PRESET_USER_UPLOAD (untrusted) a MEDIUM already escalates on trust alone, so the floor still changes no outcome there; it is live only for a caller-defined trusted policy that opts into quarantine_default. Documented so the preset is not over-read: a LOW finding on an upload now disposes WARN.
• Semantic / factual poisoning is invisible to lexicon + entropy: a false claim in clean prose carries no suspicious token. Highest impact for a wiki. The grounding module ships only a SourceGroundingCheck seam — the deterministic core does not judge semantics; a [judge] implementation must be plugged in.
• Adversarial-ML evasion can survive normalization; tokenizer mismatch between scanner and model leaves gaps. Latent / dormant memory poisoning is not judgeable at write time.
• Dormant / broken-link injection in a linked corpus (e.g. an OKF bundle): a link to a not-yet-existing target passes a per-concept write-time scan clean — the payload is planted later, when that target is written. link_graph surfaces the dangling edge as the signal, but catching the payload needs cross-write re-scan over time (the caller's disposition call).
• OKF reserved files (index.md / log.md). In a received bundle these are legitimate structure, so mode-b import_bundle scans their body and frontmatter (an injection in a directory listing is caught) rather than path-rejecting the conformant bundle. A front-end materialising individual uploads keeps the opposite rule (allow_reserved=False): a reserved basename is a listing-shadow and refused.
• A document that describes attacks is a false positive. Content documenting prompt-injection payloads (security notes, this project's own corpus) trips carrier-strip / fail-secure. At the text layer "about an attack" and "carrying an attack" are indistinguishable; such content needs a deliberate, explicitly escaped path, never a silent allow.
• Bilingual text trips the Cyrillic/Latin homoglyph rule. homoglyph:cyrillic-latin-mix (MEDIUM) flags a Latin letter adjacent to a Cyrillic look-alike, so genuine bilingual prose → MEDIUM → under untrusted → QUARANTINE_REVIEW — a real false positive for an inbox that expects multilingual content. A calibration fix is pending.
• Insider in-place edits by a trusted author are out of the untrusted-content threat model.
• Text-only. The core is text -> findings: it parses no files (no pypdf/python-docx/archive deps). Extract text first, then scan it with the high-untrust upload provenance. OCR-embedded instructions and multimodal stego are out of scope beyond the sanitizer's character-layer stripping.
• Uploaded files: only the extracted text is scanned. The dev-scoped OKF inbox showcase (tests/test_okf_inbox_uploads.py + tests/inbox_frontend.py; parsers in the [dev] extra, never core dependencies) reads .txt/.md/.csv/.docx/.pptx/.xlsx, folders and .zip, materializes an OKF bundle, then guards it. What survives extraction is out of scope: macros, OLE/embedded objects, OCR-needing images, font/render stego, encrypted files — the binary layer needs a separate scanner. The front-end owns the container threats it can see (zip-slip → path gate, zip-bomb → size cap, symlink refusal, CSV/XLSX formula-lead cells). .pdf is a deliberate concession: a top-level .pdf is refused as unsupported rather than half-scanned (a PDF parser is disproportionate for a dev showcase, and the OCR/stego it would smuggle is already out of scope). One known gap: the numeric -/+ CSV false positive (a typed XLSX numeric cell does not trip it).
• Lexicon findings are deduplicated by pattern id — count=1 and the first offset are reported, so a class matched across several channels collapses to one finding at its first location: a deliberate readability tradeoff.
• Active-content severity grades on URL shape, so a pure beacon is only LOW. Since 0.3.1 a URL that merely names a remote document (bare path, no query, no userinfo, no percent-escapes, no opaque segment) is LOW, and only a URL that can move bytes outward keeps HIGH/MEDIUM. The deliberate hole: ![x](https:// evil.test/pixel.png) on an attacker-controlled host still fetches when a renderer touches it, leaking reader IP, user-agent and timing. Grading the fetch itself would re-block every ordinary document, which is precisely the 0.3.0 regression this replaced — so beaconing is conceded, not covered.
• Short opaque URL segments slip through the same grading. Opacity is decided by entropy's primitives: base64 that decodes to text (≥20 chars), a hex id (≥32 chars), or Shannon entropy ≥4.4 at ≥24 chars. A shorter payload segment — https://evil.test/aGVsbG8gd29ybGQ — cannot be told from a name, because entropy is bounded by log2(length) at short lengths. Mitigation in depth, not in this detector: a literal credential in a URL is still caught by the LLM02 egress patterns in the same scan_output pass, whatever severity the carrier gets.
• Percent-escapes count as data-carrying — a %20 in a path is a false positive. An ordinary link with an encoded space grades as carrying and reaches QUARANTINE_REVIEW / FAIL_SECURE on an untrusted upload. Obfuscated encoding is a core exfil primitive and the ambiguous case is put on the review side deliberately; it is listed here because it is the same class of over-block that 0.3.1 fixed, in a rarer shape. Measured (2026-07-25, two independent consumer corpora): zero occurrences — 0 of 347 external URLs in a 527-document vendor-docs corpus, 0 of 81 in a real capture store. That bounds the shape as rare in linked content; it does not retire it. A consumer deriving filenames from titles through a slugger produces %20 systematically rather than incidentally, and that corpus is still unmeasured — so the open question is narrower than it was, not closed.
• A non-empty query is graded as data-carrying — the over-block that actually occurs in the field. The same two corpora found this rule firing on disjoint benign populations: 16 of 16 query-carrying external URLs in the vendor-docs corpus were publisher-authored campaign tracking (utm_* on the publisher's own domains), while 35 of 35 in the capture store were content identity (?v=, ?channel_id=, ?all=true) where the parameter is the resource. No parameter-level remedy covers both: an allowlist keyed on tracking-parameter names resolves the first population entirely and the second not at all, and stripping the query is lossless for the first while dereferencing nothing for the second. The cost is bounded — a query-carrying link is MEDIUM, so it disposes QUARANTINE_REVIEW under PRESET_USER_UPLOAD and WARN under PRESET_TRUSTED_SOURCE: held or warned, never hard-failed (pinned in tests/test_wiring.py, because both consumers inferred a hard block rather than running it). An image keeps HIGH, and that is the carrier where this would bite — neither corpus contained a single remote image, so the image row of this limitation remains unmeasured in the field.
• URL fragments are not graded. A fragment is never sent to the server, so it cannot carry data to the host a renderer auto-fetches, and …/overview#section is the most common shape in real documentation. The residual: a clicked link to an attacker-controlled page can have its location.hash read by that page's script, so a fragment payload on a link (not an image) is uncovered.
• Secret egress: base64-wrapped is caught, hex-wrapped is not. The output gate decodes base64 blobs and re-scans the plaintext, so a base64-wrapped secret surfaces as decoded:egress:*. entropy exposes decoded plaintext for base64 only, so hex (and other encodings, or nested wraps) is a deliberate boundary — decode the transport layer first if you need it scanned.

The six documented gaps (tracked by the coverage matrix)

These are asserted to still hold by tests/test_coverage_matrix.py — a closed gap fails the test, forcing this doc to be updated:

1. Hex-wrapped secret egress — entropy decodes base64 only (above).
2. Semantic / factual poisoning — invisible to token analysis (above).
3. A lone HIGH in trusted prose → WARN — the §4.7 trust-scaling design (above).
4. Lexicon dedup (count=1) — first offset only, by design (above).
5. Pure beaconing — a bare-path remote image on a hostile host is LOW (above).
6. Short opaque URL segment (<24 chars) — below what entropy can resolve (above).

Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal steganography; query-time / runtime guardrails; semantic factuality verification.