llm-ingestion-pipeline-security/docs/LIMITATIONS.md

# Honest limitations

Conceding these plainly is itself a control — it prevents the false assurance that
a green scan means safe content. The README carries a summary of the highest-impact
items; this is the full list, each with the mechanism.

- **Structural unsolvability at the text layer.** Pattern/lexicon detection is
  bypassable in isolation; character-injection and novel phrasings evade it. The
  *contract* (tool-less transform, capability isolation, fail-secure) carries the
  security — the lexicon is defense-in-depth, not a wall.
- **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.**
  Under `PRESET_TRUSTED_SOURCE`, trust-scaling downgrades a single HIGH to WARN,
  and one HIGH is not "compound" (escalation needs ≥2 findings at MEDIUM+). So a
  HIGH injection reproduced verbatim under a *trusted* policy persists with a WARN.
  This is by design: if your "trusted" sources can carry attacker-influenced text,
  run them as untrusted (or add a quarantine floor).
- **The quarantine floor fires at MEDIUM+, and is a no-op under the shipped upload
  preset.** Through 0.3.0 `quarantine_default` floored *any* finding to
  QUARANTINE_REVIEW. That premise ("a finding is the exception") broke when the
  active-content detector made every ordinary markdown link a finding, so 0.3.1
  raised the floor to MEDIUM+. Under `PRESET_USER_UPLOAD` (untrusted) a MEDIUM
  already escalates on trust alone, so the floor still changes no outcome there; it
  is live only for a caller-defined *trusted* policy that opts into
  `quarantine_default`. Documented so the preset is not over-read: a LOW finding on
  an upload now disposes WARN.
- **Semantic / factual poisoning is invisible** to lexicon + entropy: a false claim
  in clean prose carries no suspicious token. **Highest impact for a wiki.** The
  `grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic
  core does not judge semantics; a `[judge]` implementation must be plugged in.
- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch**
  between scanner and model leaves gaps. **Latent / dormant memory poisoning** is
  not judgeable at write time.
- **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a
  link to a not-yet-existing target passes a per-concept write-time scan clean — the
  payload is planted later, when that target is written. `link_graph` surfaces the
  *dangling* edge as the signal, but catching the payload needs cross-write re-scan
  over time (the caller's disposition call).
- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are
  legitimate structure, so mode-b `import_bundle` scans their body and frontmatter
  (an injection in a directory listing is caught) rather than path-rejecting the
  conformant bundle. A front-end materialising individual uploads keeps the opposite
  rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
- **A document that *describes* attacks is a false positive.** Content documenting
  prompt-injection payloads (security notes, this project's own corpus) trips
  carrier-strip / fail-secure. At the text layer "*about* an attack" and "*carrying*
  an attack" are indistinguishable; such content needs a deliberate, explicitly
  escaped path, never a silent allow.
- **Bilingual text trips the Cyrillic/Latin homoglyph rule.**
  `homoglyph:cyrillic-latin-mix` (MEDIUM) flags a Latin letter adjacent to a
  Cyrillic look-alike, so genuine bilingual prose → MEDIUM → under untrusted →
  QUARANTINE_REVIEW — a real false positive for an inbox that expects multilingual
  content. A calibration fix is pending.
- **Insider in-place edits** by a trusted author are out of the untrusted-content
  threat model.
- **Text-only.** The core is `text -> findings`: it parses no files (no
  `pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the
  high-untrust upload provenance. OCR-embedded instructions and multimodal stego are
  out of scope beyond the sanitizer's character-layer stripping.
- **Uploaded files: only the *extracted text* is scanned.** The dev-scoped OKF
  inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`;
  parsers in the `[dev]` extra, never core `dependencies`) reads
  `.txt`/`.md`/`.csv`/`.docx`/`.pptx`/`.xlsx`, folders and `.zip`, materializes an
  OKF bundle, then guards it. What survives extraction is **out of scope**:
  macros, OLE/embedded objects, OCR-needing images, font/render stego, encrypted
  files — the binary layer needs a separate scanner. The front-end owns the
  container threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink
  refusal, CSV/XLSX formula-lead cells). **`.pdf` is a deliberate concession:** a
  top-level `.pdf` is *refused as unsupported* rather than half-scanned (a PDF parser
  is disproportionate for a dev showcase, and the OCR/stego it would smuggle is
  already out of scope). One known gap: the numeric `-`/`+` CSV false positive (a
  typed XLSX numeric cell does not trip it).
- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first
  offset are reported, so a class matched across several channels collapses to one
  finding at its first location: a deliberate readability tradeoff.
- **Active-content severity grades on URL shape, so a pure *beacon* is only LOW.**
  Since 0.3.1 a URL that merely names a remote document (bare path, no query, no
  userinfo, no percent-escapes, no opaque segment) is LOW, and only a URL that can
  move bytes outward keeps HIGH/MEDIUM. The deliberate hole: `![x](https://
  evil.test/pixel.png)` on an attacker-controlled host still *fetches* when a
  renderer touches it, leaking reader IP, user-agent and timing. Grading the fetch
  itself would re-block every ordinary document, which is precisely the 0.3.0
  regression this replaced — so beaconing is conceded, not covered.
- **Short opaque URL segments slip through the same grading.** Opacity is decided by
  `entropy`'s primitives: base64 that decodes to text (≥20 chars), a hex id
  (≥32 chars), or Shannon entropy ≥4.4 at ≥24 chars. A shorter payload segment —
  `https://evil.test/aGVsbG8gd29ybGQ` — cannot be told from a name, because entropy
  is bounded by `log2(length)` at short lengths. Mitigation in depth, not in this
  detector: a literal credential in a URL is still caught by the LLM02 egress
  patterns in the same `scan_output` pass, whatever severity the carrier gets.
- **Percent-escapes count as data-carrying — a `%20` in a path is a false positive.**
  An ordinary link with an encoded space grades as carrying and reaches
  QUARANTINE_REVIEW / FAIL_SECURE on an untrusted upload. Obfuscated encoding is a
  core exfil primitive and the ambiguous case is put on the review side deliberately;
  it is listed here because it is the same *class* of over-block that 0.3.1 fixed,
  in a rarer shape. **Measured (2026-07-25, two independent consumer corpora):
  zero occurrences** — 0 of 347 external URLs in a 527-document vendor-docs corpus,
  0 of 81 in a real capture store. That *bounds* the shape as rare in linked content;
  it does not retire it. A consumer deriving filenames from titles through a slugger
  produces `%20` systematically rather than incidentally, and that corpus is still
  unmeasured — so the open question is narrower than it was, not closed.
- **A non-empty query is graded as data-carrying — the over-block that actually
  occurs in the field.** The same two corpora found this rule firing on *disjoint*
  benign populations: 16 of 16 query-carrying external URLs in the vendor-docs corpus
  were publisher-authored campaign tracking (`utm_*` on the publisher's own domains),
  while 35 of 35 in the capture store were content identity (`?v=`, `?channel_id=`,
  `?all=true`) where the parameter *is* the resource. **No parameter-level remedy
  covers both:** an allowlist keyed on tracking-parameter names resolves the first
  population entirely and the second not at all, and stripping the query is lossless
  for the first while dereferencing nothing for the second. The cost is bounded — a
  query-carrying *link* is MEDIUM, so it disposes QUARANTINE_REVIEW under
  `PRESET_USER_UPLOAD` and WARN under `PRESET_TRUSTED_SOURCE`: held or warned, never
  hard-failed (pinned in `tests/test_wiring.py`, because both consumers inferred a
  hard block rather than running it). An *image* keeps HIGH, and that is the carrier
  where this would bite — neither corpus contained a single remote image, so the
  image row of this limitation remains unmeasured in the field.
- **URL fragments are not graded.** A fragment is never sent to the server, so it
  cannot carry data to the host a renderer auto-fetches, and `…/overview#section` is
  the most common shape in real documentation. The residual: a *clicked* link to an
  attacker-controlled page can have its `location.hash` read by that page's script,
  so a fragment payload on a link (not an image) is uncovered.
- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate
  decodes base64 blobs and re-scans the plaintext, so a base64-*wrapped* secret
  surfaces as `decoded:egress:*`. `entropy` exposes decoded plaintext for base64
  only, so hex (and other encodings, or nested wraps) is a deliberate boundary —
  decode the transport layer first if you need it scanned.

## The six documented gaps (tracked by the coverage matrix)

These are asserted to *still hold* by `tests/test_coverage_matrix.py` — a closed gap
fails the test, forcing this doc to be updated:

1. **Hex-wrapped secret egress** — `entropy` decodes base64 only (above).
2. **Semantic / factual poisoning** — invisible to token analysis (above).
3. **A lone HIGH in trusted prose → WARN** — the §4.7 trust-scaling design (above).
4. **Lexicon dedup (`count=1`)** — first offset only, by design (above).
5. **Pure beaconing** — a bare-path remote image on a hostile host is LOW (above).
6. **Short opaque URL segment (<24 chars)** — below what entropy can resolve (above).

## Out-of-scope (documented boundary)

Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
steganography; query-time / runtime guardrails; semantic factuality verification.