docs(readme): add concrete 'What it protects against' catalogue; split limitations
Answers the gap that the README said what it does NOT stop (a long limitations section) but never plainly listed what it DOES. Add a 'What it protects against' section high up: attack classes grouped by OWASP anchor (LLM01 injection + 83 lexicon classes + carriers, LLM02 egress, LLM05 EchoLeak, LLM06 agency, LLM10 fail-secure, OKF T1-T6, container front-end), each driven by a live coverage-matrix payload. Move the full honest-limitations list to docs/LIMITATIONS.md; README keeps a high-impact summary + link. Net: protection and limits read in balance, 261 -> 216 lines. Coverage 126/126 and 522 tests unchanged; every class listed is real.
This commit is contained in:
parent
d73cfc882f
commit
3dda1f68fa
3 changed files with 145 additions and 96 deletions
10
CHANGELOG.md
10
CHANGELOG.md
|
|
@ -34,9 +34,13 @@ This is the real-case validation gate ahead of a v1.0 freeze.
|
|||
(`import_bundle` mode-b), how to verify (coverage matrix), how to depend
|
||||
(stdlib-only core), and a checklist for *when/where* to wire it.
|
||||
- README rewritten to lead with the write-time trust-boundary framing, add a
|
||||
first-class **OKF / LLM-wiki support (shipped)** section for `import_bundle`, and
|
||||
correct the test badge (357 → 522). Every claim verified against the code; all
|
||||
honest-limitations items retained.
|
||||
first-class **OKF / LLM-wiki support (shipped)** section for `import_bundle`, a
|
||||
concrete **What it protects against** catalogue (attack classes grouped by OWASP
|
||||
anchor, driven by the coverage matrix), and correct the test badge (357 → 522).
|
||||
Every claim verified against the code.
|
||||
- `docs/LIMITATIONS.md` — the full honest-limitations list (15 items + the four
|
||||
documented gaps + out-of-scope) moved out of the README, which now carries a
|
||||
high-impact summary + link, so protection and limits read in balance.
|
||||
|
||||
## [0.2.0] — 2026-07-06
|
||||
|
||||
|
|
|
|||
139
README.md
139
README.md
|
|
@ -117,6 +117,34 @@ are the attack surface); **`resource` https-allowlist** (hard-rejects
|
|||
rejects bundle-escaping links); **provenance stamping** (`Origin` × `Channel` →
|
||||
tier + disposition, emitted to `log.md`).
|
||||
|
||||
## What it protects against
|
||||
|
||||
Concrete attack classes, grouped by OWASP LLM Top-10 (2025) anchor. Every row is
|
||||
driven by a **live payload** in the coverage matrix — run it to watch all 126 pass
|
||||
in your own environment:
|
||||
|
||||
```bash
|
||||
python -m llm_ingestion_guard.coverage # 126/126 classes; exit 0 = all as documented
|
||||
```
|
||||
|
||||
| Anchor | Attack classes it stops (representative) |
|
||||
|---|---|
|
||||
| **LLM01 · prompt injection** | 83 instruction-override lexicon classes (ignore / forget / disregard / suspend-constraints, role-play, jailbreak, …); hidden carriers — zero-width stego, BIDI override, Unicode-tag stego, HTML-comment, `data:` URI — stripped on input **and** re-checked at the persist gate; high-entropy base64/hex blobs; an injection hidden inside a base64 blob (decoded, then re-scanned) |
|
||||
| **LLM02 · sensitive-info disclosure** | Secret egress in output — AWS keys, tokens, private keys, the credential set; a base64-*wrapped* secret (decoded → `decoded:egress:*`) |
|
||||
| **LLM05 · improper output handling** | Zero-click exfil carriers (EchoLeak, CVE-2025-32711) — markdown-image auto-fetch, inline / reference / autolink links, raw active HTML, standalone `data:` URIs; non-`https` `resource` URLs |
|
||||
| **LLM06 · excessive agency** | A tool surface on the quarantined transform; credential use beyond the per-stage allowlist; capability isolation (`scoped_env`) |
|
||||
| **LLM10 · fail-secure** | Forced-fallback attack (scan hit + transform failure → halt); compound weak-signal escalation; an un-scannable artifact fails **closed** — never a silent persist |
|
||||
| **OKF structure (T1–T6)** | Body + frontmatter-value injection; YAML anchor / merge / nested / block DoS (parse-safety); `resource` allowlist; path traversal / reserved-name shadow; cross-link graph; provenance stamping |
|
||||
| **Container / upload layer** | CSV/XLSX formula-injection (lead `= + - @`); zip-slip path escape; zip-bomb size cap; symlink refusal *(upload front-end — see the showcase)* |
|
||||
|
||||
The manifest ([`coverage.py`](src/llm_ingestion_guard/coverage.py)) is the single
|
||||
source of truth for
|
||||
[`tests/test_coverage_matrix.py`](tests/test_coverage_matrix.py), which asserts
|
||||
total recall over every class, that every documented gap still holds (a closed gap
|
||||
fails the test), and that every lexicon pattern has a case — so the matrix cannot
|
||||
fall behind the code. The four classes it deliberately does **not** stop are called
|
||||
out in [Honest limitations](#honest-limitations).
|
||||
|
||||
## The reusable contract (adopt-this checklist)
|
||||
|
||||
The actual product is this checklist, encoded as code you wire in order:
|
||||
|
|
@ -143,101 +171,28 @@ The actual product is this checklist, encoded as code you wire in order:
|
|||
Steps 1-2 are `prepare_input`; steps 6-7 are `screen_output`; steps 3-5 are
|
||||
yours; the contract asserters harden step 3-4.
|
||||
|
||||
## Verify what it stops (coverage matrix)
|
||||
|
||||
Before wiring the guard into your pipeline, see — in one place — every
|
||||
vulnerability class it stops, and the ones it deliberately does not:
|
||||
|
||||
```bash
|
||||
python -m llm_ingestion_guard.coverage # prints the matrix; exit 0 = all as documented
|
||||
```
|
||||
|
||||
Each row drives the real guard with a live payload and prints
|
||||
`class -> OWASP anchor -> expected -> observed -> verdict`. As of `v0.2`: **126 of
|
||||
126 defended classes demonstrated (recall 100%)** and **4 of 4 documented gaps
|
||||
still hold**. The manifest ([`coverage.py`](src/llm_ingestion_guard/coverage.py))
|
||||
is the single source of truth for
|
||||
[`tests/test_coverage_matrix.py`](tests/test_coverage_matrix.py), which asserts
|
||||
**total recall** across every defended class, that **every documented gap still
|
||||
holds** (a closed gap fails the test, forcing a doc update), and — the honesty
|
||||
guard — that **every lexicon pattern has a case** so the matrix cannot fall behind
|
||||
the lexicon. The full LLM02 secret-egress set and the container-layer front-end
|
||||
(CSV formula-injection, zip-slip/bomb, symlink) are asserted there too.
|
||||
|
||||
## Honest limitations (shipped as a control)
|
||||
## Honest limitations
|
||||
|
||||
Conceding these plainly is itself a control — it prevents the false assurance that
|
||||
a green scan means safe content:
|
||||
a green scan means safe content. The highest-impact items:
|
||||
|
||||
- **Structural unsolvability at the text layer.** Pattern/lexicon detection is
|
||||
bypassable in isolation; character-injection and novel phrasings evade it. The
|
||||
*contract* (tool-less transform, capability isolation, fail-secure) carries the
|
||||
security — the lexicon is defense-in-depth, not a wall.
|
||||
- **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.**
|
||||
Under `PRESET_TRUSTED_SOURCE`, trust-scaling downgrades a single HIGH to WARN,
|
||||
and one HIGH is not "compound" (escalation needs ≥2 findings at MEDIUM+). So a
|
||||
HIGH injection reproduced verbatim under a *trusted* policy persists with a WARN.
|
||||
This is by design: if your "trusted" sources can carry attacker-influenced text,
|
||||
run them as untrusted (or add a quarantine floor).
|
||||
- **The upload preset's quarantine floor is currently vacuous.**
|
||||
`PRESET_USER_UPLOAD` sets `quarantine_default`, but detectors emit
|
||||
CRITICAL/HIGH/MEDIUM only, and under untrusted trust a MEDIUM already escalates to
|
||||
QUARANTINE_REVIEW — so the floor changes no outcome today. It is headroom for a
|
||||
future LOW/INFO finding, documented so the preset is not over-read.
|
||||
- **Semantic / factual poisoning is invisible** to lexicon + entropy: a false claim
|
||||
in clean prose carries no suspicious token. **Highest impact for a wiki.** The
|
||||
`grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic
|
||||
core does not judge semantics; a `[judge]` implementation must be plugged in.
|
||||
- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch**
|
||||
between scanner and model leaves gaps. **Latent / dormant memory poisoning** is
|
||||
not judgeable at write time.
|
||||
- **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a
|
||||
link to a not-yet-existing target passes a per-concept write-time scan clean — the
|
||||
payload is planted later, when that target is written. `link_graph` surfaces the
|
||||
*dangling* edge as the signal, but catching the payload needs cross-write re-scan
|
||||
over time (the caller's disposition call).
|
||||
- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are
|
||||
legitimate structure, so mode-b `import_bundle` scans their body and frontmatter
|
||||
(an injection in a directory listing is caught) rather than path-rejecting the
|
||||
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
||||
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
||||
- **A document that *describes* attacks is a false positive.** Content documenting
|
||||
prompt-injection payloads (security notes, this project's own corpus) trips
|
||||
carrier-strip / fail-secure. At the text layer "*about* an attack" and "*carrying*
|
||||
an attack" are indistinguishable; such content needs a deliberate, explicitly
|
||||
escaped path, never a silent allow.
|
||||
- **Bilingual text trips the Cyrillic/Latin homoglyph rule.**
|
||||
`homoglyph:cyrillic-latin-mix` (MEDIUM) flags a Latin letter adjacent to a
|
||||
Cyrillic look-alike, so genuine bilingual prose → MEDIUM → under untrusted →
|
||||
QUARANTINE_REVIEW — a real false positive for an inbox that expects multilingual
|
||||
content. A calibration fix is pending.
|
||||
- **Insider in-place edits** by a trusted author are out of the untrusted-content
|
||||
threat model.
|
||||
- **Text-only.** The core is `text -> findings`: it parses no files (no
|
||||
`pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the
|
||||
high-untrust upload provenance. OCR-embedded instructions and multimodal stego are
|
||||
out of scope beyond the sanitizer's character-layer stripping.
|
||||
- **Uploaded files: only the *extracted text* is scanned.** The dev-scoped OKF
|
||||
inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`;
|
||||
parsers in the `[dev]` extra, never core `dependencies`) reads
|
||||
`.txt`/`.md`/`.csv`/`.docx`/`.pptx`/`.xlsx`, folders and `.zip`, materializes an
|
||||
OKF bundle, then guards it. What survives extraction is **out of scope**:
|
||||
macros, OLE/embedded objects, OCR-needing images, font/render stego, encrypted
|
||||
files — the binary layer needs a separate scanner. The front-end owns the
|
||||
container threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink
|
||||
refusal, CSV/XLSX formula-lead cells). **`.pdf` is a deliberate concession:** a
|
||||
top-level `.pdf` is *refused as unsupported* rather than half-scanned (a PDF parser
|
||||
is disproportionate for a dev showcase, and the OCR/stego it would smuggle is
|
||||
already out of scope). One known gap: the numeric `-`/`+` CSV false positive (a
|
||||
typed XLSX numeric cell does not trip it).
|
||||
- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first
|
||||
offset are reported, so a class matched across several channels collapses to one
|
||||
finding at its first location: a deliberate readability tradeoff.
|
||||
- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate
|
||||
decodes base64 blobs and re-scans the plaintext, so a base64-*wrapped* secret
|
||||
surfaces as `decoded:egress:*`. `entropy` exposes decoded plaintext for base64
|
||||
only, so hex (and other encodings, or nested wraps) is a deliberate boundary —
|
||||
decode the transport layer first if you need it scanned.
|
||||
- **The contract carries the security, not the lexicon.** Pattern detection is
|
||||
bypassable in isolation (character-injection, novel phrasings); the tool-less
|
||||
transform + capability isolation + fail-secure are the wall.
|
||||
- **Semantic / factual poisoning is invisible.** A false claim in clean prose
|
||||
carries no suspicious token — the highest-impact gap for a wiki. Needs a `[judge]`
|
||||
plugged into the `grounding` seam.
|
||||
- **Text-only, extracted-text-only.** The core parses no files; OCR-embedded
|
||||
instructions, macros, and multimodal stego are out of scope. `.pdf` is refused as
|
||||
unsupported, not half-scanned.
|
||||
- **A lone HIGH in *trusted* prose disposes to WARN**, and **insider in-place
|
||||
edits** are outside the untrusted-content threat model — run genuinely untrusted
|
||||
sources as untrusted.
|
||||
- **Four documented gaps** the coverage matrix keeps honest: hex-wrapped secret
|
||||
egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`).
|
||||
|
||||
**Full list — 15 items, each with the mechanism, plus the out-of-scope boundary:**
|
||||
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md).
|
||||
|
||||
## Out-of-scope (documented boundary)
|
||||
|
||||
|
|
|
|||
90
docs/LIMITATIONS.md
Normal file
90
docs/LIMITATIONS.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
# Honest limitations
|
||||
|
||||
Conceding these plainly is itself a control — it prevents the false assurance that
|
||||
a green scan means safe content. The README carries a summary of the highest-impact
|
||||
items; this is the full list, each with the mechanism.
|
||||
|
||||
- **Structural unsolvability at the text layer.** Pattern/lexicon detection is
|
||||
bypassable in isolation; character-injection and novel phrasings evade it. The
|
||||
*contract* (tool-less transform, capability isolation, fail-secure) carries the
|
||||
security — the lexicon is defense-in-depth, not a wall.
|
||||
- **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.**
|
||||
Under `PRESET_TRUSTED_SOURCE`, trust-scaling downgrades a single HIGH to WARN,
|
||||
and one HIGH is not "compound" (escalation needs ≥2 findings at MEDIUM+). So a
|
||||
HIGH injection reproduced verbatim under a *trusted* policy persists with a WARN.
|
||||
This is by design: if your "trusted" sources can carry attacker-influenced text,
|
||||
run them as untrusted (or add a quarantine floor).
|
||||
- **The upload preset's quarantine floor is currently vacuous.**
|
||||
`PRESET_USER_UPLOAD` sets `quarantine_default`, but detectors emit
|
||||
CRITICAL/HIGH/MEDIUM only, and under untrusted trust a MEDIUM already escalates to
|
||||
QUARANTINE_REVIEW — so the floor changes no outcome today. It is headroom for a
|
||||
future LOW/INFO finding, documented so the preset is not over-read.
|
||||
- **Semantic / factual poisoning is invisible** to lexicon + entropy: a false claim
|
||||
in clean prose carries no suspicious token. **Highest impact for a wiki.** The
|
||||
`grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic
|
||||
core does not judge semantics; a `[judge]` implementation must be plugged in.
|
||||
- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch**
|
||||
between scanner and model leaves gaps. **Latent / dormant memory poisoning** is
|
||||
not judgeable at write time.
|
||||
- **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a
|
||||
link to a not-yet-existing target passes a per-concept write-time scan clean — the
|
||||
payload is planted later, when that target is written. `link_graph` surfaces the
|
||||
*dangling* edge as the signal, but catching the payload needs cross-write re-scan
|
||||
over time (the caller's disposition call).
|
||||
- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are
|
||||
legitimate structure, so mode-b `import_bundle` scans their body and frontmatter
|
||||
(an injection in a directory listing is caught) rather than path-rejecting the
|
||||
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
||||
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
||||
- **A document that *describes* attacks is a false positive.** Content documenting
|
||||
prompt-injection payloads (security notes, this project's own corpus) trips
|
||||
carrier-strip / fail-secure. At the text layer "*about* an attack" and "*carrying*
|
||||
an attack" are indistinguishable; such content needs a deliberate, explicitly
|
||||
escaped path, never a silent allow.
|
||||
- **Bilingual text trips the Cyrillic/Latin homoglyph rule.**
|
||||
`homoglyph:cyrillic-latin-mix` (MEDIUM) flags a Latin letter adjacent to a
|
||||
Cyrillic look-alike, so genuine bilingual prose → MEDIUM → under untrusted →
|
||||
QUARANTINE_REVIEW — a real false positive for an inbox that expects multilingual
|
||||
content. A calibration fix is pending.
|
||||
- **Insider in-place edits** by a trusted author are out of the untrusted-content
|
||||
threat model.
|
||||
- **Text-only.** The core is `text -> findings`: it parses no files (no
|
||||
`pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the
|
||||
high-untrust upload provenance. OCR-embedded instructions and multimodal stego are
|
||||
out of scope beyond the sanitizer's character-layer stripping.
|
||||
- **Uploaded files: only the *extracted text* is scanned.** The dev-scoped OKF
|
||||
inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`;
|
||||
parsers in the `[dev]` extra, never core `dependencies`) reads
|
||||
`.txt`/`.md`/`.csv`/`.docx`/`.pptx`/`.xlsx`, folders and `.zip`, materializes an
|
||||
OKF bundle, then guards it. What survives extraction is **out of scope**:
|
||||
macros, OLE/embedded objects, OCR-needing images, font/render stego, encrypted
|
||||
files — the binary layer needs a separate scanner. The front-end owns the
|
||||
container threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink
|
||||
refusal, CSV/XLSX formula-lead cells). **`.pdf` is a deliberate concession:** a
|
||||
top-level `.pdf` is *refused as unsupported* rather than half-scanned (a PDF parser
|
||||
is disproportionate for a dev showcase, and the OCR/stego it would smuggle is
|
||||
already out of scope). One known gap: the numeric `-`/`+` CSV false positive (a
|
||||
typed XLSX numeric cell does not trip it).
|
||||
- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first
|
||||
offset are reported, so a class matched across several channels collapses to one
|
||||
finding at its first location: a deliberate readability tradeoff.
|
||||
- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate
|
||||
decodes base64 blobs and re-scans the plaintext, so a base64-*wrapped* secret
|
||||
surfaces as `decoded:egress:*`. `entropy` exposes decoded plaintext for base64
|
||||
only, so hex (and other encodings, or nested wraps) is a deliberate boundary —
|
||||
decode the transport layer first if you need it scanned.
|
||||
|
||||
## The four documented gaps (tracked by the coverage matrix)
|
||||
|
||||
These are asserted to *still hold* by `tests/test_coverage_matrix.py` — a closed gap
|
||||
fails the test, forcing this doc to be updated:
|
||||
|
||||
1. **Hex-wrapped secret egress** — `entropy` decodes base64 only (above).
|
||||
2. **Semantic / factual poisoning** — invisible to token analysis (above).
|
||||
3. **A lone HIGH in trusted prose → WARN** — the §4.7 trust-scaling design (above).
|
||||
4. **Lexicon dedup (`count=1`)** — first offset only, by design (above).
|
||||
|
||||
## Out-of-scope (documented boundary)
|
||||
|
||||
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
||||
steganography; query-time / runtime guardrails; semantic factuality verification.
|
||||
Loading…
Add table
Add a link
Reference in a new issue