Kjell Tore Guttormsen
2d5a9cd0c7
A consumer measured our frontmatter gate against 0.2.0 and asked us to confirm
against a newer guard, correctly noting that a divergence would be a version
difference rather than a contradiction. Measured at 0.3.1: no divergence. Both
their results reproduce exactly.
Two things are sharper than what we documented yesterday:
ALL THREE ROUTES TO A MAPPING FAIL, each on a different rule -- flow {k: v} on the
disallowed value-start indicator, block on the nested-mapping check, dotted keys on
the key pattern. So the mapping CLASS has no expressible form; it is not a choice
between two shapes where one is better. That matters because OKF v0.2's `generated`
IS a mapping (`by` is required when it is present), so it cannot be expressed at
all. Pinned by a test asserting the three failures stay distinct.
T2 RUNS ON DOOR C ONLY. parse_frontmatter is referenced nowhere in the door A/B
persist path, so the same frontmatter that FAIL_SECUREs through import_bundle
passes screen_output unremarked. The grammar bounds what a consumer can RECEIVE,
never what a producer can EMIT -- which is the question a consumer's emitter design
was blocked on.
Also corrects one of our own rows: we reported "sources block list" as a single
FAIL_SECURE case. The parser distinguishes three shapes -- flat scalars parse
correctly, one key per item misparses silently to a string, two keys per item hard-
rejects. A one-element `verified:` block list passes today.
631 -> 642 passed. README item count synced.
245 lines
14 KiB
Markdown
245 lines
14 KiB
Markdown
# llm-ingestion-guard
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
**Write-time ingestion is the trust boundary that query-time guardrails
|
||
structurally cannot see.** When untrusted content passes through an LLM
|
||
enrichment/summarization/extraction step into a *persisted* artifact — a RAG
|
||
corpus, a knowledge base, an LLM wiki — the poisoned result is read *later* by a
|
||
downstream agent as **trusted context**. That agent's guardrail never sees where
|
||
the content came from. The only place the provenance still exists is the write.
|
||
|
||
This library packages that write-time contract — sanitize → fence → tool-less
|
||
quarantined transform → per-stage capability isolation → scan-before-commit →
|
||
fail-secure — as composable, stdlib-first, framework-agnostic code. It is the
|
||
write-time **sibling** of query-time tools (LLM Guard, NeMo Guardrails, Rebuff,
|
||
Vigil), not a competitor: those harden material as it enters the model; this
|
||
hardens it as it is committed for a *later* reader. Where existing OSS tooling is
|
||
mostly single-stage *detectors* — a risk verdict, with quarantine, capability
|
||
isolation, scan-before-persist, and fail-secure left to the integrator — this
|
||
packages the full contract as code. (Neighbours surveyed in
|
||
[`docs/BRIEF.md`](docs/BRIEF.md) §11.)
|
||
|
||
**Why an LLM wiki (e.g. Google OKF) needs this specifically.** OKF and
|
||
second-brain formats have no schema registry, no central authority, and no
|
||
signing — a bundle's claimed origin is not verifiable at the format level. So
|
||
*your ingestion pipeline is the trust boundary*: provenance must be stamped by you
|
||
at write time, never assumed from the format. Any pipeline ingesting external data
|
||
into an agent-read store has this shape; an OKF wiki is its canonical form — which
|
||
is why the guard ships a first-class OKF adapter (below).
|
||
|
||
**Status:** `v0.3`, alpha. The stdlib-only core — its detector, contract, and
|
||
OKF-adapter modules plus the top-level wiring — is built and tested, exercised by
|
||
an end-to-end showcase and adversarial + false-positive corpora. The public API
|
||
may still change. There are real limitations, stated plainly below; read them.
|
||
|
||
## Install
|
||
|
||
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
||
|
||
```bash
|
||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.1"
|
||
```
|
||
|
||
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
||
other credential. The core is stdlib-only with zero dependencies, so nothing else
|
||
resolves. Optional ML/judge detectors live behind extras (`[ml]`, `[judge]`) and
|
||
are not required — the core is deterministic and dependency-free.
|
||
|
||
**Tags are the stable contract.** Every release is version-synced before tagging,
|
||
and published tags are never moved.
|
||
|
||
Two consequences worth knowing before you depend on this:
|
||
|
||
- A git URL is a PEP 508 *direct reference*: it pins one exact tag, not a range
|
||
like `>=0.2,<0.3`. Real range pinning — and therefore automatic pickup of patch
|
||
releases — arrives with a Forgejo PyPI registry, which becomes the durable
|
||
channel at the first patch release or the second downstream consumer, whichever
|
||
comes first. The distribution name (`llm-ingestion-guard`) and the version
|
||
scheme are unchanged by that move, so pins written today keep their meaning.
|
||
- **Vendoring the source into a consumer is not supported.** It severs the patch
|
||
channel that a shared security dependency exists to provide: a copied guard
|
||
keeps running the vulnerabilities the original has already fixed.
|
||
|
||
## Quickstart — the two bookends
|
||
|
||
The library never makes the model call itself. It gives you the two library-side
|
||
halves around your own **tool-less** transform:
|
||
|
||
```python
|
||
from llm_ingestion_guard import (
|
||
prepare_input, screen_output, Disposition, PRESET_USER_UPLOAD,
|
||
)
|
||
|
||
prepared = prepare_input(untrusted_content) # sanitize + fence
|
||
enriched = your_model(prepared.fenced) # tool-less — YOUR call
|
||
decision = screen_output(enriched, PRESET_USER_UPLOAD) # scan + dispose
|
||
|
||
if decision.disposition is Disposition.FAIL_SECURE:
|
||
alert(gate_code=decision.reasons) # minimal payload, no content
|
||
raise SystemExit # halt — never persist
|
||
```
|
||
|
||
`screen_output` fails **closed**: if the scanner itself errors on crafted input,
|
||
the disposition is `FAIL_SECURE`, never a silent persist. Pass
|
||
`transform_failed=True` when your model call raised or fell back — a scan hit
|
||
together with a transform failure is treated as a probable forced-fallback attack
|
||
and halts regardless of trust tier.
|
||
|
||
Every primitive is also exported for pipelines that compose the checklist
|
||
themselves — `sanitize`, `scan_lexicon`, `scan_entropy`, `scan_output`,
|
||
`scan_active_content`, `neutralize`, the `decide` / `guard` disposition
|
||
machinery, and the contract asserters `assert_tool_less` /
|
||
`assert_credential_allowlist` / `scoped_env`. See
|
||
[the end-to-end showcase](tests/test_showcase.py) for a full worked pipeline.
|
||
|
||
## OKF / LLM-wiki support (shipped)
|
||
|
||
For a bundle-shaped store, the `okf` submodule sits **on top of** the
|
||
format-agnostic core: it knows OKF structure (frontmatter, paths, links,
|
||
`resource`, bundles) and feeds scannable regions into the same `sanitize` /
|
||
`scan_output` / disposition machinery — no YAML/format awareness leaks into the
|
||
core. Two ingestion modes:
|
||
|
||
- **(a) Own enrichment output** — your agent writes concepts. Run the two bookends
|
||
above per concept before commit.
|
||
- **(b) Received external bundle** — you merge a whole third-party OKF bundle.
|
||
`import_bundle` iterates concept-by-concept and runs the full per-concept gate:
|
||
|
||
```python
|
||
from llm_ingestion_guard.okf import import_bundle, Origin, Channel
|
||
|
||
# bundle: {concept_path -> raw document text}, e.g. {"tables/users.md": "---\n..."}
|
||
result = import_bundle(bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC)
|
||
|
||
for c in result.concepts:
|
||
if c.error: # hard reject: bad path, unsafe frontmatter, non-https resource
|
||
skip(c.path) # disposition is FAIL_SECURE; do not merge this concept
|
||
# result.disposition = most-severe across concepts
|
||
# result.links = the cross-link graph (dangling/rejected/resolved edges)
|
||
# result.log() = the log.md body (one provenance-stamped line per concept)
|
||
```
|
||
|
||
Per-concept gates: **path / reserved-name** (rejects `..` traversal and reserved
|
||
`index.md`/`log.md` shadowing); **frontmatter parse-safety** — a strict,
|
||
reject-by-default loader that refuses anchors, aliases, and explicit tags *by
|
||
construction*, so a billion-laughs alias expansion or a `!!python/object` coercion
|
||
cannot occur (it is deliberately **not** a general YAML engine, whose own features
|
||
are the attack surface); **`resource` https-allowlist** (hard-rejects
|
||
`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang);
|
||
**whole-concept scan** (frontmatter *values* + body through `scan_output`);
|
||
**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and
|
||
rejects bundle-escaping links); **provenance stamping** (`Origin` × `Channel` →
|
||
tier + disposition, emitted to `log.md`).
|
||
|
||
## What it protects against
|
||
|
||
Concrete attack classes, grouped by OWASP LLM Top-10 (2025) anchor. Every row is
|
||
driven by a **live payload** in the coverage matrix — run it to watch all 126 pass
|
||
in your own environment:
|
||
|
||
```bash
|
||
python -m llm_ingestion_guard.coverage # 126/126 classes; exit 0 = all as documented
|
||
```
|
||
|
||
| Anchor | Attack classes it stops (representative) |
|
||
|---|---|
|
||
| **LLM01 · prompt injection** | 83 instruction-override lexicon classes (ignore / forget / disregard / suspend-constraints, role-play, jailbreak, …); hidden carriers — zero-width stego, BIDI override, Unicode-tag stego, HTML-comment, `data:` URI — stripped on input **and** re-checked at the persist gate; high-entropy base64/hex blobs; an injection hidden inside a base64 blob (decoded, then re-scanned) |
|
||
| **LLM02 · sensitive-info disclosure** | Secret egress in output — AWS keys, tokens, private keys, the credential set; a base64-*wrapped* secret (decoded → `decoded:egress:*`) |
|
||
| **LLM05 · improper output handling** | Zero-click exfil carriers (EchoLeak, CVE-2025-32711) — markdown-image auto-fetch, inline / reference / autolink links, raw active HTML, standalone `data:` URIs; non-`https` `resource` URLs |
|
||
| **LLM06 · excessive agency** | A tool surface on the quarantined transform; credential use beyond the per-stage allowlist; capability isolation (`scoped_env`) |
|
||
| **LLM10 · fail-secure** | Forced-fallback attack (scan hit + transform failure → halt); compound weak-signal escalation; an un-scannable artifact fails **closed** — never a silent persist |
|
||
| **OKF structure (T1–T6)** | Body + frontmatter-value injection; YAML anchor / merge / nested / block DoS (parse-safety); `resource` allowlist; path traversal / reserved-name shadow; cross-link graph; provenance stamping |
|
||
| **Container / upload layer** | CSV/XLSX formula-injection (lead `= + - @`); zip-slip path escape; zip-bomb size cap; symlink refusal *(upload front-end — see the showcase)* |
|
||
|
||
The manifest ([`coverage.py`](src/llm_ingestion_guard/coverage.py)) is the single
|
||
source of truth for
|
||
[`tests/test_coverage_matrix.py`](tests/test_coverage_matrix.py), which asserts
|
||
total recall over every class, that every documented gap still holds (a closed gap
|
||
fails the test), and that every lexicon pattern has a case — so the matrix cannot
|
||
fall behind the code. The four classes it deliberately does **not** stop are called
|
||
out in [Honest limitations](#honest-limitations).
|
||
|
||
## The reusable contract (adopt-this checklist)
|
||
|
||
The actual product is this checklist, encoded as code you wire in order:
|
||
|
||
1. **Sanitize before fence.** Strip carrier classes (zero-width, BIDI,
|
||
Unicode-tag, HTML comment, `data:`) from untrusted input first.
|
||
2. **Fence untrusted input.** Spotlight-mark it in a randomized per-call
|
||
delimiter; strip attacker fence markers from the payload.
|
||
3. **Tool-less transform.** Call the model with zero tools. A successful
|
||
injection then has nothing to act with.
|
||
4. **Per-stage capability isolation.** The enrichment stage holds only the model
|
||
key; the publish stage holds only the publish credential; no stage holds both.
|
||
5. **Treat output as data.** Parse to a frozen schema; reject on structural
|
||
violation. The output never reaches a shell, git, or a filesystem path.
|
||
6. **Scan output before persist.** Run the lexicon + entropy + active-content
|
||
scan over the emitted text. Verbatim-carried payloads, model-emitted
|
||
instructions, and zero-click exfil carriers (EchoLeak-class markdown
|
||
images/links, raw active HTML, `data:` URIs) are caught here; `neutralize`
|
||
additionally defangs them, opt-in.
|
||
7. **Fail-secure on compound signals.** Injection hit + transform failure = halt
|
||
+ alert, never a silent verbatim commit.
|
||
8. **Minimal alert payloads.** Alert with a gate code + run ID, never content.
|
||
|
||
Steps 1-2 are `prepare_input`; steps 6-7 are `screen_output`; steps 3-5 are
|
||
yours; the contract asserters harden step 3-4.
|
||
|
||
## Honest limitations
|
||
|
||
Conceding these plainly is itself a control — it prevents the false assurance that
|
||
a green scan means safe content. The highest-impact items:
|
||
|
||
- **The contract carries the security, not the lexicon.** Pattern detection is
|
||
bypassable in isolation (character-injection, novel phrasings); the tool-less
|
||
transform + capability isolation + fail-secure are the wall.
|
||
- **Semantic / factual poisoning is invisible.** A false claim in clean prose
|
||
carries no suspicious token — the highest-impact gap for a wiki. Needs a `[judge]`
|
||
plugged into the `grounding` seam.
|
||
- **Text-only, extracted-text-only.** The core parses no files; OCR-embedded
|
||
instructions, macros, and multimodal stego are out of scope. `.pdf` is refused as
|
||
unsupported, not half-scanned.
|
||
- **A lone HIGH in *trusted* prose disposes to WARN**, and **insider in-place
|
||
edits** are outside the untrusted-content threat model — run genuinely untrusted
|
||
sources as untrusted.
|
||
- **Active-content severity grades on URL shape, not construct type.** A URL that
|
||
only *names* a remote document is LOW; one that can carry a value outward keeps
|
||
HIGH/MEDIUM. The conceded hole: a bare-path image on a hostile host still *fetches*
|
||
when rendered, so pure beaconing (reader IP, timing) is not graded.
|
||
- **Six documented gaps** the coverage matrix keeps honest: hex-wrapped secret
|
||
egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`),
|
||
pure beaconing, and short opaque URL segments.
|
||
|
||
**Full list — 27 items, each with the mechanism, plus the out-of-scope boundary:**
|
||
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
|
||
consumer corpora, including the false positives the URL-shape rule actually produces.
|
||
|
||
## Out-of-scope (documented boundary)
|
||
|
||
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
||
steganography; query-time / runtime guardrails; semantic factuality verification.
|
||
|
||
## Design & threat model
|
||
|
||
- [Design brief](docs/BRIEF.md) — what this repo contains and why.
|
||
- [URL-shape rule](docs/URL-SHAPE.md) — `is_ordinary_url` stated precisely enough to
|
||
reconstruct, with worked examples asserted against the implementation. **Read this
|
||
before reasoning about the rule from prose:** three consumers reconstructed it from
|
||
summaries and each produced a different wrong number on a real corpus.
|
||
- [Build plan](docs/PLAN.md) — module build order and the reuse map.
|
||
- [Adoption brief](docs/ADOPTION-BRIEF.md) — wiring the guard into an OKF
|
||
second-brain / LLM wiki, and a checklist for *when* to include it.
|
||
|
||
The contract is extracted from a working reference implementation (the
|
||
`claude-code-llm-wiki` Stage B enrichment pipeline). Threat-model anchors: OWASP
|
||
LLM Top-10 2025 (LLM01/02/04/05/06 strongest, LLM08 boundary, LLM09/10),
|
||
PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak (CVE-2025-32711).
|
||
|
||
## License
|
||
|
||
MIT — see [LICENSE](LICENSE).
|