1
0
Fork 0

fix(calibration): grade active content on URL shape, not construct type

v0.3.0 made the untrusted upload path unusable: measured on both doors, an
ordinary remote image fail_secure'd and an ordinary link/autolink/refdef
quarantined, so only documents without external references persisted.

Two independent defects compounded; neither fix works alone:

1. `markdown-image: HIGH` fired on any external image. The exfil primitive is a
   URL that moves bytes outward, not an image. `is_ordinary_url` now grades on
   shape - http(s)/protocol-relative, no query, no userinfo, no percent-escape,
   no opaque host label or path segment -> LOW; anything data-carrying keeps the
   carrier's severity. raw-html and data: URIs stay HIGH unconditionally.
   Opacity reuses entropy's primitives; floors calibrated against real doc URLs
   (worst legit token H=4.08, exfil segments 4.36-4.54) and frozen in
   calibration.

2. The quarantine_default floor fired on ANY finding, a premise that broke when
   every ordinary link became a finding. It now fires at MEDIUM+ - a no-op for
   every detector that shipped before 0.3.0 (no LOW/INFO exists), which is what
   makes this a patch rather than a minor.

The corpus blind spot that let this pass 522 green tests is closed: the FP
corpus carries realistic markdown and is asserted on the OUTPUT gate under
PRESET_USER_UPLOAD, with a counter-corpus of exfil-shaped URLs that must still
block. Beaconing and short opaque segments are conceded in LIMITATIONS and
asserted by the coverage matrix rather than papered over.

No new public API; no new preset (0.4.0 work); allow_reserved default unchanged.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 15:36:02 +02:00
commit 6e9b8168e3
13 changed files with 533 additions and 46 deletions

View file

@ -1,9 +1,9 @@
# llm-ingestion-guard
![Version](https://img.shields.io/badge/version-0.3.0-blue)
![Version](https://img.shields.io/badge/version-0.3.1-blue)
![Status](https://img.shields.io/badge/status-alpha-orange)
![Python](https://img.shields.io/badge/python-3.10%2B-purple)
![Tests](https://img.shields.io/badge/tests-522_passing-green)
![Tests](https://img.shields.io/badge/tests-577_passing-green)
![License](https://img.shields.io/badge/license-MIT-lightgrey)
**Write-time ingestion is the trust boundary that query-time guardrails
@ -42,7 +42,7 @@ may still change. There are real limitations, stated plainly below; read them.
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
```bash
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.0"
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.1"
```
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
@ -207,10 +207,15 @@ a green scan means safe content. The highest-impact items:
- **A lone HIGH in *trusted* prose disposes to WARN**, and **insider in-place
edits** are outside the untrusted-content threat model — run genuinely untrusted
sources as untrusted.
- **Four documented gaps** the coverage matrix keeps honest: hex-wrapped secret
egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`).
- **Active-content severity grades on URL shape, not construct type.** A URL that
only *names* a remote document is LOW; one that can carry a value outward keeps
HIGH/MEDIUM. The conceded hole: a bare-path image on a hostile host still *fetches*
when rendered, so pure beaconing (reader IP, timing) is not graded.
- **Six documented gaps** the coverage matrix keeps honest: hex-wrapped secret
egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`),
pure beaconing, and short opaque URL segments.
**Full list — 15 items, each with the mechanism, plus the out-of-scope boundary:**
**Full list — 18 items, each with the mechanism, plus the out-of-scope boundary:**
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md).
## Out-of-scope (documented boundary)