1
0
Fork 0

fix(calibration): grade active content on URL shape, not construct type

v0.3.0 made the untrusted upload path unusable: measured on both doors, an
ordinary remote image fail_secure'd and an ordinary link/autolink/refdef
quarantined, so only documents without external references persisted.

Two independent defects compounded; neither fix works alone:

1. `markdown-image: HIGH` fired on any external image. The exfil primitive is a
   URL that moves bytes outward, not an image. `is_ordinary_url` now grades on
   shape - http(s)/protocol-relative, no query, no userinfo, no percent-escape,
   no opaque host label or path segment -> LOW; anything data-carrying keeps the
   carrier's severity. raw-html and data: URIs stay HIGH unconditionally.
   Opacity reuses entropy's primitives; floors calibrated against real doc URLs
   (worst legit token H=4.08, exfil segments 4.36-4.54) and frozen in
   calibration.

2. The quarantine_default floor fired on ANY finding, a premise that broke when
   every ordinary link became a finding. It now fires at MEDIUM+ - a no-op for
   every detector that shipped before 0.3.0 (no LOW/INFO exists), which is what
   makes this a patch rather than a minor.

The corpus blind spot that let this pass 522 green tests is closed: the FP
corpus carries realistic markdown and is asserted on the OUTPUT gate under
PRESET_USER_UPLOAD, with a counter-corpus of exfil-shaped URLs that must still
block. Beaconing and short opaque segments are conceded in LIMITATIONS and
asserted by the coverage matrix rather than papered over.

No new public API; no new preset (0.4.0 work); allow_reserved default unchanged.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 15:36:02 +02:00
commit 6e9b8168e3
13 changed files with 533 additions and 46 deletions

View file

@ -60,6 +60,25 @@ def test_active_content_severity_frozen():
}
def test_url_shape_thresholds_frozen():
# 0.3.1: severity grades on URL shape. These floors sit above every
# legitimate documentation URL token measured on 2026-07-25 (worst: H=4.08)
# and below the base64/hex payload segments an exfil path uses (4.36-4.54).
assert cal.ACTIVE_CONTENT_ORDINARY_SEVERITY is Severity.LOW
assert (cal.URL_OPAQUE_ENTROPY_H, cal.URL_OPAQUE_MIN_LEN) == (4.4, 24)
assert cal.URL_OPAQUE_HEX_MIN_LEN == 32
def test_no_detector_emitted_low_before_the_url_shape_change():
"""The floor change (any finding -> MEDIUM+) is only honest as a *patch* if
nothing that shipped before it emitted LOW — otherwise it would silently
loosen an existing consumer's gate. The lexicon is the only table-driven
severity source; assert it still holds no LOW/INFO pattern."""
from llm_ingestion_guard.lexicon import load_lexicon
assert not [p for p in load_lexicon()
if p.severity in (Severity.LOW, Severity.INFO)]
# --- binding: each detector reads its threshold from calibration ------------
# The freeze is meaningful only if the modules actually READ these values. An
# import alias binds the SAME object, so identity (`is`) proves the single
@ -97,3 +116,7 @@ def test_disposition_module_sources_from_calibration():
def test_active_content_module_sources_from_calibration():
from llm_ingestion_guard import active_content
assert active_content._SEVERITY is cal.ACTIVE_CONTENT_SEVERITY
assert active_content._ORDINARY_SEVERITY is cal.ACTIVE_CONTENT_ORDINARY_SEVERITY
assert active_content._OPAQUE_H is cal.URL_OPAQUE_ENTROPY_H
assert active_content._OPAQUE_MIN_LEN is cal.URL_OPAQUE_MIN_LEN
assert active_content._OPAQUE_HEX_LEN is cal.URL_OPAQUE_HEX_MIN_LEN