1
0
Fork 0

fix(active-content): raw-html graded two inert shapes HIGH, and the fix moved a second surface

`is_active_tag`'s URL-attribute branch was a presence test: any element carrying
`href=`/`src=`/`action=` graded HIGH regardless of where the URL pointed. An MDX
`<Card href="/en/agent-sdk/quickstart">` reaches no attacker-controlled host, and
neither does APIM policy XML's `<set-header>`. It now requires an external target
-- the rule the markdown paths have applied since 0.3.1. `<base>` left the active
name set in the same change: HTML's `<base>` has its whole affordance in an `href`
the attribute branch still catches, and APIM's attribute-less `<base />` is inert.

Measured before and after in ONE session against one corpus state, because two of
the three corpora are living and a split would mix this with re-harvest drift:

  reference-corpus  389 docs   133 -> 108   (ceiling 107)
  vendor-harvest    187 docs   100 ->  98   (ceiling  62)
  generated-notes   550 docs    90 ->  88   (ceiling  49)

96% of the achievable reduction in reference-corpus, 5% in the wiki corpora. The
two classes had to be measured TOGETHER -- alone they free 3 and 13 documents,
together 25, because a document carrying one usually carries the other.

The second surface: `neutralize` imported `is_active_tag` by name, so this would
have silently narrowed the opt-in mutator too -- and no test discriminated the two
halves, since every `neutralize:raw-html` payload stays active under any narrowing
considered. That test is written first here. The predicates are now separate
symbols; the mutator keeps defanging anything, because over-defanging is auditable
and blocks nothing while under-defanging hands a human a live construct.

Behaviour change: a document whose only finding was one of these classes now WARNs
instead of holding. Detection is unchanged -- 128/128 classes, 6/6 gaps hold.

Self-safety: reading an attribute VALUE needs a pattern the presence test lacks. It
reuses the same literal alternation so no new run shape enters the table; its
`_REDOS_PAYLOADS` row denies the `=` the pattern requires, since a unit supplying it
matches at once and never exercises the run (the lexicon's `script-tag` row is the
cautionary case). 0.031-0.046s across five attack shapes at 100_000 chars against a
2.0s bound; `docs/redos-sweep.py` reports 0 candidates of 152. An attribute the
presence test saw but the value parser cannot read counts as external -- fail secure.

`docs/rawhtml-census.py` gains a PRODUCTION row that re-measures the shipped
predicate rather than a hypothesis, so a published number and the code cannot drift
apart unnoticed. README's limitation count moves 34 -> 33.

727 passed (was 717).
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 16:56:31 +02:00
commit 736f370cfb
9 changed files with 287 additions and 72 deletions

View file

@ -268,11 +268,6 @@ def test_default_source_is_output_and_override_respected():
# closed over-block should fail here and force the doc to be updated.
@pytest.mark.parametrize("cid,text", [
# Fires on the URL-attr branch although the target is a relative doc route,
# which cannot reach an attacker-controlled host. `Card` is not in the active
# name set — the href alone carries it.
("relative-href-on-inactive-name",
'<Card title="Quickstart" icon="play" href="/en/agent-sdk/quickstart">'),
# Fires on the *name* branch: names are lower-cased and `frame` is in the
# active set (legacy HTML framesets), while `Frame` is a common MDX component.
("mdx-component-named-like-a-tag", "<Frame>"),
@ -284,6 +279,51 @@ def test_raw_html_overblocks_are_still_high(cid, text):
assert finding[0].severity is Severity.HIGH, f"{cid}: {finding[0].severity}"
# --- the two over-blocks CLOSED in 0.6.0 (the `A + base-url` narrowing) -------
# Measured together, never one at a time: the classes co-occur, so a document
# blocked by both is freed by neither alone. On the reference corpus the pair
# frees 25 of 133 non-WARN documents; on the two wiki corpora, 2 each. Recall is
# unchanged (128/128 + 6/6). Method and numbers: `docs/rawhtml-census.py`.
def test_relative_url_attr_on_an_inactive_name_is_not_active():
# `_URL_ATTR_RE` is a presence test, so a doc-relative route on a name outside
# the active set used to carry HIGH on its own. A relative target resolves
# against the rendering host and reaches nothing attacker-controlled — the rule
# the markdown paths have applied since 0.3.1.
assert not [
f for f in scan_active_content(
'<Card title="Quickstart" icon="play" href="/en/agent-sdk/quickstart">'
).findings if f.label == "active:raw-html"
]
def test_attributeless_base_is_not_active():
# `<base>`'s whole affordance is its `href`, which the URL-attribute branch
# still catches (below). An attribute-less `<base />` — Azure APIM policy XML,
# 25 documents in the reference corpus — has no affordance in any renderer.
assert not [f for f in scan_active_content("<base />").findings
if f.label == "active:raw-html"]
@pytest.mark.parametrize("cid,text", [
("absolute", '<Card title="Docs" href="https://evil.example/leak?d=x">'),
("protocol-relative", '<Card href="//evil.example/leak">'),
("non-http-scheme", '<Card href="file:///etc/passwd">'),
("base-keeps-its-href", '<base href="https://evil.example/">'),
# `srcset` is a comma-separated candidate list. A relative FIRST candidate must
# not mask an external one behind it — the value is not one URL.
("srcset-second-candidate", '<Card srcset="a.png 1x, https://evil.example/b.png 2x">'),
# The value parser saw the attribute but can resolve no value. The gap must
# over-block, never under-block.
("unreadable-value-fails-secure", "<Card href= >"),
])
def test_external_url_attr_is_still_active(cid, text):
finding = [f for f in scan_active_content(text).findings
if f.label == "active:raw-html"]
assert len(finding) == 1, f"{cid}: raw-html not reported"
assert finding[0].severity is Severity.HIGH, f"{cid}: {finding[0].severity}"
def test_raw_html_counts_end_tags():
# `</a>` is active by name on its own, so a corpus census counting only opening
# tags understates this detector's `count`. The class still collapses to ONE