open/llm-ingestion-pipeline-security
1
0
Fork 0
Code Issues Releases 4 Activity

fix(active-content): raw-html graded two inert shapes HIGH, and the fix moved a second surface

Browse source 
`is_active_tag`'s URL-attribute branch was a presence test: any element carrying
`href=`/`src=`/`action=` graded HIGH regardless of where the URL pointed. An MDX
`<Card href="/en/agent-sdk/quickstart">` reaches no attacker-controlled host, and
neither does APIM policy XML's `<set-header>`. It now requires an external target
-- the rule the markdown paths have applied since 0.3.1. `<base>` left the active
name set in the same change: HTML's `<base>` has its whole affordance in an `href`
the attribute branch still catches, and APIM's attribute-less `<base />` is inert.

Measured before and after in ONE session against one corpus state, because two of
the three corpora are living and a split would mix this with re-harvest drift:

  reference-corpus  389 docs   133 -> 108   (ceiling 107)
  vendor-harvest    187 docs   100 ->  98   (ceiling  62)
  generated-notes   550 docs    90 ->  88   (ceiling  49)

96% of the achievable reduction in reference-corpus, 5% in the wiki corpora. The
two classes had to be measured TOGETHER -- alone they free 3 and 13 documents,
together 25, because a document carrying one usually carries the other.

The second surface: `neutralize` imported `is_active_tag` by name, so this would
have silently narrowed the opt-in mutator too -- and no test discriminated the two
halves, since every `neutralize:raw-html` payload stays active under any narrowing
considered. That test is written first here. The predicates are now separate
symbols; the mutator keeps defanging anything, because over-defanging is auditable
and blocks nothing while under-defanging hands a human a live construct.

Behaviour change: a document whose only finding was one of these classes now WARNs
instead of holding. Detection is unchanged -- 128/128 classes, 6/6 gaps hold.

Self-safety: reading an attribute VALUE needs a pattern the presence test lacks. It
reuses the same literal alternation so no new run shape enters the table; its
`_REDOS_PAYLOADS` row denies the `=` the pattern requires, since a unit supplying it
matches at once and never exercises the run (the lexicon's `script-tag` row is the
cautionary case). 0.031-0.046s across five attack shapes at 100_000 chars against a
2.0s bound; `docs/redos-sweep.py` reports 0 candidates of 152. An attribute the
presence test saw but the value parser cannot read counts as external -- fail secure.

`docs/rawhtml-census.py` gains a PRODUCTION row that re-measures the shipped
predicate rather than a hypothesis, so a published number and the code cannot drift
apart unnoticed. README's limitation count moves 34 -> 33.

727 passed (was 717).
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 16:56:31 +02:00
commit 736f370cfb
9 changed files with 287 additions and 72 deletions
Download patch file Download diff file Expand all files Collapse all files

94
docs/LIMITATIONS.md
View file

@ -216,55 +216,42 @@ items; this is the full list, each with the mechanism.
where this would bite — the two corpora that reported a carrier breakdown contained
zero remote images and the third did not report one, so the image row of this
limitation remains unmeasured in the field.
- **Raw HTML with a *relative* URL attribute is HIGH, though it can reach no
attacker-controlled host.** The markdown paths test for an external target before
flagging; the raw-HTML path deliberately does not, because an active element needs
no URL at all (an `on*=` handler executes on its own). That reasoning covers event
handlers but over-reaches on the URL-attribute branch: an element outside the active
name set carrying `href="/en/agent-sdk/quickstart"` — an internal doc route — grades
HIGH. Measured on a vendor-docs corpus, where it lands on MDX components:
`<Card href="/…">` fires this way, and `<Frame>` fires on the *name* branch alone
because names are lower-cased and `frame` is in the active set — legacy HTML
framesets, which appear in essentially no modern documentation, while `Frame` is a
common component name. Case is not an available discriminator: HTML is
case-insensitive, so PascalCase cannot be treated as "component, not tag".
**Correction (2026-08-11): this bullet was named as the cause of the raw-html
false-positive rate, and it is not.** The bullet below used to attribute
`active:raw-html` in *52 of vendor-harvest's 98* to this over-reach. Measured by
re-running the gate with the URL-attribute branch narrowed to *external* targets
only, it frees **1 document in vendor-harvest, 1 in generated-notes, 3 in
reference-corpus**, because the documents carrying it almost always also carry a
tag that is active by *name*, and one raw-html finding per document is all it
takes. What actually drives the rate is the **name branch**: per wiki corpus `<a>`
298 occurrences, `<frame>` 94, `<img>` 63; in reference-corpus `<base>` 113.
**Do not read "frees 3" as "this over-reach is cheap" — the over-reach classes
co-occur, and measuring them one at a time understates both.** In reference-corpus,
narrowing the URL-attribute branch alone frees 3 and taking `<base>` off the name
branch alone frees 13, but doing **both** frees **25** — because a document
carrying one class usually carries the other, so closing either alone leaves the
document blocked by its twin. Against a ceiling of 26 (raw-html detection switched
off entirely) the pair captures 96% of what this detector costs that population.
The same pair frees only 2 of a 38-document ceiling in vendor-harvest and 2 of 41
in generated-notes: the over-reach is nearly the whole raw-html cost in APIM policy
XML and nearly none of it in vendor documentation.
**And the over-reach is not only MDX.** In reference-corpus it lands on Azure APIM
policy XML — `<set-header>` 50, `<ip-filter>` 3, `<set-query-parameter>` 2 — which
shares nothing with the `<Card href="/…">` shape beyond having a URL attribute.
**Any fix here moves two surfaces, not one:** `neutralize` imports `is_active_tag`
from this module, so narrowing it also stops the opt-in *mutator* from defanging
the same tags. No test covers that half — the suite's `neutralize:raw-html` cases
use payloads that stay active under every narrowing considered — so the mutator
would change behaviour silently. Whoever closes this writes that test first.
- **`<base />` in Azure APIM policy XML grades HIGH on the name branch alone.**
APIM's `<base />` means "run the inherited policy" and carries no attributes; it
collides with HTML's `<base>`, whose entire affordance is its `href`. It appears in
**25 of reference-corpus's 389** documents (`grep -rlE '<base\s*/?>'` — a loose
`grep '<base'` says 30 and is wrong: it also matches the literal `<base64_string>`
placeholder, which is not a tag this detector fires on). An attribute-less `<base>` has no
affordance in any renderer, and a `<base href="…">` would still be caught by the
URL-attribute branch — so the name-branch entry is doing no work here that the
attribute branch does not already do. This is a distinct over-reach class from the
MDX/APIM one above, and it was unnamed until the 2026-08-11 census.
- **What `active:raw-html` still costs benign documentation is mostly ordinary HTML,
not over-reach — and that residual is not a narrowing away.** Two over-reach
classes were closed in 0.6.0: the URL-attribute branch now requires an *external*
target (an element outside the active name set carrying `href="/en/agent-sdk/quickstart"`
reaches no attacker-controlled host), and `<base>` left the active *name* set (APIM's
attribute-less `<base />` means "run the inherited policy"; HTML's `<base>` has its
whole affordance in an `href` the attribute branch still catches). Measured before
and after in one session against one corpus state — `docs/rawhtml-census.py`, whose
`PRODUCTION` row re-measures the shipped predicate rather than a hypothesis:
reference-corpus 133 → **108** of 389, vendor-harvest 100 → **98** of 187,
generated-notes 90 → **88** of 550. Against a ceiling of 26 / 38 / 41 (raw-HTML
detection switched off entirely) that is **96% of the achievable reduction in
reference-corpus and 5% in the two wiki corpora** — the over-reach was nearly the
whole raw-html cost in APIM policy XML and nearly none of it in vendor
documentation.
**The classes had to be measured together.** Alone, the URL-attribute branch frees
3 documents in reference-corpus and `<base>` frees 13; together they free 25. A
document carrying one usually carries the other, so closing either alone leaves it
blocked by its twin — one-at-a-time measurement understates both, and reading
"frees 3" as "this over-reach is cheap" is how this document got the attribution
wrong for three releases.
**What remains is real HTML in vendor documentation**, flagged by the *name*
branch, correctly: per wiki corpus `<a>` 298 occurrences, `<frame>` 94, `<img>` 63
— identical in both because 184 of their documents share ancestors. `<frame>` is
the one arguable member: names are lower-cased and `frame` is in the active set for
legacy HTML framesets, which appear in essentially no modern documentation, while
`Frame` is a common MDX component name. Case is not an available discriminator:
HTML is case-insensitive, so PascalCase cannot be treated as "component, not tag".
Recovering the rest is **not** a further narrowing — it needs a carrier split and a
new label (`active:raw-html-link`), because raw HTML has no ordinary form and the
class collapses to one finding per document.
**The scanner and the mutator no longer share a predicate.** Until 0.6.0 `neutralize`
imported `is_active_tag` by name, so any narrowing moved the opt-in mutator too, and
no test discriminated the two halves. They are now `is_active_tag` and
`is_defangable_tag`; the mutator kept the broader behaviour deliberately, pinned by
`tests/test_neutralize.py::test_mutator_still_defangs_what_the_scanner_now_lets_pass`.
- **Raw-HTML findings count end tags.** `</a>` is active by name on its own, so a
corpus census that counts only opening tags understates what this detector reports
by roughly the ratio of closing to opening active tags (measured at 1.6× on one
@ -284,6 +271,15 @@ items; this is the full list, each with the mechanism.
| generated-notes | model-written notes at their own persist gate | 547 | **88 (16.1%)** — 61 fail-secure, 27 held |
| reference-corpus | first-party authored reference material | 389 | **133 (34.2%)** — 80 fail-secure, 53 held |
**These three numbers predate the 0.6.0 raw-HTML narrowing and are left as
published**, because two of the three corpora are living — re-harvested by their
owning repo, now 187 and 550 documents against the 185 and 547 measured here — so
rewriting the cells would mix a code change with corpus drift. The narrowing's
effect was measured separately, before and after against one corpus state:
reference-corpus (static, and reproduced at exactly 389/133) drops to **108**, the
two wiki rows to **98** and **88** on their current state. The bullet above carries
the method; `docs/rawhtml-census.py LABEL=<path>` reproduces it.
**The unit is a document and the gate is the strict one:** `screen_output(doc,
PRESET_USER_UPLOAD)`, counting `disposition is not WARN`. WARN is the benign
outcome (persisted, with a note), so a *finding* is not a false positive — only a

21
docs/rawhtml-census.py
View file

@ -22,10 +22,15 @@ TWO METHOD TRAPS IT EXISTS TO AVOID:
effect with corpus drift. Every candidate here runs against the same corpus state
in one process, and `base` is re-measured rather than quoted from the doc.
The candidates are applied by replacing `active_content.is_active_tag` in-process.
That mirrors a real edit for the *scanner* path only: `neutralize` imports the
symbol by name, so a real edit would also change the mutator, which this script does
not simulate. See the raw-HTML bullets in `docs/LIMITATIONS.md`.
The candidates are applied by replacing `active_content.is_active_tag` in-process,
which mirrors a real edit to the *scanner*. Since 0.6.0 that is the whole story:
`neutralize` calls its own `is_defangable_tag`, so patching this symbol cannot
move the mutator. Before 0.6.0 the two shared one symbol and this caveat read the
other way. See the raw-HTML bullets in `docs/LIMITATIONS.md`.
The `PRODUCTION` row is the only one that is not a hypothetical: it leaves the
shipped predicate in place. A shipped narrowing must equal its candidate row, and
saying so in the output is what keeps the doc's numbers checkable after the fact.
USAGE — corpus roots are arguments, never hardcoded; the corpora live in private
consumer repos and their paths must not reach a public mirror:
@ -88,7 +93,7 @@ def _variant(*, drop: frozenset[str] = frozenset(), external_only: bool = False)
CANDIDATES = [
("base", _variant()),
("pre-0.6.0 (no narrowing)", _variant()),
# The URL-attribute branch requires an EXTERNAL target — the rule the markdown
# paths already apply. A relative `href` reaches no attacker-controlled host.
("A: url-attr external-only", _variant(external_only=True)),
@ -98,6 +103,10 @@ CANDIDATES = [
("base-url: <base> needs a URL", _variant(drop=frozenset({"base"}))),
("A + base-url (both)",
_variant(drop=frozenset({"base"}), external_only=True)),
# Not a hypothetical: the shipped predicate, unpatched. `A + base-url` is what
# 0.6.0 shipped, so these two rows must agree — a mismatch means the code and
# this script have drifted apart and every number below is suspect.
("PRODUCTION (as shipped)", None),
# The CEILING: no narrowing can free more than switching the detector off.
("NONE (ceiling)", lambda name, attrs: False),
]
@ -180,7 +189,7 @@ def main() -> None:
baseline = None
for name, fn in CANDIDATES:
ac.is_active_tag = fn
ac.is_active_tag = original if fn is None else fn
non_warn = sum(
1 for t in texts
if screen_output(t, PRESET_USER_UPLOAD).disposition is not BENIGN