1
0
Fork 0

docs(limitations): bound two URL-shape false positives with field measurements

Two consumers measured the 0.3.1 URL-shape rule against real corpora on the
same day. Record what they found, and pin the dispositions both of them
inferred wrongly.

Percent-escape FP: zero occurrences across both corpora (0/347 external URLs
in a 527-document vendor-docs corpus, 0/81 in a capture store). This bounds
the shape rather than closing it -- a consumer that slugs filenames from
titles produces %20 systematically, and that corpus is still unmeasured.

Query FP: the over-block that actually occurs. The two corpora hit disjoint
benign populations -- 16/16 publisher-authored utm_* tracking versus 35/35
content identity (?v=, ?channel_id=) where the parameter IS the resource.
An allowlist keyed on tracking-parameter names resolves the first entirely
and the second not at all, so no parameter-level remedy covers both. That
is input to the 0.4.0 axis-separation scope, not a fix here.

Both consumers reported dispositions they had inferred rather than run, and
both were wrong: a query-carrying link is MEDIUM, so it is held or warned,
never hard-failed. tests/test_wiring.py now pins that, plus the active-tag
gate -- counting raw HTML tags overcounts what the gate flags, since
formatting markup is inert and only name/on*=/URL-attr tags are active.

593 passed.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 20:56:14 +02:00
commit 956c835d38
2 changed files with 83 additions and 1 deletions

View file

@ -92,7 +92,27 @@ items; this is the full list, each with the mechanism.
QUARANTINE_REVIEW / FAIL_SECURE on an untrusted upload. Obfuscated encoding is a
core exfil primitive and the ambiguous case is put on the review side deliberately;
it is listed here because it is the same *class* of over-block that 0.3.1 fixed,
in a rarer shape, and it is the first thing to re-measure against a real corpus.
in a rarer shape. **Measured (2026-07-25, two independent consumer corpora):
zero occurrences** — 0 of 347 external URLs in a 527-document vendor-docs corpus,
0 of 81 in a real capture store. That *bounds* the shape as rare in linked content;
it does not retire it. A consumer deriving filenames from titles through a slugger
produces `%20` systematically rather than incidentally, and that corpus is still
unmeasured — so the open question is narrower than it was, not closed.
- **A non-empty query is graded as data-carrying — the over-block that actually
occurs in the field.** The same two corpora found this rule firing on *disjoint*
benign populations: 16 of 16 query-carrying external URLs in the vendor-docs corpus
were publisher-authored campaign tracking (`utm_*` on the publisher's own domains),
while 35 of 35 in the capture store were content identity (`?v=`, `?channel_id=`,
`?all=true`) where the parameter *is* the resource. **No parameter-level remedy
covers both:** an allowlist keyed on tracking-parameter names resolves the first
population entirely and the second not at all, and stripping the query is lossless
for the first while dereferencing nothing for the second. The cost is bounded — a
query-carrying *link* is MEDIUM, so it disposes QUARANTINE_REVIEW under
`PRESET_USER_UPLOAD` and WARN under `PRESET_TRUSTED_SOURCE`: held or warned, never
hard-failed (pinned in `tests/test_wiring.py`, because both consumers inferred a
hard block rather than running it). An *image* keeps HIGH, and that is the carrier
where this would bite — neither corpus contained a single remote image, so the
image row of this limitation remains unmeasured in the field.
- **URL fragments are not graded.** A fragment is never sent to the server, so it
cannot carry data to the host a renderer auto-fetches, and `…/overview#section` is
the most common shape in real documentation. The residual: a *clicked* link to an