`docs/fp-sweep.py` and `docs/rawhtml-census.py` produce the numbers published in
`docs/LIMITATIONS.md`, and both reach past the public API into private module
state -- `active_content._ACTIVE_TAGS`, `_EVENT_ATTR_RE`, `_URL_ATTR_RE`,
`_has_external_target`, `calibration.RISK_RANK`. A rename inside `src/` broke
them while the suite stayed green, and the breakage would have surfaced months
later, at the moment someone tried to re-measure a published claim. This was the
last uncovered contract in the repo.
The coverage found a live one. `rawhtml-census.has_external_url_attr` parsed
attributes with its own pattern instead of calling the shipped reader, and the
copy had drifted on two shapes:
- a URL attribute reached through a prefix. `_URL_ATTR_RE` matches `src=`
inside `data-src=` on a word boundary, so the shipped predicate reads the
value and blocks; the census's own name table saw `data-src` and skipped it.
- a multi-candidate `srcset`. The shipped reader splits on `[,\s]+` so a
relative first candidate cannot mask an external one behind it; the census
tested the whole attribute value as a single URL.
Both made the `A` candidate rows free documents the shipped predicate keeps --
under-counting against the `PRODUCTION` row printed directly beside them, which
that row exists to expose. The census now delegates to
`active_content._url_attr_is_external`, so there is one reader, not two. This
repo already carries the general form of that lesson in `docs/URL-SHAPE.md`:
three consumers reconstructed a predicate from prose and each got a different
wrong answer.
NO PUBLISHED NUMBER MOVED. Re-measured against all three live populations after
the fix, in one session each: reference-corpus 389 docs (A frees 3, base-url 13,
both 25 -- the numbers in `docs/LIMITATIONS.md`, unchanged), vendor-harvest 187,
generated-notes 550. `PRODUCTION` equals `A + base-url` in all three (108/108,
98/98, 88/88), and vendor-harvest exercises the corrected branch for real (8
external `Card` attributes). The defect was latent, not published.
The tests are scoped to what a test here can honestly hold. The corpora live
outside this repo in private consumer repos, so neither script can be run end to
end from the suite and a stand-in corpus would only pin a fiction. What is
pinned: every imported name still exists with the shape used; `fp-sweep`'s
metric guard fails when the action map is re-mapped (a guard that cannot fail
protects nothing); `measure` still reads `.disposition` and `.assessment` and
excludes empty files from the denominator; the census's in-process patch point
still moves the gate, so a census patching a dead symbol cannot print six
identical rows and read as a finding; `PRODUCTION` equals `A + base-url` across
every branch of the predicate; and both scripts still refuse an argument-less
run rather than measuring nothing.
759 passing (was 736). Coverage matrix unchanged at 128/128 with 6/6 documented
gaps holding. The `736` in `docs/ADOPTION-BRIEF.md` is scoped "as of v0.6.1" and
is correct for that tag; it moves to 759 at the next version bump.
`is_active_tag`'s URL-attribute branch was a presence test: any element carrying
`href=`/`src=`/`action=` graded HIGH regardless of where the URL pointed. An MDX
`<Card href="/en/agent-sdk/quickstart">` reaches no attacker-controlled host, and
neither does APIM policy XML's `<set-header>`. It now requires an external target
-- the rule the markdown paths have applied since 0.3.1. `<base>` left the active
name set in the same change: HTML's `<base>` has its whole affordance in an `href`
the attribute branch still catches, and APIM's attribute-less `<base />` is inert.
Measured before and after in ONE session against one corpus state, because two of
the three corpora are living and a split would mix this with re-harvest drift:
reference-corpus 389 docs 133 -> 108 (ceiling 107)
vendor-harvest 187 docs 100 -> 98 (ceiling 62)
generated-notes 550 docs 90 -> 88 (ceiling 49)
96% of the achievable reduction in reference-corpus, 5% in the wiki corpora. The
two classes had to be measured TOGETHER -- alone they free 3 and 13 documents,
together 25, because a document carrying one usually carries the other.
The second surface: `neutralize` imported `is_active_tag` by name, so this would
have silently narrowed the opt-in mutator too -- and no test discriminated the two
halves, since every `neutralize:raw-html` payload stays active under any narrowing
considered. That test is written first here. The predicates are now separate
symbols; the mutator keeps defanging anything, because over-defanging is auditable
and blocks nothing while under-defanging hands a human a live construct.
Behaviour change: a document whose only finding was one of these classes now WARNs
instead of holding. Detection is unchanged -- 128/128 classes, 6/6 gaps hold.
Self-safety: reading an attribute VALUE needs a pattern the presence test lacks. It
reuses the same literal alternation so no new run shape enters the table; its
`_REDOS_PAYLOADS` row denies the `=` the pattern requires, since a unit supplying it
matches at once and never exercises the run (the lexicon's `script-tag` row is the
cautionary case). 0.031-0.046s across five attack shapes at 100_000 chars against a
2.0s bound; `docs/redos-sweep.py` reports 0 candidates of 152. An attribute the
presence test saw but the value parser cannot read counts as external -- fail secure.
`docs/rawhtml-census.py` gains a PRODUCTION row that re-measures the shipped
predicate rather than a hypothesis, so a published number and the code cannot drift
apart unnoticed. README's limitation count moves 34 -> 33.
727 passed (was 717).
Yesterday's correction fixed a wrong causal claim and published a second one.
It reported that narrowing the URL-attribute branch frees 3 documents in
reference-corpus and left the reader to conclude the over-reach is cheap.
Measured together with the `<base>` name-branch fix it frees 25, against a
ceiling of 26 — 96% of what the detector costs that population. A document
blocked by two over-reach classes is freed by neither alone.
The same pair frees 2 of a 38-document ceiling in vendor-harvest and 2 of 41
in generated-notes. The over-reach is nearly the whole raw-html cost in Azure
APIM policy XML and nearly none of it in vendor documentation.
Three further corrections:
- `<base />` appears in 25 reference-corpus documents, not 30. The published
count came from `grep '<base'`, which also matched the literal
`<base64_string>` placeholder — not a tag this detector fires on.
- Any fix moves TWO surfaces: `neutralize` imports `is_active_tag` by name, so
narrowing it also stops the opt-in mutator defanging the same tags. No test
covers that half; the suite's `neutralize:raw-html` payloads stay active
under every narrowing considered.
- Adds `docs/rawhtml-census.py` so the ladder is reproducible instead of
living in a session scratchpad. Corpus roots are arguments, never hardcoded.
Measured under the real edit, not reasoned: the suite fails exactly one test,
`test_raw_html_overblocks_are_still_high[relative-href-on-inactive-name]`,
which exists to force this doc update when an over-block closes. Recall holds
at 128/128 and all 6 documented gaps still hold.
