Two changes that had to ship together, because they co-occur.
`active:raw-html-link` (MEDIUM) splits the click-required carriers out of
`active:raw-html`. The same URL was LOW as `[t](url)` and HIGH as
`<a href="url">` — an asymmetry produced by syntax, not by affordance, on a
carrier the markdown path has graded MEDIUM since 0.3.1. The event-handler test
runs first, so `<a onclick=...>` stays HIGH. The url-attribute branch stays HIGH
too: a name outside the active set has unknown rendering, and grading
`<Card src=...>` as a link would be reasoning rather than measurement.
The no-URL narrowing makes `</a>`, `<Frame>`, `<video />` and `<img alt=...>`
without `src` inert — `<base />`'s argument from 0.6.0 applied to the rest of the
name branch. It tests for the URL attribute's PRESENCE, not for a readable value,
so the fail-secure gap `_url_attr_is_external` leaves open is not reopened here.
WHY TOGETHER: the narrowing strips a document's `</a>`/`<Frame>` and what remains
is the `<a href=...>` the split grades down, so each alone leaves the document
blocked by the other's residue.
`active_tag_class` is now the classification point and `is_active_tag` wraps it.
The census patches the former: a boolean could only express a narrowing, never a
regrade, so every carrier candidate would have measured equal to PRODUCTION —
silently, and in the direction that reads as "no change helps".
TWO COSTS, BOTH RECORDED RATHER THAN GLOSSED:
- The split TIGHTENS the trusted tier. One finding becomes two, and >=2 findings
at MEDIUM+ trip the compound overlay, so a document carrying both an `<img src>`
and an `<a href>` goes WARN -> quarantine_review on PRESET_TRUSTED_SOURCE. On
that preset it is the only direction the split can move anything. The census
now reports a TIGHTENS column on both trust tiers against the previously
shipped row — "frees N" without "tightens M" is a one-sided number.
- `count` drops on documents containing `</a>`, a published field moving under a
meaning that did not change.
MEASURED: reference-corpus (389) 54 -> 53 fail_secure, tightens 0/0, and the
census `PRODUCTION` row equals its `C1 + D` candidate row for row. The census
also reproduces 133/3/13/108/25 exactly, so it is calibrated against every
published historical number. The two wiki corpora are NOT yet re-measured; the
tree says so explicitly in the docstring, LIMITATIONS and CHANGELOG rather than
carrying probe numbers as fact.
791 tests (was 759), coverage 129/129, 6/6 documented gaps holding. Version
bumped to 0.7.0 across every surface; no tag is set until the measurement lands.
The output gate claimed LLM10 self-safety on the grounds that its patterns have
no nested quantifiers. True, and irrelevant: nesting is not what makes these
blow up. A run in front of a REQUIRED literal, reachable from a short anchor, is
enough -- crafted input repeats the anchor and never supplies the literal, so
every start position rescans the tail. Quadratic, not exponential, and the
max_scan_chars cap does not help: it bounds the input, and quadratic work on a
bounded input is still hours.
Measured, not argued. `<a:` x 100_000 took 23.4s in AUTOLINK_RE alone; the
composed gate on that payload took 458.7s, extrapolating to ~5.7 hours at the
1_000_000-char input the gate itself accepts. Size-matched ordinary prose runs
0.31s, so the separation is 18x-660x -- unlike the blob in the neighbouring
test, which is the *faster* side of prose and never exercised backtracking.
Two fixes, chosen per pattern rather than uniformly:
- active_content + lexicon JSON (15 runs): exclude the character that opens the
pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot reach
past the next start position and the per-start costs telescope. Verified to
cost no recall: long URLs, long alt text, and `<` inside a quoted attribute
all still match. Bounding instead would have been linear too but wrong here --
the content is attacker-controlled, so padding past a bound would be a
one-line bypass of the EchoLeak class this table exists to catch.
- connstr egress (4 runs): bound the password at MAX_CONNSTR_VALUE. The
exclusion fix is unavailable -- the anchor character is `/` and passwords
containing `/` are the common case (measured: they match today). The residual
miss is a credential over 256 chars; a token that long is still caught by
egress:jwt-token.
hybrid-xss:script-tag had neither option: its run is the script BODY, which may
legitimately contain `<`. It now matches the opening tag and drops the
`</script>` requirement. That also closes a fail-open -- `<script>alert(1)`
unclosed was silently missed -- at the cost of flagging prose that merely
mentions `<script>`, now documented.
Found by the composed-gate test staying red after every individual scanner was
already linear: the lexicon's six html-obfuscation patterns were the remaining
813x. A per-scanner test alone would have shipped that.
662 passed (was 642), and faster than before the fix.
v0.3.0 made the untrusted upload path unusable: measured on both doors, an
ordinary remote image fail_secure'd and an ordinary link/autolink/refdef
quarantined, so only documents without external references persisted.
Two independent defects compounded; neither fix works alone:
1. `markdown-image: HIGH` fired on any external image. The exfil primitive is a
URL that moves bytes outward, not an image. `is_ordinary_url` now grades on
shape - http(s)/protocol-relative, no query, no userinfo, no percent-escape,
no opaque host label or path segment -> LOW; anything data-carrying keeps the
carrier's severity. raw-html and data: URIs stay HIGH unconditionally.
Opacity reuses entropy's primitives; floors calibrated against real doc URLs
(worst legit token H=4.08, exfil segments 4.36-4.54) and frozen in
calibration.
2. The quarantine_default floor fired on ANY finding, a premise that broke when
every ordinary link became a finding. It now fires at MEDIUM+ - a no-op for
every detector that shipped before 0.3.0 (no LOW/INFO exists), which is what
makes this a patch rather than a minor.
The corpus blind spot that let this pass 522 green tests is closed: the FP
corpus carries realistic markdown and is asserted on the OUTPUT gate under
PRESET_USER_UPLOAD, with a counter-corpus of exfil-shaped URLs that must still
block. Beaconing and short opaque segments are conceded in LIMITATIONS and
asserted by the coverage matrix rather than papered over.
No new public API; no new preset (0.4.0 work); allow_reserved default unchanged.
Session D: move every calibration constant (entropy floors 5.4/128, 5.1/64,
4.7/40 + shape floors; MAX_SCAN_CHARS; rot13-min; cognitive-load lengths
2000/2500; disposition ranks; active-content severities) into one documented
calibration.py, so a parallel Node/TS port can mirror exactly the same numbers.
Pure refactor, zero behavior change: calibration is a leaf module (imports only
report.Severity) that entropy/lexicon/disposition/active_content now source
their thresholds from. MAX_SCAN_CHARS is re-exported from lexicon so output.py
and existing callers are unaffected. The 347 pre-existing tests pass unmodified;
new test_calibration.py freezes the values and asserts each detector actually
reads its threshold from calibration (identity-checked, not a dead copy).
