`5667063` moved test_output.py's ReDoS bounds off the wall clock, because a
loaded machine steals wall seconds without adding any cycles and two rows
failed at 2.24s / 3.66s against a 2.0s bound while census had the CPU. The
remaining ten bounds in five other files still ran on `time.monotonic()` and
carried the same defect. They now share ONE clock.
The clock is IMPORTED, not copied: `tests/redos_clock.py`. Five private copies
would leave four of them unpinned -- the instrument test
(test_the_redos_clock_ignores_time_this_process_did_not_spend) can only pin the
implementation it calls, and the suite already holds that rule for the code it
measures.
Every ported row was verified the only way a time bound can be: the vulnerable
form patched back in, red demanded, `git checkout --` after. Measured against
the 2.0s bound (3.0s for the url arm):
active_content long-attr `{0,63}` -> `*` RED
neutralize long-attr same patch RED
output gate long-attr same patch 12.41s
okf link graph `[^\]\[]` -> `[^\]]` 6.91s
sanitize comment str.find -> `<!--.*?-->` 17.56s
lexicon md-link-anchor-text 319.14s
lexicon md-link-anchor-url 8.55s
lexicon md-link-ref-comment 37.82s
Two rows did not go red, for two different reasons.
test_sanitize.py::test_legitimate_comment_heavy_document is the legitimate SIDE
of a separation, not a second pin on the defect: closed comments never withhold
the required literal, so the lazy form runs it in 0.016s. Recorded in place.
test_lexicon.py::test_redos_pathological_subagent_input_returns_fast is DEAD --
the same zero-signal shape the `<a ` carrier had, found by the same method. The
seed form is `(?:.*?\s+)?` (llm-security 7.8.0, injection-patterns.mjs:84) and
this repo has never carried it: the bounded `{0,12}?` port is in the pattern
table's first commit. Patched in by hand at the row's own size: shipped 0.135s
vs seed 0.113s, separation 1.2x. Not the keyword gate either -- a variant that
reaches the inner branch stays linear over four doublings (exponent ~1.0),
because the nesting is one lazy run inside an OPTIONAL group, never a repeated
one. Left standing with the measurement written into it; picking a new carrier
is an operator call, like the wall-clock row above it.
The dead sibling row named in STATE is fixed: test_active_content.py's
long-attribute row swaps carrier `<a ` -> `<script `, for the reason `5667063`
established on its composed-gate twin -- 0.7.0's own no-URL narrowing put `<a>`
in `_URL_AFFORDANCE_TAGS`, so the tag returns inert BEFORE its body reaches the
arm the row guards. Re-measured here, not inherited: `<a ` 0.041s and NO
findings against the vulnerable form; `<script ` 19.349s against 0.052s
shipped, 373x apart.
`test_pathological_input_returns_within_a_bound` deliberately keeps its wall
clock (operator decision): it claims to catch a hang, and only a wall clock
catches one.
792 tests, 129/129, 6/6.
`_ZERO_WIDTH` (sanitize, input) and `_ZERO_WIDTH_CPS` (output,
`_scan_invisible_carriers`) tested U+200D on codepoint membership alone.
`disposition._CARRIER_LABELS` grades both as any-tier FAIL_SECURE with no
appeal, so any first-party document containing a ZWJ-composed emoji --
professions, families, skin tones, flag variants -- was hard-blocked forever.
Reported by ms-ai-architect; confirmed here against the code.
The strip was the worse half and was not in the report: sanitize *removed* the
joiner, silently decomposing the emoji into two unrelated ones. A module whose
contract is "only ever removes carriers" was corrupting content.
The fix is the one our own lexicon row `unicode:zero-width-in-word` (`\w[ZW]\w`)
already used: judge the joiner by CONTEXT, not identity. A ZWJ is exempt only
when BOTH neighbours are emoji-context codepoints. Half-context is not context,
so `a<ZWJ>{emoji}` stays a carrier and an attacker cannot buy exemption with a
single emoji.
Blocks, not an emoji table. Measured against Unicode 17.0's
`emoji-zwj-sequences.txt`: 1614 RGI sequences use 122 distinct codepoints
adjacent to a ZWJ, and the five ranges cover 122/122. The measurement earned
its keep -- the hand-reasoned candidate table missed U+2194, U+2195 and U+2B1B.
Shipping the RGI list itself would be exact on the day it landed and stale at
the next Unicode release, reopening this same false positive for every new
emoji; whole blocks carry the unassigned headroom (458 Cn codepoints) that
future emoji are allocated into, so the table does not age.
The predicate is defined once in sanitize and imported by output. A second copy
is how the input side stops flagging while the output side keeps blocking; the
cross-surface test asserts the two agree on six inputs.
Two residuals, both in LIMITATIONS (33 -> 34, README bumped): a ZWJ between two
emoji is now exempt and could carry a covert channel (one emoji per bit, cannot
split a word); and U+200C (ZWNJ) still has no context test, so Persian, Arabic
and Devanagari documents -- where it is orthographically required -- stay
blocked. That needs a script-based criterion and no corpus is here to verify it
against, so it is parked as a known FP class rather than guessed at.
736 green (was 727), coverage 128/128, 6/6 documented gaps hold.
The generalised sweep found what 0.3.2's hand-written rows missed. All three are
the documented class -- a run in front of a required literal that never arrives,
so every start position rescans the tail -- and all three are worse than the
0.3.3 findings, because `sanitize`, `neutralize`, `scan_active_content` and the
okf link graph apply NO input cap. `scan_lexicon`/`scan_output` are the only
entry points that do, so there is no ceiling to extrapolate to.
sanitize._HTML_COMMENT_RE `<!--`*100_000 20.1s, exponent 1.96-2.14
active_content.URL_IN_TEXT_RE `<a `+`A`*100_000 12.99s / 14.9s, exponent ~2.0
okf._MD_LINK_RE `[`*100_000 7.1s, exponent 1.99-2.05
Each fix is the one the pattern's own shape allows, not a copied choice:
- The comment stripper drops the regex for `str.find`. Excluding `<` would lose
every comment containing markup; bounding the run would be a carrier bypass
of the exact construct the stripper exists to remove.
- `URL_IN_TEXT_RE` bounds its scheme run to an RFC 3986 scheme (`{0,63}`).
Bounding is safe *here* only because it is a defanger inside a tag already
flagged `active:raw-html`. A lookbehind was measured too and rejected: it
drops `-http://evil.com`, a one-character evasion. Bounded: 0.185s at 1M.
- `_MD_LINK_RE` excludes `[`, matching `active_content.MD_LINK_RE` exactly,
including the nested-label trade already documented there.
`sanitize` claimed "no catastrophic backtracking" in a comment; that claim was
wrong in the same way `output`'s was before 0.3.2, and is corrected in place.
676 tests (+10), coverage 128/128 + 6/6 gaps, sweep clean across 150 patterns.
The okf destination run gets no row: `[^)\s]+` cannot fail, so a row for it
could never go red.
