open/llm-ingestion-pipeline-security
1
0
Fork 0
Code Issues Releases 3 Activity

test(output): the ReDoS bounds move to CPU time, and the long-attribute row was dead

Browse source 
Two rows failed at 2.24s against a 2.0s wall-clock bound while two census
processes had the CPU, and passed 3/3 on an idle machine. Reproduced under
artificial oversubscription before touching anything (16 logical / 8 physical
cores), `lexicon-script-tag` shipped form, idle vs 2x vs 4x:

    wall  0.74s -> 3.55s -> 8.13s   (11x, still climbing with load)
    cpu   0.74s -> 1.42s -> 1.50s   (2.0x, flat from 2x to 4x)

Wall-clock inflation tracks how many other processes want the CPU and has no
ceiling; process CPU inflation is bounded by SMT and memory contention. The
bound itself is UNCHANGED at 2.0s: on an idle machine the two clocks are the
same number (measured ratio 1.00), so every figure in the derivation comment
stays true as a CPU-time figure. Raising the bound instead was rejected by
measurement -- the vulnerable `[^>]` script-tag form runs 4.0s idle, so any
wall bound loose enough to survive load would let the defect pass.

Signal verified by patching the pre-fix forms back in: both rows go red
(4.49s and 18.85s against 2.0s) and green again with the shipped forms.

That verification found the second, worse defect. 0.7.0's own no-URL narrowing
killed `test_gate_is_bounded_on_the_long_attribute_arm`: `<a>` is in
`_URL_AFFORDANCE_TAGS`, so the bare `<a ` + 100k + `>` payload is now inert and
returns BEFORE the body reaches `URL_IN_TEXT_RE` -- the arm the row exists to
guard. Measured against the vulnerable form through `scan_active_content`:

    <a ...>       0.028s and NO findings   <- dead: separation 1.0x
    <script ...> 12.475s
    <a href=x …> 12.719s
    <form ...>   17.092s

The carrier is now `<script `: active by NAME with no attributes, so no future
URL-shaped narrowing can make it inert the same way. 0.53s shipped vs 18.85s
vulnerable through `scan_output` -- 35x apart, bound 3.8x above the shipped side.

`test_pathological_input_returns_within_a_bound` deliberately keeps its wall
clock: it claims to catch "a hang or a blowup", and only a wall clock catches
the first. New instrument test pins the clock via the same helper the bounds
use, so the choice cannot drift silently.

792 tests, 129/129 coverage, 6/6 gaps. The ReDoS block passes 3/3 under the
4x oversubscription that produced 8.13s wall.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 21:02:31 +02:00
commit 566706360a
1 changed files with 77 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

90
tests/test_output.py
View file

@ -351,6 +351,25 @@ def test_pathological_input_returns_within_a_bound():
# --- crafted ReDoS payloads against OUR OWN patterns (OWASP LLM10) -----------
#
# Every bound below goes through `_scan_seconds`, so the rows share ONE clock
# and one derivation. The neighbouring test above keeps its own wall clock on
# purpose -- see the instrument test for why the two must not be merged.
def _scan_seconds(scanner, payload) -> float:
"""CPU seconds a scan cost -- the clock the ReDoS bounds are derived against.
Process CPU time, not wall clock: a ReDoS blowup is spent cycles, and a
loaded machine steals wall clock without adding any. Pinned by
``test_the_redos_clock_ignores_time_this_process_did_not_spend``, which
carries the measurements and what this clock gives up.
"""
start = time.process_time()
scanner(payload)
return time.process_time() - start
# The gap the test above explicitly does NOT cover. Every pattern here has the
# same shape: a `+`/`*` run followed by a REQUIRED literal, reachable from a
# short anchor. The payload repeats that anchor and never supplies the literal,
@ -422,9 +441,41 @@ _REDOS_PAYLOADS = [
)
def test_crafted_redos_payload_stays_bounded(scanner, unit, n):
payload = (unit * (n // len(unit) + 1))[:n]
start = time.monotonic()
scanner(payload)
assert time.monotonic() - start < 2.0
assert _scan_seconds(scanner, payload) < 2.0
def test_the_redos_clock_ignores_time_this_process_did_not_spend():
# The instrument the bounds above are measured on, pinned -- because getting
# it wrong makes a GREEN suite look red. In 0.7.0 these bounds ran on
# `time.monotonic()`, and two rows failed at 2.24s / 3.66s against the 2.0s
# bound while two census processes had the CPU; the same rows passed 3/3 on
# an idle machine. The scans had not slowed down -- they were descheduled.
#
# Measured on this machine (16 logical / 8 physical cores), `lexicon-script-tag`,
# shipped form, idle vs 2x vs 4x oversubscription:
#
# wall 0.74s -> 3.55s -> 8.13s (11x, still climbing with load)
# cpu 0.74s -> 1.42s -> 1.50s (2.0x, flat from 2x to 4x)
#
# Wall-clock inflation is proportional to how many other processes want the
# CPU and has no ceiling. Process CPU inflation is bounded by SMT and memory
# contention -- a sibling hyperthread can cost you roughly 2x and nothing
# beyond it, which is why the two right-hand columns barely differ. On an
# idle machine the two clocks are the same number (measured ratio 1.00), so
# switching instrument re-derives NOTHING above: every figure in the bound
# derivation stays true as a CPU-time figure.
#
# What this clock gives up: a scan that BLOCKS forever burns no CPU, so it
# would hang the suite instead of failing it. Acceptable here -- these
# scanners are pure regex over an in-memory string, with no I/O and no locks,
# so the only way they can be slow is by spending cycles. It is also why
# `test_pathological_input_returns_within_a_bound` above keeps a wall clock:
# that test claims to catch "a hang or a blowup", and only a wall clock
# catches the first.
#
# A sleep is the defect class at its purest: wall-clock seconds this process
# did not spend. 0.4s is 4x the assertion, so this cannot pass by timing luck.
assert _scan_seconds(lambda _: time.sleep(0.4), "") < 0.1
def test_crafted_redos_payload_bounded_through_the_public_gate():
@ -433,9 +484,7 @@ def test_crafted_redos_payload_bounded_through_the_public_gate():
# invokes is bounded too -- with the worst measured payload (`<a:`, 660x the
# slowest legitimate content of the same size).
payload = ("<a:" * (_REDOS_N // 3 + 1))[:_REDOS_N]
start = time.monotonic()
scan_output(payload)
assert time.monotonic() - start < 2.0
assert _scan_seconds(scan_output, payload) < 2.0
def test_gate_is_bounded_on_the_payload_the_first_sweep_missed():
@ -447,9 +496,7 @@ def test_gate_is_bounded_on_the_payload_the_first_sweep_missed():
# test_lexicon.py::test_crafted_redos_payload_stays_bounded_in_the_lexicon;
# this row exists so the composed gate a caller actually invokes is covered.
payload = "[" * _REDOS_N
start = time.monotonic()
scan_output(payload)
assert time.monotonic() - start < 2.0
assert _scan_seconds(scan_output, payload) < 2.0
def test_gate_is_bounded_on_the_long_attribute_arm():
@ -458,10 +505,27 @@ def test_gate_is_bounded_on_the_long_attribute_arm():
# caller actually invokes inherits it. Not expressible as a repeating unit —
# the tag has to CLOSE for the body to be handed on — which is exactly why
# the unit-table above never covered it.
payload = "<a " + "A" * _REDOS_N + ">"
start = time.monotonic()
scan_output(payload)
assert time.monotonic() - start < 2.0
#
# The carrier is `<script `, not the `<a ` this row shipped with through
# 0.7.0, because 0.7.0's own no-URL narrowing killed the row: `<a>` is in
# `_URL_AFFORDANCE_TAGS`, so a bare `<a ...>` carrying no URL attribute is
# now inert and returns BEFORE the body reaches `URL_IN_TEXT_RE` — the arm
# this row exists to guard. Measured with the pre-fix uncapped scheme run
# patched back in, at _REDOS_N through `scan_active_content`:
#
# <a ...> 0.028s and NO findings <- dead: never reaches the arm
# <script ...> 12.475s <- the arm, still quadratic
# <a href=x …> 12.719s
# <form ...> 17.092s
#
# So the row was green against the vulnerable form: separation 1.0x, zero
# signal. With `<script ` it is 0.53s shipped vs 18.85s vulnerable through
# `scan_output` — 35x apart, with the bound 3.8x above the shipped side.
# `<script>` is the durable choice of the three: it is active by NAME with no
# attributes at all, so no future URL-shaped narrowing can make it inert the
# way it just did to `<a >`.
payload = "<script " + "A" * _REDOS_N + ">"
assert _scan_seconds(scan_output, payload) < 2.0
# --- ZWJ inside emoji sequences on the output gate ---------------------------