1
0
Fork 0

test(output): the "pathological" DoS payload is not pathological -- bound was 6% over

Measured this session, the reason the bound was flaky at all:

  size-matched ordinary prose  3.50-4.20s  (cold up to ~4.2s)
  the "pathological" blob      3.35-3.86s warm, 4.70s cold
  ratio patho/ordinary         0.93x, and 0.96x with the order swapped

The blob is the FASTER side. The test therefore never proved what its comment
claimed -- it measured throughput on 1MB of text, not catastrophic backtracking,
and could only fail when the process was cold or the machine loaded. That is
exactly how it failed: 4.39 / 4.46 / 4.92s against a 5.0s bound.

Catastrophic backtracking IS covered, elsewhere and properly, by
test_lexicon.py::test_redos_pathological_subagent_input_returns_fast, which
crafts against a known-bad nested `.*?` pattern (0.137s against a 2.0s bound --
14x headroom, measured, left untouched).

So: raise the bound rather than make it relative. A relative bound was the other
option on the table and the measurement killed it -- with a ratio below 1.0 you
would need k >= 3 to clear the noise, and an assertion with 3x headroom over a
case already under 1.0 can never fire. Vacuous, plus it would add a second
cold-start asymmetry and double the wall clock.

10.0s is ~2.4x the slowest observed legitimate run (ordinary prose, cold). The
comment now carries the measurements, says the name overstates the payload, and
warns that the 1_000_200-char size is deliberate -- 200 over the max_scan_chars
default, so the test also exercises the truncate-and-flag oversize path.

No production code changed. 642 passed.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 17:18:15 +02:00
commit f74245f01f

View file

@ -327,8 +327,20 @@ def test_no_double_oversize_flag_from_lexicon():
def test_pathological_input_returns_within_a_bound():
# A scanner that hangs on crafted input IS the DoS. Bound the runtime.
# A scanner that hangs on crafted input IS the DoS. This bounds the runtime
# so a hang or a blowup fails loudly; it is NOT a throughput regression test.
# The name overstates what the payload proves: measured against size-matched
# ordinary prose this blob is the FASTER side (0.93x / 0.96x, order swapped),
# so it does not exercise catastrophic backtracking. That duty is carried by
# test_lexicon.py::test_redos_pathological_subagent_input_returns_fast, which
# crafts against a known-bad nested `.*?` pattern.
# Bound set from measurement, not preference: the slowest legitimate run of
# this size is ordinary prose on a cold process (~4.2s); the blob itself runs
# 4.70s cold / 3.40-3.73s warm. The old 5.0s sat ~6% over that and failed on
# a loaded machine. 10.0s is ~2.4x the slowest observed legitimate run.
# The payload is 1_000_200 chars -- 200 over the max_scan_chars default, so
# this also exercises the truncate-and-flag oversize path. Do not resize it.
payload = ("A" * 5000 + " ") * 200 # ~1MB of blob-ish text
start = time.monotonic()
scan_output(payload)
assert time.monotonic() - start < 5.0
assert time.monotonic() - start < 10.0