1
0
Fork 0

test(redos): the dead row had the wrong payload, and the last wall clock is retired

Both rows that could not go red are decided, each by measurement.

test_lexicon.py::test_redos_pathological_subagent_input_returns_fast is REVIVED,
not retired. The row was not dead because the seed form is safe -- it was dead
because both earlier payloads made the prefix match at ONE start position, and
the cost is per-prefix-match. Repeating `spawn an agent that ` instead makes it
match K times, each driving its own O(N) lazy scan for a keyword never supplied:
K x O(N) against the seed's `(?:.*?\s+)?`, K x O(1) against the shipped
`{0,12}?` bound. Measured through scan_lexicon at 1500/3000/6000/12000 words:
seed 0.091/0.283/1.085/4.091s (exponent 1.92), shipped 0.047/0.051/0.094/0.190s
(exponent 1.01). Verified red with the seed form patched in: 4.21s against the
2.0s bound. The nesting the old comment blamed was a red herring.

test_output.py::test_pathological_input_returns_within_a_bound moves to CPU time
with a 20.0s bound, and the "or a hang" half of its claim is retired. The wall
clock was kept because a blocking hang burns no CPU -- true in general, and
inapplicable to a path with no open(), socket, subprocess, thread, lock or sleep
anywhere on it. Same payload, idle vs ~4x oversubscription: wall 3.30 -> 21.63s
(2x over the old 10.0s bound), cpu 3.30 -> 7.62s. It guarded a mode it could not
have while paying the full false-red premium. No in-repo vulnerable form can
turn this row red, so the bound was proved live against what it actually guards
-- a future pattern quadratic on long runs, `A+\s*EXFILTRATE` -- which failed it
at 64.77s CPU, 3.2x over.

redos_clock.py and the clock's pin test both documented this row as the
deliberate wall-clock holdout; both corrected.

792 passed, 6/6 documented gaps hold.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 21:47:43 +02:00
commit 2466d260d3
3 changed files with 78 additions and 47 deletions

View file

@ -228,34 +228,34 @@ def test_oversize_input_is_capped_and_flagged():
def test_redos_pathological_subagent_input_returns_fast():
# A crafted string that would force catastrophic backtracking on the
# ORIGINAL nested-`.*?` sub-agent pattern. The bounded port stays linear.
# The seed's `(?:.*?\s+)?` is quadratic on this payload; the bounded
# `{0,12}?` port that shipped instead is linear. Seed form: llm-security
# 7.8.0, scanners/lib/injection-patterns.mjs:84 — this repo has never
# carried it (the bound is in the pattern table's FIRST commit, f397cd9),
# so the vulnerable form is patched in by hand, never reverted to.
#
# MEASURED DEAD, and left standing pending an operator decision — the same
# zero-signal shape the `<a ` carrier had in test_active_content.py, found by
# the same method (patch the vulnerable form back in and demand red). The
# seed's actual form is `(?:.*?\s+)?` (llm-security 7.8.0,
# scanners/lib/injection-patterns.mjs:84); this repo has never carried it —
# the bounded `{0,12}?` port is in the pattern table's FIRST commit (f397cd9),
# so there is no in-repo form to revert to. Patched in by hand, at the row's
# own 8000-word size:
# WHAT THE PAYLOAD HAS TO DO, because two earlier shapes did neither and
# this row sat measured-dead (1.2x) until it was found: the cost is
# per-PREFIX-MATCH, so the payload must make the prefix match at MANY start
# positions, not at one. `spawn an agent that ` REPEATED does that; the
# earlier `spawn an agent that ` + filler matched the prefix once and paid
# one lazy run, which is linear no matter how long the filler is. The
# nesting the old comment blamed is a red herring — the inner `.*?` sits in
# an OPTIONAL group, never a repeated one. What costs is that each of the
# K prefix matches drives its own O(N) lazy scan to end-of-string looking
# for a capability keyword the payload never supplies: K x O(N) = O(N^2).
# The bound caps each scan at 12 tokens, so K x O(1) = O(N).
#
# shipped 0.135s seed form 0.113s <- separation 1.2x, no signal
# Measured through `scan_lexicon` at be9759b+, seed form patched in:
#
# Not a payload-size problem and not the keyword gate either: the payload
# never supplies the trailing keyword the outer alternation requires, and a
# variant that DOES reach the inner branch (`...that reads ` + the same
# padding) stays linear too — 0.026 / 0.029 / 0.060 / 0.129s over four
# doublings, exponent ~1.0. The nesting the comment names is one lazy run
# inside an OPTIONAL group, never inside a repeated one, so there is no
# per-start rescan for the payload to pay for.
# words 1500 3000 6000 12000
# seed 0.091s 0.283s 1.085s 4.091s <- exponent 1.92
# shipped 0.047s 0.051s 0.094s 0.190s <- exponent 1.01
#
# Reviving it needs a payload shape that makes the seed form actually blow
# up; two shapes were tried and neither did. Until then this row proves the
# scanner runs, not that the port is bounded. Deliberately NOT redesigned
# here: choosing a new carrier is the same call the operator reserved for the
# `test_pathological_input_returns_within_a_bound` row.
evil = "spawn an agent that " + ("word " * 8000)
# At the 12000 words this row carries: 4.091s vs 0.190s = 22x, and the seed
# form breaks the 2.0s bound outright — the row failed at 4.21s with it
# patched in. Verified red, not assumed.
evil = "spawn an agent that " * 3000
assert scan_seconds(scan_lexicon, evil) < 2.0
assert isinstance(scan_lexicon(evil), Report)