test(redos): the dead row had the wrong payload, and the last wall clock is retired
Both rows that could not go red are decided, each by measurement.
test_lexicon.py::test_redos_pathological_subagent_input_returns_fast is REVIVED,
not retired. The row was not dead because the seed form is safe -- it was dead
because both earlier payloads made the prefix match at ONE start position, and
the cost is per-prefix-match. Repeating `spawn an agent that ` instead makes it
match K times, each driving its own O(N) lazy scan for a keyword never supplied:
K x O(N) against the seed's `(?:.*?\s+)?`, K x O(1) against the shipped
`{0,12}?` bound. Measured through scan_lexicon at 1500/3000/6000/12000 words:
seed 0.091/0.283/1.085/4.091s (exponent 1.92), shipped 0.047/0.051/0.094/0.190s
(exponent 1.01). Verified red with the seed form patched in: 4.21s against the
2.0s bound. The nesting the old comment blamed was a red herring.
test_output.py::test_pathological_input_returns_within_a_bound moves to CPU time
with a 20.0s bound, and the "or a hang" half of its claim is retired. The wall
clock was kept because a blocking hang burns no CPU -- true in general, and
inapplicable to a path with no open(), socket, subprocess, thread, lock or sleep
anywhere on it. Same payload, idle vs ~4x oversubscription: wall 3.30 -> 21.63s
(2x over the old 10.0s bound), cpu 3.30 -> 7.62s. It guarded a mode it could not
have while paying the full false-red premium. No in-repo vulnerable form can
turn this row red, so the bound was proved live against what it actually guards
-- a future pattern quadratic on long runs, `A+\s*EXFILTRATE` -- which failed it
at 64.77s CPU, 3.2x over.
redos_clock.py and the clock's pin test both documented this row as the
deliberate wall-clock holdout; both corrected.
792 passed, 6/6 documented gaps hold.
This commit is contained in:
parent
c48a2923ac
commit
2466d260d3
3 changed files with 78 additions and 47 deletions
|
|
@ -228,34 +228,34 @@ def test_oversize_input_is_capped_and_flagged():
|
|||
|
||||
|
||||
def test_redos_pathological_subagent_input_returns_fast():
|
||||
# A crafted string that would force catastrophic backtracking on the
|
||||
# ORIGINAL nested-`.*?` sub-agent pattern. The bounded port stays linear.
|
||||
# The seed's `(?:.*?\s+)?` is quadratic on this payload; the bounded
|
||||
# `{0,12}?` port that shipped instead is linear. Seed form: llm-security
|
||||
# 7.8.0, scanners/lib/injection-patterns.mjs:84 — this repo has never
|
||||
# carried it (the bound is in the pattern table's FIRST commit, f397cd9),
|
||||
# so the vulnerable form is patched in by hand, never reverted to.
|
||||
#
|
||||
# MEASURED DEAD, and left standing pending an operator decision — the same
|
||||
# zero-signal shape the `<a ` carrier had in test_active_content.py, found by
|
||||
# the same method (patch the vulnerable form back in and demand red). The
|
||||
# seed's actual form is `(?:.*?\s+)?` (llm-security 7.8.0,
|
||||
# scanners/lib/injection-patterns.mjs:84); this repo has never carried it —
|
||||
# the bounded `{0,12}?` port is in the pattern table's FIRST commit (f397cd9),
|
||||
# so there is no in-repo form to revert to. Patched in by hand, at the row's
|
||||
# own 8000-word size:
|
||||
# WHAT THE PAYLOAD HAS TO DO, because two earlier shapes did neither and
|
||||
# this row sat measured-dead (1.2x) until it was found: the cost is
|
||||
# per-PREFIX-MATCH, so the payload must make the prefix match at MANY start
|
||||
# positions, not at one. `spawn an agent that ` REPEATED does that; the
|
||||
# earlier `spawn an agent that ` + filler matched the prefix once and paid
|
||||
# one lazy run, which is linear no matter how long the filler is. The
|
||||
# nesting the old comment blamed is a red herring — the inner `.*?` sits in
|
||||
# an OPTIONAL group, never a repeated one. What costs is that each of the
|
||||
# K prefix matches drives its own O(N) lazy scan to end-of-string looking
|
||||
# for a capability keyword the payload never supplies: K x O(N) = O(N^2).
|
||||
# The bound caps each scan at 12 tokens, so K x O(1) = O(N).
|
||||
#
|
||||
# shipped 0.135s seed form 0.113s <- separation 1.2x, no signal
|
||||
# Measured through `scan_lexicon` at be9759b+, seed form patched in:
|
||||
#
|
||||
# Not a payload-size problem and not the keyword gate either: the payload
|
||||
# never supplies the trailing keyword the outer alternation requires, and a
|
||||
# variant that DOES reach the inner branch (`...that reads ` + the same
|
||||
# padding) stays linear too — 0.026 / 0.029 / 0.060 / 0.129s over four
|
||||
# doublings, exponent ~1.0. The nesting the comment names is one lazy run
|
||||
# inside an OPTIONAL group, never inside a repeated one, so there is no
|
||||
# per-start rescan for the payload to pay for.
|
||||
# words 1500 3000 6000 12000
|
||||
# seed 0.091s 0.283s 1.085s 4.091s <- exponent 1.92
|
||||
# shipped 0.047s 0.051s 0.094s 0.190s <- exponent 1.01
|
||||
#
|
||||
# Reviving it needs a payload shape that makes the seed form actually blow
|
||||
# up; two shapes were tried and neither did. Until then this row proves the
|
||||
# scanner runs, not that the port is bounded. Deliberately NOT redesigned
|
||||
# here: choosing a new carrier is the same call the operator reserved for the
|
||||
# `test_pathological_input_returns_within_a_bound` row.
|
||||
evil = "spawn an agent that " + ("word " * 8000)
|
||||
# At the 12000 words this row carries: 4.091s vs 0.190s = 22x, and the seed
|
||||
# form breaks the 2.0s bound outright — the row failed at 4.21s with it
|
||||
# patched in. Verified red, not assumed.
|
||||
evil = "spawn an agent that " * 3000
|
||||
assert scan_seconds(scan_lexicon, evil) < 2.0
|
||||
assert isinstance(scan_lexicon(evil), Report)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue