docs(redos): the sweep now measures on the clock its own numbers justify
docs/redos-sweep.py timed on time.monotonic() while every ReDoS bound in the suite had moved to process CPU time, so the 1.5 ms sensitivity floor and the "~23 s at the cap" figure in docs/LIMITATIONS.md were produced by a different instrument than the bounds they support. The script imports scan_seconds now. Re-derived on that instrument, the floor came back UNCHANGED at 1.5 ms. Twelve full runs of all 2585 arms: median ratio 1.95-2.03 in every size bucket above 50 us, but two-point excursions past the 2.6 threshold survive at every magnitude (p99 2.9-3.3 even above 1 ms). Flagged arms per run by floor: 6.9 at 0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms. Four arms flagged across the twelve runs, each in exactly one; six arms that have ever flagged re-measure at exponent 0.97-1.09 over six doublings, at most 1.2 s at the cap. Descheduling was never what made this sweep noisy -- a ratio from two points is. Measured before publishing and it cost a correction: an earlier draft of this change said the sweep "reports 0 candidates". The next run reported 2. Nine of twelve instrumented runs are clean and eight consecutive shipped-script runs under load flagged 0-3, so neither a clean run nor a flagged one is evidence on its own -- the doc says that now. Also corrects the pattern count in the same bullet: 150 -> 152, the script's own printed total. The 0.3.4 CHANGELOG entry keeps its 150 as a snapshot. No exported surface, no detector behaviour, no calibration touched. 802 passed, 129/129 classes, 6/6 gaps, 43 limitations, gitleaks clean.
This commit is contained in:
parent
4472f209a4
commit
74656123f9
3 changed files with 90 additions and 26 deletions
|
|
@ -37,17 +37,30 @@ import importlib
|
|||
import json
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
SRC = Path(__file__).resolve().parent.parent / "src" / "llm_ingestion_guard"
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SRC = ROOT / "src" / "llm_ingestion_guard"
|
||||
sys.path.insert(0, str(SRC.parent))
|
||||
sys.path.insert(0, str(ROOT / "tests"))
|
||||
|
||||
from llm_ingestion_guard.lexicon import load_lexicon # noqa: E402
|
||||
from redos_clock import scan_seconds # noqa: E402
|
||||
|
||||
N1, N2 = 4_000, 8_000
|
||||
RATIO_FLAG = 2.6
|
||||
# RE-DERIVED on the CPU clock, not inherited from the wall clock this script used
|
||||
# through 1.1.0. The clock move fixed false REDS in the suite's bounds; it bought
|
||||
# this sweep no sensitivity. Over twelve full runs (2585 arms each) the median
|
||||
# ratio is 1.95-2.03 in every size bucket above 50 us -- the whole surface
|
||||
# measures linear -- yet two-point excursions past RATIO_FLAG survive at every
|
||||
# magnitude (p99 ratio 2.9-3.3 even above 1 ms). Flagged arms per run by floor:
|
||||
# 6.9 at 0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms. The knee is here. Four arms
|
||||
# flagged across those twelve runs, each in exactly ONE of them, and six arms
|
||||
# that have ever flagged re-measure at exponent 0.97-1.09 over six doublings.
|
||||
# Descheduling was never what made this sweep noisy -- a two-point ratio is, so
|
||||
# read a clean run and a flagged run with the same suspicion.
|
||||
NOISE_FLOOR = 0.0015
|
||||
HARD_CAP = 20.0
|
||||
|
||||
|
|
@ -134,21 +147,35 @@ def build(unit: str, n: int) -> str:
|
|||
|
||||
|
||||
def t(rx: re.Pattern[str], text: str, mode: str = "search") -> float:
|
||||
"""Time one scan of ``text`` in the mode the production code actually uses."""
|
||||
"""Time one scan of ``text`` in the mode the production code actually uses.
|
||||
|
||||
On the SAME clock every ReDoS bound in the suite is measured against --
|
||||
``tests/redos_clock.py``, process CPU time -- imported rather than restated
|
||||
here, for the reason that module gives: a blowup is spent cycles, and a
|
||||
loaded machine steals wall clock without adding any. Until 1.1.0 this timed
|
||||
on ``time.monotonic()``, which made the floor below and the cap figure
|
||||
derived from it numbers from a different instrument than the bounds they
|
||||
justify. The closure is built BEFORE the clock starts, so only the scan is
|
||||
charged.
|
||||
"""
|
||||
mode = mode.rstrip("*")
|
||||
start = time.monotonic()
|
||||
if mode == "finditer":
|
||||
for _ in rx.finditer(text):
|
||||
pass
|
||||
def scan(s: str) -> None:
|
||||
for _ in rx.finditer(s):
|
||||
pass
|
||||
elif mode == "sub":
|
||||
rx.sub("", text)
|
||||
def scan(s: str) -> None:
|
||||
rx.sub("", s)
|
||||
elif mode == "match":
|
||||
rx.match(text)
|
||||
def scan(s: str) -> None:
|
||||
rx.match(s)
|
||||
elif mode == "fullmatch":
|
||||
rx.fullmatch(text)
|
||||
def scan(s: str) -> None:
|
||||
rx.fullmatch(s)
|
||||
else:
|
||||
rx.search(text)
|
||||
return time.monotonic() - start
|
||||
def scan(s: str) -> None:
|
||||
rx.search(s)
|
||||
return scan_seconds(scan, text)
|
||||
|
||||
|
||||
# --- targets ----------------------------------------------------------------
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue