feat(sanitize,fence,neutralize): reject oversize input instead of half-transforming it
The scanners cap by truncating: they return findings, so reading a prefix costs detection in the tail and nothing else. The three transform surfaces return *content*, where the same move is not available — a shortened document is silent data loss, and a transformed prefix followed by an untransformed tail is a bypass, since the attacker chooses where in the document the payload sits. So they fail secure instead. Above MAX_INPUT_CHARS (1 000 000) sanitize, fence and neutralize raise OversizeInputError. sanitize is step 1 of prepare_input and only ever removes, so that one refusal bounds the whole input path. OversizeInputError subclasses ContractViolation: a pipeline already bracketing its quarantined stage keeps failing closed rather than meeting a type it has never heard of. It inherits the alert-routable property too — sizes in the message, refusing surface in details, no input in either. Invariant now pinned across all three: returned text is always fully transformed, or not returned at all. Still uncapped and recorded in LIMITATIONS: scan_active_content called directly (through scan_output it inherits that cap) and the okf link graph. Both are detection-shaped, so truncate-and-flag transfers unchanged — mechanical, not policy. 699 tests (+23), coverage 128/128 + 6/6, ReDoS sweep 0 candidates / 150.
This commit is contained in:
parent
adf93e47fb
commit
2d98d6809d
10 changed files with 272 additions and 21 deletions
|
|
@ -302,19 +302,20 @@ items; this is the full list, each with the mechanism.
|
|||
*closes* around a long body, and a run of plain characters carrying no anchor
|
||||
at all.
|
||||
|
||||
- **Only `scan_lexicon` and `scan_output` cap their input; the input-side entry
|
||||
points do not.** `MAX_SCAN_CHARS` (1 000 000) is applied in those two functions
|
||||
only. `sanitize`, `fence`, `neutralize`, `scan_active_content` and the okf link
|
||||
graph accept text of any length, so their cost is bounded by the caller's
|
||||
input, not by this library. Every *known* quadratic run on those paths is
|
||||
fixed, and the residual above states what the sweep can and cannot claim — but
|
||||
where an output-path residual is capped at ~23 s, the same residual on the
|
||||
input path has no ceiling. A caller that ingests untrusted documents of
|
||||
unbounded size should impose its own limit before `prepare_input`. Extending
|
||||
the cap into the input path is deliberately **not** done as part of a ReDoS
|
||||
patch: it changes the contract for existing callers (what happens to the
|
||||
truncated remainder is a policy question), and that deserves its own decision
|
||||
rather than being smuggled in.
|
||||
- **Two detection surfaces still accept unbounded input; the transform surfaces
|
||||
no longer do.** Since 0.3.5 `sanitize`, `fence` and `neutralize` raise
|
||||
`OversizeInputError` above `MAX_INPUT_CHARS` (1 000 000) rather than returning
|
||||
a partially transformed document, which bounds the whole input path — `sanitize`
|
||||
is step 1 of `prepare_input`, and it only ever removes, so everything after it
|
||||
is already under the cap. They reject rather than truncate because they return
|
||||
*content*: a shortened document is silent data loss, and a transformed prefix
|
||||
followed by an untransformed tail is a bypass an attacker positions the payload
|
||||
into. The scanners keep truncating, which costs only detection in the tail.
|
||||
What remains uncapped is `scan_active_content` **called directly** (reached
|
||||
through `scan_output` it inherits that cap) and the okf link graph, whose cost
|
||||
is a bundle-wide `findall` over every document body. Both are detection-shaped,
|
||||
so the scanners' truncate-and-flag mechanism transfers to them unchanged — that
|
||||
is a mechanical follow-up, not a policy question, and it is not yet done.
|
||||
|
||||
## The six documented gaps (tracked by the coverage matrix)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue