1
0
Fork 0

feat(sanitize,fence,neutralize): reject oversize input instead of half-transforming it

The scanners cap by truncating: they return findings, so reading a prefix costs
detection in the tail and nothing else. The three transform surfaces return
*content*, where the same move is not available — a shortened document is silent
data loss, and a transformed prefix followed by an untransformed tail is a
bypass, since the attacker chooses where in the document the payload sits.

So they fail secure instead. Above MAX_INPUT_CHARS (1 000 000) sanitize, fence
and neutralize raise OversizeInputError. sanitize is step 1 of prepare_input and
only ever removes, so that one refusal bounds the whole input path.

OversizeInputError subclasses ContractViolation: a pipeline already bracketing
its quarantined stage keeps failing closed rather than meeting a type it has
never heard of. It inherits the alert-routable property too — sizes in the
message, refusing surface in details, no input in either.

Invariant now pinned across all three: returned text is always fully
transformed, or not returned at all.

Still uncapped and recorded in LIMITATIONS: scan_active_content called directly
(through scan_output it inherits that cap) and the okf link graph. Both are
detection-shaped, so truncate-and-flag transfers unchanged — mechanical, not
policy.

699 tests (+23), coverage 128/128 + 6/6, ReDoS sweep 0 candidates / 150.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-02 21:13:08 +02:00
commit 2d98d6809d
10 changed files with 272 additions and 21 deletions

View file

@ -7,6 +7,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
> **Behaviour change, not a pure fix.** The three transform surfaces gain a
> refusal path they did not have. A caller that today passes a document larger
> than 1 000 000 characters gets an exception where it previously got a result.
> Both pinned consumers were measured against this before it was written; git
> pins are exact, so nobody picks it up without re-pinning.
### Added — input-size cap on the transform surfaces (OWASP LLM10)
`sanitize`, `fence` and `neutralize` now raise `OversizeInputError` above
`MAX_INPUT_CHARS` (1 000 000) instead of accepting text of any length. Since
`sanitize` is step 1 of `prepare_input` and only ever *removes*, that single
refusal bounds the whole input path.
They **reject** where the scanners **truncate**, and the asymmetry is the point:
- `scan_lexicon` / `scan_output` return findings. Reading a prefix costs
detection in the tail and nothing else — a lossy answer, but an answer.
- `sanitize` / `fence` / `neutralize` return *content*. Truncating would return
a shortened document (silent data loss for anything that persists the result)
or a transformed prefix followed by an untransformed tail — a bypass, since an
attacker chooses where in the document the payload sits.
The invariant the three now keep: **returned text is always fully transformed,
or not returned at all.**
`OversizeInputError` subclasses `ContractViolation`, so a pipeline already
bracketing its quarantined stage in `except ContractViolation` keeps failing
closed. Like its parent it is alert-routable: the message carries the size and
the cap, `details` names the refusing surface, and neither carries input.
`max_input_chars` is a per-call parameter, defaulting to the single calibrated
constant.
### Still uncapped, and deliberately
`scan_active_content` **called directly** and the okf link graph. Reached through
`scan_output`, `scan_active_content` inherits that function's cap. Both are
detection-shaped, so the truncate-and-flag mechanism transfers to them unchanged
— mechanical follow-up work, not a policy question. Recorded in
`docs/LIMITATIONS.md`.
## [0.3.4] — 2026-08-01
> **Denial-of-service fix on the INPUT path. Upgrade from 0.3.3.** 0.3.3 swept