1
0
Fork 0

feat(sanitize,fence,neutralize): reject oversize input instead of half-transforming it

The scanners cap by truncating: they return findings, so reading a prefix costs
detection in the tail and nothing else. The three transform surfaces return
*content*, where the same move is not available — a shortened document is silent
data loss, and a transformed prefix followed by an untransformed tail is a
bypass, since the attacker chooses where in the document the payload sits.

So they fail secure instead. Above MAX_INPUT_CHARS (1 000 000) sanitize, fence
and neutralize raise OversizeInputError. sanitize is step 1 of prepare_input and
only ever removes, so that one refusal bounds the whole input path.

OversizeInputError subclasses ContractViolation: a pipeline already bracketing
its quarantined stage keeps failing closed rather than meeting a type it has
never heard of. It inherits the alert-routable property too — sizes in the
message, refusing surface in details, no input in either.

Invariant now pinned across all three: returned text is always fully
transformed, or not returned at all.

Still uncapped and recorded in LIMITATIONS: scan_active_content called directly
(through scan_output it inherits that cap) and the okf link graph. Both are
detection-shaped, so truncate-and-flag transfers unchanged — mechanical, not
policy.

699 tests (+23), coverage 128/128 + 6/6, ReDoS sweep 0 candidates / 150.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-02 21:13:08 +02:00
commit 2d98d6809d
10 changed files with 272 additions and 21 deletions

View file

@ -53,6 +53,8 @@ from .active_content import (
is_active_tag,
redact,
)
from .calibration import MAX_INPUT_CHARS
from .contract import assert_within_input_cap
from .report import Finding, Report, Severity, Source
@ -64,13 +66,23 @@ class NeutralizeResult:
report: Report
def neutralize(text: str, source: Source = Source.OUTPUT) -> NeutralizeResult:
def neutralize(
text: str,
source: Source = Source.OUTPUT,
max_input_chars: int = MAX_INPUT_CHARS,
) -> NeutralizeResult:
"""Defang active-content constructs in ``text`` and report each class.
Rewrites markdown images/links, reference-link definitions, angle-bracket
autolinks, raw active HTML, and ``data:`` URIs into inert forms. Text with no
such construct is returned byte-identical with an empty report.
Raises :class:`~llm_ingestion_guard.contract.OversizeInputError` above
``max_input_chars``. A partially defanged artifact is the worst outcome
available here: it *looks* neutralized, and the live constructs are all in
the tail nobody re-reads.
"""
assert_within_input_cap(text, surface="neutralize", max_input_chars=max_input_chars)
report = Report()
out = text