1
0
Fork 0
Commit graph

2 commits

Author SHA1 Message Date
c72e648a94
fix(sanitize): catch every invisible codepoint, exempt only on context
Variation-selector smuggling (A01) and Sneaky Bits (A02) gave 0 findings
on 1.4.1 and stayed in the sanitized text. The denominator is now the
frozen Unicode 18.0 set (DICP | Cf | Co, 141 674 codepoints); each one is
stripped and flagged unless a known legitimate use exempts it on context:
one VS15/16 after an emoji base, one ideographic selector after a CJK
ideograph, ZWNJ/ZWJ beside Arabic-script or Indic letters, LRM/RLM/ALM
in a document with RTL text, an Arabic number sign before a digit. A run
is always a carrier; private use is caught in full (operator decisions
2026-10-28). One predicate serves sanitize, the output gate and the
lexicon's word-split row, whose regex stays byte-identical with commons.

A01 and A02 are green; the FP populations and the pinned OKF corpus are
identical before and after.
2026-10-29 08:31:43 +01:00
e4e18b38d7
test(threat-2026): one case per known 2025-2026 attack, red where not caught
Nineteen attacks that can reach an ingestion pipeline, verified against
primary sources, with four out of scope and their reasons. Each arrival
runs through the canonical wiring; twelve of 31 arrivals are red today
and marked strict xfail, so a fix fails until its marker is removed.
PoisonedRAG asserts that its documented gap still stands.
2026-10-28 21:27:33 +01:00