`_ZERO_WIDTH` (sanitize, input) and `_ZERO_WIDTH_CPS` (output,
`_scan_invisible_carriers`) tested U+200D on codepoint membership alone.
`disposition._CARRIER_LABELS` grades both as any-tier FAIL_SECURE with no
appeal, so any first-party document containing a ZWJ-composed emoji --
professions, families, skin tones, flag variants -- was hard-blocked forever.
Reported by ms-ai-architect; confirmed here against the code.
The strip was the worse half and was not in the report: sanitize *removed* the
joiner, silently decomposing the emoji into two unrelated ones. A module whose
contract is "only ever removes carriers" was corrupting content.
The fix is the one our own lexicon row `unicode:zero-width-in-word` (`\w[ZW]\w`)
already used: judge the joiner by CONTEXT, not identity. A ZWJ is exempt only
when BOTH neighbours are emoji-context codepoints. Half-context is not context,
so `a<ZWJ>{emoji}` stays a carrier and an attacker cannot buy exemption with a
single emoji.
Blocks, not an emoji table. Measured against Unicode 17.0's
`emoji-zwj-sequences.txt`: 1614 RGI sequences use 122 distinct codepoints
adjacent to a ZWJ, and the five ranges cover 122/122. The measurement earned
its keep -- the hand-reasoned candidate table missed U+2194, U+2195 and U+2B1B.
Shipping the RGI list itself would be exact on the day it landed and stale at
the next Unicode release, reopening this same false positive for every new
emoji; whole blocks carry the unassigned headroom (458 Cn codepoints) that
future emoji are allocated into, so the table does not age.
The predicate is defined once in sanitize and imported by output. A second copy
is how the input side stops flagging while the output side keeps blocking; the
cross-surface test asserts the two agree on six inputs.
Two residuals, both in LIMITATIONS (33 -> 34, README bumped): a ZWJ between two
emoji is now exempt and could carry a covert channel (one emoji per bit, cannot
split a word); and U+200C (ZWNJ) still has no context test, so Persian, Arabic
and Devanagari documents -- where it is orthographically required -- stay
blocked. That needs a script-based criterion and no corpus is here to verify it
against, so it is parked as a known FP class rather than guessed at.
736 green (was 727), coverage 128/128, 6/6 documented gaps hold.
Kjell Tore Guttormsen
a59184bb7f