1
0
Fork 0

feat(output): decode hex-wrapped credentials at the persist gate

Gap 1 closed. The output gate finds runs of 32+ hex digits
(calibration.HEX_WRAP_MIN_LEN), decodes both byte alignments, keeps text
that is at least 80% printable (the base64 bar, now one shared helper in
entropy) and runs the credential patterns on it. A hex-wrapped AWS key
surfaces as decoded:egress:aws-access-key-id with "in hex run @ N" as
evidence; the value is never in it. Credentials only: a hex-wrapped
injection is not decoded.

Measured before and after on the same trees: all 3583 clean documents
(three FP populations and the 78 pinned OKF concepts) are identical line
for line under both presets (disposition, labels, severity, count,
offset); OKF import stays 47/31. They and 569 READMEs hold 2403 hex runs
of 32+ digits, none of which decodes to text. scan_output CPU 122.9 s
before, 123.4 s after; one solid hex run at the 1M cap 4.2 s -> 5.8 s.

14 new tests (10 red for the right reason first), 6 of 6 mutations red.
Matrix 146/146 + 5/5 gaps, suite 1274 passed, ReDoS sweep 169 patterns.
This commit is contained in:
Kjell Tore Guttormsen 2026-11-01 17:06:44 +01:00
commit 0eab4345bf
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
14 changed files with 265 additions and 40 deletions

View file

@ -623,11 +623,14 @@ items; this is the full list, each with the mechanism.
the most common shape in real documentation. The residual: a *clicked* link to an
attacker-controlled page can have its `location.hash` read by that page's script,
so a fragment payload on a link (not an image) is uncovered.
- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate
decodes base64 blobs and re-scans the plaintext, so a base64-*wrapped* secret
surfaces as `decoded:egress:*`. `entropy` exposes decoded plaintext for base64
only, so hex (and other encodings, or nested wraps) is a deliberate boundary —
decode the transport layer first if you need it scanned.
- **Secret egress: base64- and hex-wrapped are caught, other wraps are not.** The
output gate decodes base64 blobs and, since 1.10.0, hex runs of 32 characters or
more (both byte alignments, so one stray digit cannot hide the credential), and
re-scans the plaintext, so a wrapped secret surfaces as `decoded:egress:*`. Hex
split by separators (`41:4b`, `41 4b`, `\x41`), other encodings and nested wraps
are a deliberate boundary — decode the transport layer first if you need it
scanned. The hex plaintext goes to the credential patterns only: an injection
wrapped in hex is not decoded (one wrapped in base64 is).
- **Prose that merely mentions `<script>` fires `hybrid-xss:script-tag`.** The
pattern matches the opening tag and no longer requires `</script>`, so a
@ -686,7 +689,7 @@ items; this is the full list, each with the mechanism.
missed; on one preset it is held for review instead of halted.
- **The ReDoS sweep has a measured sensitivity floor, not a clean bill of
health.** All 168 compiled patterns across all eleven regex-bearing modules are
health.** All 169 compiled patterns across all eleven regex-bearing modules are
swept arm by arm — payloads synthesised per run from each pattern's own
skeleton, so `[`, `[system]` and `[system](` are each probed separately rather
than relying on generic units, and each pattern is timed in the call mode the
@ -816,8 +819,11 @@ today (WARN) that the rule would stop. Because those 3,583 hold no external imag
569 package READMEs (from installed npm and PyPI packages, deduplicated by sha256)
were measured too.
1. **Hex-wrapped secret egress** — `entropy` decodes base64 only (above). Open.
Measured to close without new alarms; next in line.
1. **Hex-wrapped secret egress** — **closed in 1.10.0.** The output gate decodes
hex runs of 32 characters or more and runs the credential patterns on the text
(see "Secret egress" above). Measured as built: every one of the 3,583 documents
keeps its disposition, labels and severities. They and the 569 READMEs hold
2,403 hex runs this long (checksums and ids), and none decodes to text.
2. **Semantic / factual poisoning** — invisible to token analysis (above).
**Measured reason:** the cheapest rule, a factual claim of the form "is made of"
or "was completed in", gave 0 new alarms on the 3,583 and fired on 3 of the 569