llm-security/tests
Kjell Tore Guttormsen 8d990e06d3 test(llm-security): v8 Phase 5 step 1 - golden baseline before any table swap
Records the reference artifacts Phase 5 swaps will be measured against, and
the gate that reads them. No extraction yet: this is the "before" picture,
and it had to land first or every later comparison would be confounded.
Typed test/ rather than feat/ deliberately - the only production change is
one widened export; the rest is gate, artifacts and generator.

Written failing-first (7 red on missing artifacts), then generated.

Three layers, because the plan's "assert .source/.flags of every regex" is
necessary but not sufficient:

  1. regex records - .source/.flags off the COMPILED object. After a swap a
     pattern is new RegExp(jsonString, flags), so the plan's named hazard
     (JSON backslash-doubling on 83+18 regexes) is visible here and nowhere
     else. Source-text comparison cannot see it.
  2. table records - key/value digests. Most of what Phase 4 moves is not a
     regex at all: HOMOGLYPH_MAP (x3, AS-IS), the typosquat tokens and the
     four OWASP maps are char->char and string->string data. A regex-only
     dump is blind to a broken homoglyph swap, i.e. to the bulk of the
     payload. Operator decision: widen the dump.
  3. file records - sha256 of the five moving-set sources. This dissolves
     STATE's open question (how to enumerate every regex): it is complete by
     construction, covering inline regexes in function bodies that no export
     walk reaches, with no JS parser in a zero-dep repo. A lexical count
     would have pinned a lie - severity.mjs scores 4 "regexes" that way and
     exports none.

Both layers were proven to fire, not assumed to: mutating one HOMOGLYPH_MAP
entry reddens the table layer, and widening an inline regex inside
decodeHexEscapes (unreachable by any export walk) reddens the file layer.

HOMOGLYPH_MAP is now exported from string-utils.mjs. That export is a source
change the plan already flags as a surface hazard ("private tables become
loaded"), so it is pre-paid here rather than confounding the before/after.

Reference run: the 61 showcase payloads through the real hook entry points,
sequentially - array order is semantic and the plan forbids key-sorting, so
concurrency is removed rather than sorted away. 61/61 match expectation,
which also settles the plan's open assumption that payloads.json expectations
match current behaviour. Coverage is recorded, not assumed: 47/83 patterns,
with the other 36 listed by key so the gate never implies coverage it lacks.

Suite counts are per-file, each file run alone - a total is unattributable,
and the three known timing-sensitive files flake only under concurrency.
2045 pass / 0 fail across 91 files, matching the pre-existing count exactly.
The gate's own file is excluded (it reads the artifact the run produces) and
that exclusion is named in the artifact.

Suite: 2053/2053, 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
2026-08-09 12:57:51 +02:00
..
e2e test(llm-security): cover TRG/SIG/AST in e2e pipeline + SIG miner/hacktool/rot13 families (#58,#59) 2026-07-18 10:46:19 +02:00
fixtures feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs 2026-08-09 10:25:03 +02:00
golden test(llm-security): v8 Phase 5 step 1 - golden baseline before any table swap 2026-08-09 12:57:51 +02:00
helpers test(llm-security): add JetBrains fixture tree + build helper 2026-04-18 10:49:49 +02:00
hooks feat(llm-security)!: v8 Phase 3 step 1 - remove the deprecated mode env-vars 2026-08-09 10:17:47 +02:00
lib test(llm-security): v8 Phase 5 step 1 - golden baseline before any table swap 2026-08-09 12:57:51 +02:00
scanners feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs 2026-08-09 10:25:03 +02:00