open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

test(llm-security): store the malicious-skill demo encoded, materialize at run time

Browse source 
v8.1.0 S2. examples/malicious-skill-demo/evil-project-health/ (7 files,
30 Unicode Tag chars, a base64 exfil blob) is now one archive,
evil-project-health.archive.json: rot13 text, every codepoint above U+007E
stored as a number, sha256 of each retired file recorded. materialize.mjs
writes it to a temp dir (CLI prints the path); run-demo.sh materializes
and deletes it itself; the six scanner tests that scanned the tree use it.
payload-trees.test.mjs asserts byte identity (mutation-checked).

av-surface: b 8->6, c 1->0, d 2->1. Demo 13/13 before and after. All
scanners report identical findings except git-forensics: it used to scan
this repository's own history (21 findings, none about the demo) and now
reports skipped in a temp dir, which git.test.mjs already accepts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 13:21:22 +02:00
commit b3c47330e1
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
18 changed files with 639 additions and 409 deletions
Download patch file Download diff file Expand all files Collapse all files

15
examples/malicious-skill-demo/run-demo.sh
View file

@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Malicious Skill Demo — Regression test for the deep-scan pipeline
# Malicious Skill Demo - Regression test for the deep-scan pipeline
# NOTE: Unix/macOS only. Requires bash. Not available on Windows without WSL.
#
# Usage:
@ -8,19 +8,28 @@
# ./examples/malicious-skill-demo/run-demo.sh
#
# Expected: BLOCK verdict, 40+ findings across 7 scanners, exit code 2
#
# The demo plugin is not stored on disk as files (v8.1.0: antivirus products
# quarantine it). It lives encoded in evil-project-health.archive.json and is
# materialized into a temp dir for the run, then deleted.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PLUGIN_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
TARGET="$SCRIPT_DIR/evil-project-health"
OUTPUT_FILE="$(mktemp)"
PASS=0
FAIL=0
cleanup() { rm -f "$OUTPUT_FILE"; }
TARGET=""
cleanup() {
rm -f "$OUTPUT_FILE"
if [ -n "$TARGET" ]; then rm -rf "$(dirname "$TARGET")"; fi
}
trap cleanup EXIT
TARGET="$(node "$SCRIPT_DIR/materialize.mjs")"
assert() {
local desc="$1" result="$2"
if [ "$result" -eq 0 ]; then