open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

test(llm-security): store the malicious-skill demo encoded, materialize at run time

Browse source 
v8.1.0 S2. examples/malicious-skill-demo/evil-project-health/ (7 files,
30 Unicode Tag chars, a base64 exfil blob) is now one archive,
evil-project-health.archive.json: rot13 text, every codepoint above U+007E
stored as a number, sha256 of each retired file recorded. materialize.mjs
writes it to a temp dir (CLI prints the path); run-demo.sh materializes
and deletes it itself; the six scanner tests that scanned the tree use it.
payload-trees.test.mjs asserts byte identity (mutation-checked).

av-surface: b 8->6, c 1->0, d 2->1. Demo 13/13 before and after. All
scanners report identical findings except git-forensics: it used to scan
this repository's own history (21 findings, none about the demo) and now
reports skipped in a temp dir, which git.test.mjs already accepts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 13:21:22 +02:00
commit b3c47330e1
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
18 changed files with 639 additions and 409 deletions
Download patch file Download diff file Expand all files Collapse all files

19
examples/malicious-skill-demo/README.md
View file

@ -44,18 +44,28 @@ cd plugins/llm-security
./examples/malicious-skill-demo/run-demo.sh
```
Or directly:
The demo plugin is not stored in the repository as files: antivirus products (Windows
Defender among them) quarantine a tree like this on clone. It lives encoded in
`evil-project-health.archive.json` and is written to a temp directory when you need it.
`run-demo.sh` does that for you and deletes it afterwards.
Or directly — materialize first, then scan the printed path:
```bash
node scanners/scan-orchestrator.mjs examples/malicious-skill-demo/evil-project-health/
DEMO="$(node examples/malicious-skill-demo/materialize.mjs)"
node scanners/scan-orchestrator.mjs "$DEMO"
rm -rf "$(dirname "$DEMO")" # when you are done
```
**Expected:** BLOCK verdict, ~59 findings, all active scanners reporting.
**Expected:** BLOCK verdict, 44 findings (measured 2026-09-22). The temp directory has no git
history, so the git-forensics scanner reports `skipped`.
### Full: LLM-Enhanced Deep Scan
Materialize as above, then:
```
/security scan examples/malicious-skill-demo/evil-project-health/ --deep
/security scan <printed path> --deep
```
This runs both the deterministic scanners AND the LLM agents (skill-scanner, mcp-scanner).
@ -66,6 +76,7 @@ This runs both the deterministic scanners AND the LLM agents (skill-scanner, mcp
- No real malicious code is executable — URLs are fake/non-functional
- The package.json typosquatting packages don't exist or are harmless names
- Install scripts reference non-existent domains
- The tree is stored encoded (rot13 text, non-ASCII characters as codepoint numbers) and only exists as files in a temp directory while you use it
- Discoverable files use `.fixture.{md,json}` suffix to prevent Claude Code's plugin loader from picking them up during recursive tree-walking
## Security Assessment