open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity
llm-security/examples/malicious-skill-demo/README.md
Kjell Tore Guttormsen b3c47330e1
 test(llm-security): store the malicious-skill demo encoded, materialize at run time 
v8.1.0 S2. examples/malicious-skill-demo/evil-project-health/ (7 files,
30 Unicode Tag chars, a base64 exfil blob) is now one archive,
evil-project-health.archive.json: rot13 text, every codepoint above U+007E
stored as a number, sha256 of each retired file recorded. materialize.mjs
writes it to a temp dir (CLI prints the path); run-demo.sh materializes
and deletes it itself; the six scanner tests that scanned the tree use it.
payload-trees.test.mjs asserts byte identity (mutation-checked).

av-surface: b 8->6, c 1->0, d 2->1. Demo 13/13 before and after. All
scanners report identical findings except git-forensics: it used to scan
this repository's own history (21 findings, none about the demo) and now
reports skipped in a temp dir, which git.test.mjs already accepts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 13:21:22 +02:00

4.5 KiB
Raw Blame History

Malicious Skill Demo

WARNING: This is a security test fixture, NOT a real plugin. All "malicious" patterns are intentionally planted for scanner testing.

What Is This?

A fake Claude Code plugin called "Project Health Dashboard" that looks legitimate but contains security threats across every category the LLM Security plugin can detect. It serves as:

  1. Showcase — demonstrates what the 7 deterministic scanners + 5 LLM agents can find
  2. Regression test — verifiable fixture for scanner development
  3. Educational resource — real attack patterns in a safe, contained context

Embedded Threat Categories

Scanner Threats Files
UNI (Unicode) Zero-width chars, Unicode Tags steganography, BIDI overrides, Cyrillic homoglyphs SKILL.fixture.md, health-check-agent.fixture.md
ENT (Entropy) Base64-encoded payloads, high-entropy credentials SKILL.fixture.md, telemetry.mjs
PRM (Permissions) Purpose-vs-tools mismatch, ghost hooks, haiku on sensitive agent, overprivileged health.fixture.md, health-check-agent.fixture.md, hooks.fixture.json
DEP (Dependencies) 6 typosquatting packages, malicious install scripts, unpinned versions package.json
TNT (Taint) 6 source-to-sink data flows (env→fetch, req.body→exec, input→eval) telemetry.mjs
GIT (Forensics) (Minimal — new directory, no git history) —
NET (Network) ngrok, webhook.site, requestbin, pipedream, pastebin, bit.ly, IP-based URLs SKILL.fixture.md, telemetry.mjs, health.fixture.md

LLM Agent Detection (Skill Scanner)

All 7 threat categories from the skill-scanner-agent are represented in SKILL.md:

  1. Prompt Injection — HTML comments with <!-- AGENT: ... -->, spoofed # SYSTEM: headers
  2. Data Exfiltration — printenv | base64, curl -X POST to external endpoints
  3. Privilege Escalation — instructions to modify hooks/ and settings.json
  4. Scope Creep — reads from ~/.ssh/, ~/.aws/, ~/.npmrc
  5. Hidden Instructions — Unicode Tags steganography, base64-encoded commands, invisible lines
  6. Toolchain Manipulation — typosquatting package installs in prerequisites
  7. Persistence — crontab, ~/.zshrc modification, LaunchAgent creation

How to Run

Quick: Deterministic Scanners Only

cd plugins/llm-security
./examples/malicious-skill-demo/run-demo.sh

The demo plugin is not stored in the repository as files: antivirus products (Windows Defender among them) quarantine a tree like this on clone. It lives encoded in evil-project-health.archive.json and is written to a temp directory when you need it. run-demo.sh does that for you and deletes it afterwards.

Or directly — materialize first, then scan the printed path:

DEMO="$(node examples/malicious-skill-demo/materialize.mjs)"
node scanners/scan-orchestrator.mjs "$DEMO"
rm -rf "$(dirname "$DEMO")"   # when you are done

Expected: BLOCK verdict, 44 findings (measured 2026-09-22). The temp directory has no git history, so the git-forensics scanner reports skipped.

Full: LLM-Enhanced Deep Scan

Materialize as above, then:

/security scan <printed path> --deep

This runs both the deterministic scanners AND the LLM agents (skill-scanner, mcp-scanner).

Safety

  • No actual secrets, tokens, or credentials are in these files
  • No real malicious code is executable — URLs are fake/non-functional
  • The package.json typosquatting packages don't exist or are harmless names
  • Install scripts reference non-existent domains
  • The tree is stored encoded (rot13 text, non-ASCII characters as codepoint numbers) and only exists as files in a temp directory while you use it
  • Discoverable files use .fixture.{md,json} suffix to prevent Claude Code's plugin loader from picking them up during recursive tree-walking

Security Assessment

A full combined security assessment (LLM skill scanner + 7 deterministic scanners) is available at security-assessment.md. This is real scanner output, not just expected findings:

  • 85 total findings (24 Critical, 24 High, 20 Medium, 6 Low, 11 Info)
  • Verdict: BLOCK 100/100 — both scanning layers independently maxed the risk score
  • All 8 scanners active (1 LLM + 7 deterministic)
  • Includes executive summary, per-scanner breakdown, combined verdict, recommendations, and methodology

Expected Findings

See expected-findings.md for the deterministic scanner catalog of ~50 expected findings with scanner, severity, file, and description.