open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity
llm-security/examples/malicious-skill-demo
History
Kjell Tore Guttormsen 52084204da
 docs(llm-security): describe the base64 exfil payloads instead of showing them 
v8.1.0 S2. security-assessment.md (two blobs, one split, plus the decoded
exfil command) and the base64 pipe-to-shell one-liner in
knowledge/skill-threat-patterns.md now describe the payload rather than
reproduce it. The entropy path-suppression test builds its 84-char blob
from 12-char fragments, so no contiguous decodable blob sits on disk.

av-surface: b 5->0. a=0 a2=0 b=0 c=0 d=0 - the probe is green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 13:23:24 +02:00
..
evil-project-health.archive.json test(llm-security): store the malicious-skill demo encoded, materialize at run time 2026-09-22 13:21:22 +02:00
expected-findings.md feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
materialize.mjs test(llm-security): store the malicious-skill demo encoded, materialize at run time 2026-09-22 13:21:22 +02:00
README.md test(llm-security): store the malicious-skill demo encoded, materialize at run time 2026-09-22 13:21:22 +02:00
run-demo.sh test(llm-security): store the malicious-skill demo encoded, materialize at run time 2026-09-22 13:21:22 +02:00
security-assessment.md docs(llm-security): describe the base64 exfil payloads instead of showing them 2026-09-22 13:23:24 +02:00

README.md

Malicious Skill Demo

WARNING: This is a security test fixture, NOT a real plugin. All "malicious" patterns are intentionally planted for scanner testing.

What Is This?

A fake Claude Code plugin called "Project Health Dashboard" that looks legitimate but contains security threats across every category the LLM Security plugin can detect. It serves as:

  1. Showcase — demonstrates what the 7 deterministic scanners + 5 LLM agents can find
  2. Regression test — verifiable fixture for scanner development
  3. Educational resource — real attack patterns in a safe, contained context

Embedded Threat Categories

Scanner Threats Files
UNI (Unicode) Zero-width chars, Unicode Tags steganography, BIDI overrides, Cyrillic homoglyphs SKILL.fixture.md, health-check-agent.fixture.md
ENT (Entropy) Base64-encoded payloads, high-entropy credentials SKILL.fixture.md, telemetry.mjs
PRM (Permissions) Purpose-vs-tools mismatch, ghost hooks, haiku on sensitive agent, overprivileged health.fixture.md, health-check-agent.fixture.md, hooks.fixture.json
DEP (Dependencies) 6 typosquatting packages, malicious install scripts, unpinned versions package.json
TNT (Taint) 6 source-to-sink data flows (env→fetch, req.body→exec, input→eval) telemetry.mjs
GIT (Forensics) (Minimal — new directory, no git history) —
NET (Network) ngrok, webhook.site, requestbin, pipedream, pastebin, bit.ly, IP-based URLs SKILL.fixture.md, telemetry.mjs, health.fixture.md

LLM Agent Detection (Skill Scanner)

All 7 threat categories from the skill-scanner-agent are represented in SKILL.md:

  1. Prompt Injection — HTML comments with <!-- AGENT: ... -->, spoofed # SYSTEM: headers
  2. Data Exfiltration — printenv | base64, curl -X POST to external endpoints
  3. Privilege Escalation — instructions to modify hooks/ and settings.json
  4. Scope Creep — reads from ~/.ssh/, ~/.aws/, ~/.npmrc
  5. Hidden Instructions — Unicode Tags steganography, base64-encoded commands, invisible lines
  6. Toolchain Manipulation — typosquatting package installs in prerequisites
  7. Persistence — crontab, ~/.zshrc modification, LaunchAgent creation

How to Run

Quick: Deterministic Scanners Only

cd plugins/llm-security
./examples/malicious-skill-demo/run-demo.sh

The demo plugin is not stored in the repository as files: antivirus products (Windows Defender among them) quarantine a tree like this on clone. It lives encoded in evil-project-health.archive.json and is written to a temp directory when you need it. run-demo.sh does that for you and deletes it afterwards.

Or directly — materialize first, then scan the printed path:

DEMO="$(node examples/malicious-skill-demo/materialize.mjs)"
node scanners/scan-orchestrator.mjs "$DEMO"
rm -rf "$(dirname "$DEMO")"   # when you are done

Expected: BLOCK verdict, 44 findings (measured 2026-09-22). The temp directory has no git history, so the git-forensics scanner reports skipped.

Full: LLM-Enhanced Deep Scan

Materialize as above, then:

/security scan <printed path> --deep

This runs both the deterministic scanners AND the LLM agents (skill-scanner, mcp-scanner).

Safety

  • No actual secrets, tokens, or credentials are in these files
  • No real malicious code is executable — URLs are fake/non-functional
  • The package.json typosquatting packages don't exist or are harmless names
  • Install scripts reference non-existent domains
  • The tree is stored encoded (rot13 text, non-ASCII characters as codepoint numbers) and only exists as files in a temp directory while you use it
  • Discoverable files use .fixture.{md,json} suffix to prevent Claude Code's plugin loader from picking them up during recursive tree-walking

Security Assessment

A full combined security assessment (LLM skill scanner + 7 deterministic scanners) is available at security-assessment.md. This is real scanner output, not just expected findings:

  • 85 total findings (24 Critical, 24 High, 20 Medium, 6 Low, 11 Info)
  • Verdict: BLOCK 100/100 — both scanning layers independently maxed the risk score
  • All 8 scanners active (1 LLM + 7 deterministic)
  • Includes executive summary, per-scanner breakdown, combined verdict, recommendations, and methodology

Expected Findings

See expected-findings.md for the deterministic scanner catalog of ~50 expected findings with scanner, severity, file, and description.