open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity
llm-security/README.md
Kjell Tore Guttormsen fd7de230e4
 docs(readme): lead with the value proposition, matching the new forge description 
The Forgejo repository description is now the value proposition
(operator decision 2026-09-23), so README line 3 can carry it instead
of the old, drier description line. repo-standard README-DESC requires
exact equality, so the line is plain text, not bold.

repo-standard 22/22, doc-consistency 45/0, self-scan README 0 findings.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 10:11:48 +02:00

225 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# LLM Security Plugin for Claude Code
Check skills, plugins and MCP servers before they run with your agent's permissions — and stop secret leaks, destructive commands and prompt injection while you work.
> **Solo-maintained, fork-and-own.** A starting point, not a vendor product. Issues are welcome as signals; pull requests are not accepted. See [GOVERNANCE.md](https://git.fromaitochitta.com/open/repo-standard/src/branch/main/GOVERNANCE.md).
*AI-generated: all code produced by Claude Code through dialog-driven development. Every change is human-directed, reviewed, and validated before commit. Per Anthropic Consumer Terms §4, ownership of outputs is assigned to the user; this plugin is licensed MIT.*
![Version](https://img.shields.io/badge/version-8.1.2-blue)
![Platform](https://img.shields.io/badge/platform-Claude_Code_Plugin-purple)
![Scanners](https://img.shields.io/badge/scanners-22-cyan)
![License](https://img.shields.io/badge/license-MIT-lightgrey)
> [!WARNING]
> **Rolling this out in an organization?** Antivirus and EDR products such as Microsoft Defender **can** raise alerts on files in this repository, because it contains detection rules and descriptions of attack techniques. Read [Antivirus and EDR alerts](#antivirus-and-edr-alerts) before you deploy, and report any alert to **security@fromaitochitta.com** so it can be fixed.
## Why
A Claude Code extension does not just run code — it can give your agent instructions. A malicious skill can tell Claude to read `~/.ssh`, post your environment variables to a webhook, or rewrite its own configuration, while presenting itself as a harmless "project health dashboard". This is the npm/PyPI supply-chain problem with LLM agency added.
What you get:
- **Guardrails from the moment you install.** Nine hooks run on every prompt and tool call. Five block: secrets written to files, writes to `.env`/`.ssh`/credentials, destructive shell commands, compromised or typosquatted packages, and prompt injection. Four warn.
- **A verdict before you install anything.** `/security scan <url> --deep` clones a repository into a temp directory (OS-sandboxed on macOS and on Linux with `bwrap`), runs 14 deterministic scanners plus LLM analysis, and returns ALLOW / WARNING / BLOCK — without loading the repository into your session.
- **A grade for your own setup.** `/security posture` scores your Claude Code configuration across 16 posture categories in seconds; `/security audit` gives an A–F report with the fixes.
Findings map to [OWASP LLM Top 10 (2025)](https://genai.owasp.org/llm-top-10/), [OWASP Agentic AI Top 10](https://genai.owasp.org/agentic-ai/), OWASP Skills Top 10, MCP Top 10 and the [AI Agent Traps](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6372438) taxonomy (Google DeepMind).
## Install
Requires [Claude Code](https://docs.anthropic.com/en/docs/claude-code) v2.x+ and Node.js (any recent LTS).
```bash
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
claude plugin install llm-security@ktg-plugin-marketplace
```
The hooks are active immediately. No configuration is needed.
## First five minutes
| Run | You get |
|-----|---------|
| `/security posture` | Scorecard for this project's Claude Code setup, with the top gaps |
| `/security scan <path\|url> --deep` | ALLOW / WARNING / BLOCK for a skill, plugin, MCP server or repository |
| `/security plugin-audit <path\|url>` | Install / Review / Do Not Install for a plugin |
| `/security audit` | Full A–F audit with a prioritized action plan |
| `/security harden --apply` | Grade A reference config (`settings.json`, `CLAUDE.md`, `.gitignore`), with backup |
> [!IMPORTANT]
> **Scan a repository before you clone it.** A poisoned `CLAUDE.md` enters the model's context the moment you open a cloned repository — before any hook can run. `/security scan <url> --deep` inspects it from a pre-extracted evidence package instead.
## What runs automatically
| Hook | When | Effect |
|------|------|--------|
| Prompt injection scan | Every prompt | **Blocks** override instructions, spoofed system headers, identity redefinition; decodes obfuscation (Unicode Tag, hex, URL, base64, rot13) first |
| Secret detection | Edit, Write | **Blocks** AWS/Azure/GitHub/npm tokens, PEM keys, database URLs, JWTs, webhook URLs — 19 patterns, extendable in policy |
| Path guard | Edit, Write | **Blocks** writes to `.env*`, `.ssh/`, `.aws/`, `.gnupg/`, credentials, hook scripts, `settings.json` |
| Destructive commands | Bash | **Blocks** `rm -rf /`, pipe-to-shell, fork bombs, `chmod 777`, eval tricks — after undoing shell obfuscation |
| Supply chain | Bash | **Blocks** known-compromised packages (incl. known typosquats), critical CVEs, and brand-new packages (<72 h, few releases); **warns** on look-alike names — npm, pip, brew, docker, go, cargo, gem |
| Output check | After every tool | Warns on injection in tool output and on MCP tool descriptions that drift over time |
| Session guard | After every tool | Warns on the "lethal trifecta": untrusted input + sensitive read + an exfiltration step |
| Pre-compact scan | Before compaction | Warns on injection or credentials about to survive into the compacted context |
| Update check | Once a day | Tells you when a newer version exists (`LLM_SECURITY_UPDATE_CHECK=off` disables it) |
Injection and trifecta modes (`block`/`warn`/`off`) are set per project in `.llm-security/policy.json`; the pre-compact mode via `LLM_SECURITY_PRECOMPACT_MODE`.
## All commands
| Command | Purpose |
|---------|---------|
| `/security` | List the sub-commands |
| `/security scan [path\|url]` | Supply-chain gate; `--deep` adds the 14 deterministic scanners |
| `/security deep-scan [path]` | The 14 deterministic scanners only, with a synthesized report |
| `/security plugin-audit [path\|url]` | Plugin trust assessment |
| `/security mcp-audit [--live]` | Audit installed MCP server configs |
| `/security mcp-inspect` | Connect to running MCP servers and scan their live tool descriptions |
| `/security mcp-baseline-reset` | Reset the MCP drift baseline after a legitimate server upgrade |
| `/security ide-scan [target\|url]` | Scan VS Code (and forks) and JetBrains extensions, installed or by URL |
| `/security supply-check [path]` | Re-audit installed dependencies against blocklists, OSV.dev and typosquats |
| `/security posture` | Quick scorecard, 16 categories |
| `/security audit` | Full audit, A–F |
| `/security dashboard` | Posture across every project on the machine |
| `/security diff [path]` | New / resolved / unchanged findings against a baseline |
| `/security watch [path]` | Run `diff` on an interval |
| `/security registry` | Skill fingerprint registry |
| `/security clean [path]` | Fix what is safe to fix, ask about the rest (`--dry-run` to preview) |
| `/security harden [path]` | Generate a Grade A config |
| `/security threat-model` | Guided STRIDE × MAESTRO session, 15–30 min |
| `/security red-team` | 72 attack scenarios against the hooks |
| `/security pre-deploy` | Pre-deployment checklist |
Every report command also writes a self-contained HTML version and prints a link to it. Scanner-by-scanner detail: [`docs/scanner-reference.md`](docs/scanner-reference.md).
## For teams and CI
- **Policy as code** — `.llm-security/policy.json` travels with the repository: hook modes, thresholds, audit log path. A scanned *foreign* repository's own policy and ignore file are not applied, so it cannot suppress its findings. Known exceptions — a tarball or `git archive` export under your working directory, a Python venv, a `vendor/` folder — are listed under 8.1.2 in the changelog.
- **Runs without Claude Code** — `node bin/llm-security.mjs scan <target>` (also `deep-scan`, `posture`, `ide-scan`, `audit-bom`, `benchmark`); no API key needed. It is **not** air-gapped: OSV.dev lookups (package names and versions), `npm audit`/`pip-audit`, and DNS checks on URLs it finds reach the network, with no switch to turn them off. Block egress at the network layer if you need an offline run.
- **CI gates** — `--fail-on <severity>` exit codes, `--compact` output, SARIF 2.1.0. Templates for GitHub Actions, Azure DevOps and GitLab CI in [`ci/`](ci/); guide in [`docs/ci-cd-guide.md`](docs/ci-cd-guide.md).
- **Evidence** — JSONL audit trail with OWASP tags for your SIEM, CycloneDX 1.6 AI bill of materials, and posture checks mapped to EU AI Act, NIST AI RMF and ISO 42001.
## Antivirus and EDR alerts
This is a security tool. It ships regular expressions for known-bad code, knowledge files that explain how injection, exfiltration and persistence attacks work, and an encoded demo of a malicious plugin. Nothing in it is malware, but products that use machine learning and heuristics — Microsoft Defender among them ([next-generation protection](https://learn.microsoft.com/defender-endpoint/next-generation-protection)) — can still flag such text. A plain `SKILL.md` in another security repository has been quarantined as `Trojan:Script/Wacatac.H!ml`.
What still sits on disk as readable text, and is therefore the likeliest to be flagged: the regex signature tables in `scanners/commons/` (some name hacking tools as plain words), the attack descriptions in `knowledge/`, and the injection phrases and sample commands the examples feed to the hooks.
### What v8.0 → v8.1.2 did about it
All antivirus work landed in **v8.1.0**; v8.1.1 and v8.1.2 did not touch it. (v8.0.0 moved the detection tables, including the malware signatures, into the vendored `scanners/commons/` data that ships with the plugin.)
| Change (v8.1.0) | Effect |
|-----------------|--------|
| Test payloads are built at test time in a temp directory, from split strings or rot13 | No runnable malicious fixture is stored in the repository |
| The malicious-plugin demo and the poisoned `CLAUDE.md` example ship as encoded archives | They exist in decoded form only in a temp directory while you run them |
| Skill-scanner detection lists moved from an agent file to `knowledge/`; runnable one-liners in `knowledge/*.md` rewritten as descriptions | A quarantined knowledge file does not break the plugin; a quarantined agent file would |
| Download-into-shell literals removed from hook source | The hook that blocks these commands no longer contains one |
| Gate: [`tests/av-surface.test.mjs`](tests/av-surface.test.mjs) | The suite fails if a runnable payload file, a base64 shell-command blob, hidden Unicode (outside the commons conformance corpus), a known payload tree, or a payload literal in agents/commands/hooks returns to disk |
Detection is unchanged: the signature tables and the golden baseline are identical before and after.
**Not measured:** no clone or install has been tested against Defender on a Windows machine. "No alerts" is the design goal, not a verified result.
### If you get an alert
1. **Do not exclude the folder.** Microsoft advises against exclusions as a fix: "Every exclusion is a protection gap" ([exclusions overview](https://learn.microsoft.com/defender-endpoint/microsoft-defender-antivirus-exclusions-overview)).
2. **Report it to Microsoft as a false positive.**
- *Managed devices (Defender for Endpoint / Defender XDR):* your security team submits the file in the Defender portal under **Submissions → Files**, classified as **Clean (false positive)** ([how](https://learn.microsoft.com/defender-endpoint/admin-submissions-mde)).
- *Single machine:* restore it under Windows Security → Protection history ([how](https://learn.microsoft.com/defender-endpoint/restore-quarantined-files-microsoft-defender-antivirus)) and submit it at <https://www.microsoft.com/wdsi/filesubmission>.
3. **Email security@fromaitochitta.com** with the file path, the detection name, the plugin version (or commit), and the antivirus product and its signature version. The alert is handled as a bug: the goal is a patch release in which the file no longer trips the product, with the gate above extended so it stays that way. There is no SLA (see [`SECURITY.md`](SECURITY.md)).
If you only need the scanners and hooks, a sparse checkout keeps knowledge files, tests, examples and docs off disk:
```bash
git clone --filter=blob:none --no-checkout https://git.fromaitochitta.com/open/llm-security.git
cd llm-security
git sparse-checkout set scanners hooks
git checkout main
```
The hooks do not need `knowledge`; add it to the `set` line for the typosquat checks, `ide-scan`, the skill registry and `red-team`. This affects your own clone only — not what `claude plugin install` puts in the plugin cache.
## Known limitations
Prompt injection cannot be fully prevented with today's model architectures; motivated adaptive attackers defeat every published defense. This plugin raises the cost of an attack with independent layers — it does not guarantee against one.
| Limit | What to do |
|-------|------------|
| A cloned repository's `CLAUDE.md` loads before any hook runs | Scan remotely first: `/security scan <url> --deep` |
| Regex cannot catch every novel phrasing of an injection | Add an ML classifier such as [parry-guard](https://github.com/vaporif/parry); they run side by side |
| Remote-scan clones run without an OS sandbox on Windows, and on Linux where `bwrap` is missing or blocked (Ubuntu 24.04+ by default) — only a warning is printed | Git hardening still applies; on Windows run Claude Code in WSL2 or Docker, on Ubuntu allow `bwrap` in AppArmor |
| RAG pipelines, LLM gateways, SSO/SCIM are outside a plugin's reach | Use platform or infrastructure tooling |
| Opus with extended context: subagents do not support it and fail | Run `/model Opus` before the security commands |
Design rationale: [`docs/defense-philosophy.md`](docs/defense-philosophy.md). Hardening guide: [`docs/security-hardening-guide.md`](docs/security-hardening-guide.md).
## Project scope
A solo open-source project in **stabilization mode** since 2026-05-01: bug and security fixes, Claude Code compatibility, knowledge-base refresh.
## Non-goals
Out of scope — fork and own them under MIT if you need them: web dashboards or fleet servers, an inline prompt firewall, real-time IDE scanning, compliance evidence packs, ticketing/chat connectors, hosted ML detectors, SSO/RBAC. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the fork-and-own guide.
## Verify it yourself
No CI runs the tests for you — this forge has no Actions runner. From a clean clone:
```bash
npm test # the full suite
node scanners/scan-orchestrator.mjs . # the plugin scanning itself
```
The self-scan is not clean on purpose: a tool that documents attacks and tests against hostile code trips its own scanners. `.llm-security-ignore` suppresses the expected findings and explains each one; delete it to see everything.
The encoded malicious-plugin demo is a realistic attack. LLM agents and deterministic scanners together find 85 issues and return BLOCK 100/100 (see its [assessment](examples/malicious-skill-demo/security-assessment.md)). The commands below run the deterministic half:
```bash
DEMO="$(node examples/malicious-skill-demo/materialize.mjs)" # decodes into a temp dir, prints the path
node scanners/scan-orchestrator.mjs "$DEMO"
rm -rf "$(dirname "$DEMO")"
```
For the full pipeline, run `/security scan <printed path> --deep` in Claude Code before removing the directory.
More runnable demonstrations — injection showcase, trifecta walkthrough, MCP rug pull, supply-chain attack, bash evasion — are in [`examples/`](examples/).
## Upgrading from v7
v8.0.0 removed four environment variables. A removed variable is **silently ignored** — `LLM_SECURITY_INJECTION_MODE=off` no longer turns anything off. Find them before you upgrade:
```bash
env | grep '^LLM_SECURITY_\(INJECTION_MODE\|TRIFECTA_MODE\|ESCALATION_WINDOW\|AUDIT_LOG\|DEPRECATION_QUIET\)='
grep -rn 'LLM_SECURITY_\(INJECTION_MODE\|TRIFECTA_MODE\|ESCALATION_WINDOW\|AUDIT_LOG\)' \
~/.zshenv ~/.bashrc .envrc .github/workflows/ 2>/dev/null
```
Move each one into `.llm-security/policy.json`:
| Removed env var | Policy key | Default |
|-----------------|------------|---------|
| `LLM_SECURITY_INJECTION_MODE` | `injection.mode` | `block` |
| `LLM_SECURITY_TRIFECTA_MODE` | `trifecta.mode` | `warn` |
| `LLM_SECURITY_ESCALATION_WINDOW` | `trifecta.escalation_window` | `5` |
| `LLM_SECURITY_AUDIT_LOG` | `audit.log_path` | unset (off) |
`LLM_SECURITY_DEPRECATION_QUIET` is gone with the warning it silenced. Details in [`CHANGELOG.md`](CHANGELOG.md) under 8.0.0.
## Changelog
- **8.1.2** (2026-09-22) — Installed packages (`node_modules`) and Claude Code's plugin directory count as foreign, so their ignore and policy files are not applied.
- **8.1.1** (2026-09-22) — A clone, submodule or worktree *under* your working directory counts as foreign.
- **8.1.0** (2026-09-22) — Antivirus surface (see above); a scanned foreign repository's own ignore and policy files are no longer applied.
- **8.0.0** (2026-09-18) — Breaking cleanup (see [Upgrading from v7](#upgrading-from-v7)); detection tables built from shared, versioned data.
Full history: [`CHANGELOG.md`](CHANGELOG.md).
## License and contact
MIT — see [`LICENSE`](LICENSE). Built on published research from OWASP, ToxicSkills, ClawHavoc, MCPTox, Pillar Security, Invariant Labs, GHSL Security Lab, Operant AI and Google DeepMind; sources are cited in `knowledge/`.
- **Security vulnerabilities and antivirus alerts:** security@fromaitochitta.com — not a public issue. See [`SECURITY.md`](SECURITY.md).
- **Bugs and feature requests:** open an issue in the [marketplace repository](https://git.fromaitochitta.com/open/ktg-plugin-marketplace).
- **Pull requests:** not accepted; see [`CONTRIBUTING.md`](CONTRIBUTING.md).
Reference in a new issue View git blame Copy permalink