Kjell Tore Guttormsen
9a51e832b9
Ran repo-standard v0.1.1 against this repo (class: plugin, trait: security). 6 ERROR, 4 WARN, 1 SKIP recorded before any edit; now 0 ERROR, 12 checks pass. BROKEN - INSTALL-NO-CLI: added `claude plugin install llm-security@ktg-plugin-marketplace` beside the existing `marketplace add` line. The settings.json `enabledPlugins` block stays — it is a legitimate second form, just not a CLI command. - LINK-OUTSIDE-REPO (README:7, reported by catalog): the disclosure link pointed at `../../README.md#ai-generated-code-disclosure`, relative to the pre-split monorepo and anchored at a heading that never existed. Replaced with the inline text the polyrepo migration's step 5 was meant to write. MISSING - `## Non-goals`: added as its own heading over the existing out-of-scope table inside `## Project scope`. The `## Project scope` heading is kept because SECURITY.md and CONTRIBUTING.md reference it by name. - `## Changelog`: promoted from the trailing "Full history in CHANGELOG.md" line. - `## Known limitations` (required by the `security` trait): renamed from `## What this plugin does NOT cover` — same table, contract heading. WEAKENING - README-DESC: opening line now matches the forge description verbatim, so description == catalog == README holds. This replaced the tagline "Automated defense and advisory analysis for the agentic AI attack surface." - HEADING-LEVEL: `### Install` promoted to `## Install`; `## Quick Start` split into `## Requirements` / `## Install` / `## First scan`. - BADGE-STATIC-CLAIM: dropped the static `tests-2034` badge. No runner exists on this forge, so the badge asserted a run nothing performs. Replaced under `## Self-scan` with the command that runs the suite from a clean clone, and the plain statement that nothing runs it automatically. - LINK-NON-REPO x2: `open/claude-code-llm-security` (pre-split name) in V3-ANNOUNCEMENT.md updated to `open/llm-security`. Same dead name found in sarif-formatter.mjs TOOL_URI, which ships into SARIF output consumed by other tools, so it is corrected too. Left standing, deliberately: - WARN README-H1 `# LLM Security Plugin for Claude Code` != `# llm-security`. The gate defers this to the operator, and the description thread it protects is now carried by the opening line instead. Suite 2034/2034. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016h37aUdBLVDT9xrvG9osA9
6.6 KiB
6.6 KiB
LLM Security Plugin v3.0.0 — Release Notes
Security scanning, auditing, and threat modeling for Claude Code.
What's New in v3
Eight development sessions (v2.5 -> v3.0) adding six major capabilities:
| Capability | What It Does |
|---|---|
| Toxic flow analysis | Detects lethal trifecta patterns — when combinations of individually safe tools create exfiltration chains (untrusted input + sensitive data access + exfiltration sink) |
| Runtime session guard | Monitors tool call sequences in real time for trifecta forming during a session. Sliding window of 20 calls, advisory warning |
| MCP live inspection | Connects to running MCP servers via JSON-RPC 2.0, fetches live tool descriptions, scans for injection and tool shadowing |
| Report diffing | Compares scans against stored baselines. Shows new, resolved, unchanged, and moved findings with fuzzy line matching |
| Continuous scanning | Recurring scans via /security watch (in-session) or watch-cron.mjs (system cron). Reports delta only |
| Skill signature registry | SHA-256 fingerprinting of scanned skills. Instant results for known fingerprints, 7-day cache |
Plus: 4 OWASP framework coverage (LLM Top 10, Agentic AI, Skills, MCP), MEDIUM-severity injection patterns, architecture diagram.
By the Numbers
| v1.0 | v2.0 | v3.0 | |
|---|---|---|---|
| Commands | 8 | 10 | 14 |
| Hooks | 4 | 5 | 8 |
| Scanners | 0 | 7 | 10 (8 orchestrated + 2 standalone) |
| Agents | 4 | 4 | 6 |
| Knowledge files | 6 | 6 | 9 |
| Tests | 0 | 177 | 544 |
| OWASP frameworks | 2 | 2 | 4 |
| Lines of code | ~3,000 | ~8,000 | ~25,400 |
Competitive Landscape
Feature Comparison
| Feature | llm-security v3 | Snyk Agent Scan | Lasso Claude Hooks |
|---|---|---|---|
| Type | Plugin (hooks + scanners + agents) | CLI scanner | Runtime hook |
| Runtime blocking | 5 blocking hooks | No | Warn-only |
| Runtime monitoring | 3 advisory hooks | No | 1 advisory hook (96 patterns) |
| Deterministic scanners | 10 (8 orchestrated + 2 standalone) | Hybrid (local + cloud API) | None |
| LLM agents | 6 purpose-built | Cloud-side analysis | None |
| MCP scanning | Static + live (JSON-RPC 2.0) | Live connection | Output monitoring only |
| Skills scanning | 7 threat categories | Yes (--skills flag) | No |
| Toxic flow analysis | Yes (TFA scanner + session guard) | Yes | No |
| Supply chain | 7 package managers + OSV.dev + typosquat | Partial (URL detection) | No |
| Prompt injection | 44 patterns + obfuscation decode | Semantic (cloud) | 96 regex patterns |
| Secret detection | 13 patterns (blocking) | Yes (hardcoded only) | No |
| Path guarding | 8 categories (blocking) | No | No |
| Destructive command blocking | 8 block + 6 warn rules | No | No |
| Report diffing | Yes (baselines, fuzzy match) | No | No |
| Continuous scanning | Yes (watch + cron) | Enterprise only (Snyk Evo) | Always-on (hooks) |
| Skill registry | Yes (fingerprint cache) | No | No |
| Remediation | Auto/semi-auto/manual pipeline | No | No |
| Threat modeling | STRIDE x MAESTRO interview | No | No |
| Audit & posture | Full audit (A-F) + posture scorecard | No | No |
| Pre-deploy checklist | 10 auto + 3 manual checks | No | No |
| OWASP mapping | 4 frameworks (LLM, ASI, AST, MCP) | Not explicit | Not explicit |
| Agent auto-discovery | No (Claude Code focus) | 12 agents, 3 OS | No |
| Cloud dependency | None (100% local) | Snyk API required | None |
| API key required | No | Yes (Snyk token) | No |
| Cost | Free (MIT) | Snyk account | Free (MIT) |
| Tests | 544 | Not published | Not published |
Key Differentiators
-
Full lifecycle coverage. The only tool offering hooks (runtime) + scanners (static) + agents (advisory) + remediation + threat modeling in a single package. Snyk scans but doesn't block. Lasso blocks but doesn't scan.
-
100% local, zero cloud dependency. All analysis runs on the user's machine. No API keys, no telemetry, no data leaves the environment. Snyk requires a cloud API for semantic analysis.
-
Pre-extraction defense. When scanning untrusted remote repos, structured evidence is extracted and injection patterns stripped BEFORE LLM agents see the content. No other tool does this.
-
Supply chain depth. Covers 7 package managers (npm/yarn/pnpm, pip/pip3/uv, brew, docker, go, cargo, gem) with OSV.dev CVE checks, Levenshtein typosquatting, age-gating (<72h packages), and npm audit integration. Snyk has URL detection. Lasso has none.
-
Remediation pipeline. Three-tier approach: deterministic auto-fix, LLM-generated proposals with user confirmation, and manual findings in report. Neither Snyk nor Lasso offer remediation.
-
Cross-scanner correlation. Toxic flow analyzer consumes output from all prior scanners to detect lethal trifecta patterns that no single scanner would flag. Runtime session guard does the same in real time.
Complementary Usage
These tools are not mutually exclusive:
- llm-security + Lasso: Both hook systems run sequentially. Lasso adds 96 PostToolUse patterns focused on indirect injection; llm-security adds pre-tool blocking and broader coverage. No conflicts.
- llm-security + Snyk: Snyk's cloud-side semantic analysis and 12-agent auto-discovery complement the local deterministic scanning. Use both for maximum coverage.
- llm-security + parry-guard: ML-based injection classification (DeBERTa/Llama Guard) catches novel phrasings that regex patterns miss. Different layers, no overlap.
Demo Scenario
Scanning an untrusted plugin before installation
# 1. Quick scan — is this safe to install?
/security scan https://github.com/example/cool-plugin --deep
# Result: BLOCK 72/100
# - 3 CRITICAL: prompt injection in skill instructions
# - 2 HIGH: typosquatting packages (lodsh, axois)
# - 1 HIGH: toxic flow — Read(~/.ssh) -> Bash(curl)
# - 4 MEDIUM: undisclosed outbound URLs
# 2. Want to see what changed since last scan?
/security diff path/to/project
# Result: 2 NEW findings, 1 RESOLVED, 12 UNCHANGED
# 3. Set up continuous monitoring
/security watch path/to/project --interval 6h
# Reports delta every 6 hours via /loop
Installation
git clone https://git.fromaitochitta.com/open/llm-security.git \
~/.claude/plugins/llm-security
Hooks activate immediately. No configuration required.
Links
- Source: git.fromaitochitta.com/open/llm-security
- Full README: See README.md
- Changelog: See CHANGELOG.md
- License: MIT