feat(llm-security-copilot): port llm-security v5.1.0 to GitHub Copilot CLI
Full port of llm-security plugin for internal use on Windows with GitHub Copilot CLI. Protocol translation layer (copilot-hook-runner.mjs) normalizes Copilot camelCase I/O to Claude Code snake_case format — all original hook scripts run unmodified. - 8 hooks with protocol translation (stdin/stdout/exit code) - 18 SKILL.md skills (Agent Skills Open Standard) - 6 .agent.md agent definitions - 20 scanners + 14 scanner lib modules (unchanged) - 14 knowledge files (unchanged) - 39 test files including copilot-port-verify.mjs (17 tests) - Windows-ready: node:path, os.tmpdir(), process.execPath, no bash Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
901bf0ae12
commit
f418a8fe08
169 changed files with 37631 additions and 0 deletions
46
plugins/llm-security-copilot/skills/audit/SKILL.md
Normal file
46
plugins/llm-security-copilot/skills/audit/SKILL.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
name: security-audit
|
||||
description: Full project security audit with OWASP LLM Top 10 assessment, scoring, and remediation plan
|
||||
---
|
||||
|
||||
# Security Audit
|
||||
|
||||
Full project audit combining deterministic scanning with threat analysis.
|
||||
|
||||
## Step 1: Run Posture Scanner
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/posture-scanner.mjs
|
||||
```
|
||||
|
||||
Parse JSON: grade A-F, risk score, categories, findings, counts.
|
||||
|
||||
## Step 2: Gather Context
|
||||
|
||||
Glob for project structure: commands, agents, hooks, skills, MCP configs, CLAUDE.md, settings files. Record what exists.
|
||||
|
||||
## Step 3: Skill/Command Analysis
|
||||
|
||||
Read `<plugin-root>/knowledge/skill-threat-patterns.md` and `<plugin-root>/knowledge/secrets-patterns.md`.
|
||||
|
||||
Scan all command, agent, hook, and skill files for the 7 threat categories (injection, exfiltration, privilege escalation, scope creep, hidden instructions, toolchain manipulation, persistence). Apply frontmatter analysis, content analysis, and cross-reference checks.
|
||||
|
||||
## Step 4: MCP Analysis (if MCP configs found)
|
||||
|
||||
Read `<plugin-root>/knowledge/mcp-threat-patterns.md`.
|
||||
|
||||
Scan MCP server configs for: tool poisoning, hidden instructions, rug pull signals, supply chain risks, permission surface.
|
||||
|
||||
## Step 5: Merge and Report
|
||||
|
||||
Merge posture scanner findings with threat analysis findings. Recalculate:
|
||||
`risk_score = min(100, critical*25 + high*10 + medium*4 + low*1)`
|
||||
|
||||
Output:
|
||||
1. **Risk Dashboard** — Grade, risk score, verdict, finding counts
|
||||
2. **Executive Summary** — Key risks, posture overview
|
||||
3. **10 Category Sections** — One per OWASP/posture category with PASS/PARTIAL/FAIL and findings
|
||||
4. **Summary Table** — All findings sorted by severity
|
||||
5. **Action Items** — Grouped by urgency: IMMEDIATE (critical), HIGH (high), MEDIUM (medium)
|
||||
|
||||
If grade C or lower: suggest running `threat-model` for deeper analysis.
|
||||
58
plugins/llm-security-copilot/skills/clean/SKILL.md
Normal file
58
plugins/llm-security-copilot/skills/clean/SKILL.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
name: security-clean
|
||||
description: Scan and remediate security findings — auto-fixes deterministic issues, confirms semi-auto with user, reports manual findings
|
||||
---
|
||||
|
||||
# Security Clean
|
||||
|
||||
Scan, classify, and remediate security findings with user confirmation.
|
||||
|
||||
## Step 1: Parse Arguments
|
||||
|
||||
- Target path = `$ARGUMENTS` or current working directory
|
||||
- `--dry-run` flag = report only, no changes
|
||||
|
||||
## Step 2: Create Backup
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/fs-utils.mjs backup "<target>"
|
||||
```
|
||||
|
||||
## Step 3: Run Scan
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/fs-utils.mjs tmppath clean-findings.json
|
||||
node <plugin-root>/scanners/scan-orchestrator.mjs "<target>" --output-file "<findings_file>"
|
||||
```
|
||||
|
||||
Show banner with verdict, risk score, finding counts.
|
||||
|
||||
## Step 4: Auto-fix Deterministic Issues
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/auto-cleaner.mjs "<target>" --findings "<findings_file>" [--dry-run]
|
||||
```
|
||||
|
||||
Report: Applied, Skipped, Failed counts.
|
||||
|
||||
## Step 5: Semi-auto Remediation
|
||||
|
||||
For findings classified as semi-auto (entropy strings, permission mismatches, typosquatted deps, ghost hooks, suspicious URLs, credential access, hidden MCP directives, homoglyphs):
|
||||
|
||||
1. Read the referenced files and understand the surrounding context
|
||||
2. Propose specific, minimal changes grouped by fix type
|
||||
3. Present each proposal to the user for confirmation before applying
|
||||
4. Apply confirmed changes via Edit tool
|
||||
|
||||
## Step 6: LLM Threat Scan
|
||||
|
||||
Read `<plugin-root>/knowledge/skill-threat-patterns.md`. Scan modified files for remaining threats. Report manual findings that require human judgment.
|
||||
|
||||
## Step 7: Validate and Report
|
||||
|
||||
Re-scan to verify fixes didn't introduce new issues. If validation fails, offer to restore from backup:
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/fs-utils.mjs restore "<target>"
|
||||
```
|
||||
|
||||
Final report: pre/post comparison, fix summaries, remaining manual findings, rollback instructions.
|
||||
44
plugins/llm-security-copilot/skills/dashboard/SKILL.md
Normal file
44
plugins/llm-security-copilot/skills/dashboard/SKILL.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
name: security-dashboard
|
||||
description: Cross-project security dashboard — machine-wide posture overview with per-project grades
|
||||
---
|
||||
|
||||
# Security Dashboard
|
||||
|
||||
Machine-wide posture overview. Scans all detected projects, aggregates to a single machine grade (weakest link).
|
||||
|
||||
## Step 1: Run Aggregator
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/dashboard-aggregator.mjs [--no-cache]
|
||||
```
|
||||
|
||||
Uses 24h cache by default. Pass `--no-cache` to force fresh scan.
|
||||
|
||||
## Step 2: Format Dashboard
|
||||
|
||||
Parse JSON: `meta`, `machine.grade` (weakest link), `projects[]`, `errors[]`.
|
||||
|
||||
```
|
||||
# Security Dashboard
|
||||
|
||||
| Machine Grade | [A-F] (weakest link) |
|
||||
| Projects Scanned | N |
|
||||
| Cached | Yes/No |
|
||||
|
||||
## Per-Project Grades
|
||||
|
||||
| Project | Grade | Risk Score | Verdict | Last Scan |
|
||||
|---------|-------|------------|---------|-----------|
|
||||
[sorted by grade, worst first]
|
||||
|
||||
## Errors (if any)
|
||||
|
||||
[List projects that failed to scan]
|
||||
```
|
||||
|
||||
## Step 3: Recommendations
|
||||
|
||||
- Grade A/B: "Machine posture is strong."
|
||||
- Grade C: "Review projects with grade C or below."
|
||||
- Grade D/F: "Significant exposure. Run audit on failing projects."
|
||||
42
plugins/llm-security-copilot/skills/deep-scan/SKILL.md
Normal file
42
plugins/llm-security-copilot/skills/deep-scan/SKILL.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
---
|
||||
name: security-deep-scan
|
||||
description: Run deterministic deep-scan — 10 Node.js scanners for Unicode attacks, entropy analysis, permission mapping, dependency auditing, taint tracing, git forensics, network mapping, memory poisoning, supply chain recheck, and toxic flow analysis
|
||||
---
|
||||
|
||||
# Deep Scan
|
||||
|
||||
10 deterministic Node.js scanners — entropy, Unicode, typosquatting, git forensics, taint tracing, dep audit, network mapping, memory poisoning, supply chain recheck, toxic flow analysis.
|
||||
|
||||
## Step 1: Setup
|
||||
|
||||
- If `$ARGUMENTS` is empty, target = current working directory. Otherwise target = `$ARGUMENTS` (strip `--deep` if present).
|
||||
- Create a temporary file path for results (e.g. using `node -p "require('path').join(require('os').tmpdir(), 'deep-scan-results.json')"`).
|
||||
|
||||
## Step 2: Run Orchestrator
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/scan-orchestrator.mjs "<target>" --output-file "<results_file>"
|
||||
```
|
||||
|
||||
Exit codes: 0=ALLOW, 1=WARNING, 2=BLOCK. Stdout = compact aggregate JSON. Full results in file.
|
||||
|
||||
## Step 3: Show Banner
|
||||
|
||||
```
|
||||
## Deep Scan: [VERDICT]
|
||||
Risk Score: X/100 | Findings: XC XH XM XL XI
|
||||
Scanners: X ok, X error, X skipped
|
||||
```
|
||||
|
||||
## Step 4: Synthesize Report
|
||||
|
||||
Read the full results from `<results_file>`. Also read `<plugin-root>/knowledge/mitigation-matrix.md` for remediation context.
|
||||
|
||||
Produce a complete report with:
|
||||
1. **Executive Summary** — 3-5 sentences: posture assessment, dominant issue themes, intent assessment
|
||||
2. **Per-Scanner Details** — Group findings by severity (CRITICAL first). Highlight important findings, explain implications.
|
||||
3. **Toxic Flow Analysis** — If toxic-flow findings exist, show the trifecta chain legs (Input, Access, Exfil) with evidence
|
||||
4. **Recommendations** — Prioritized by urgency. Include finding IDs and actionable fix steps.
|
||||
5. **OWASP Coverage** — Map findings to OWASP LLM Top 10 and Agentic AI Top 10 categories.
|
||||
|
||||
Do NOT invent findings. Do NOT downplay CRITICAL or HIGH severity issues.
|
||||
43
plugins/llm-security-copilot/skills/diff/SKILL.md
Normal file
43
plugins/llm-security-copilot/skills/diff/SKILL.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
name: security-diff
|
||||
description: Compare scan results against a stored baseline — shows new, resolved, unchanged, and moved findings
|
||||
---
|
||||
|
||||
# Security Diff
|
||||
|
||||
Compare current scan against a stored baseline to track security changes over time.
|
||||
|
||||
## Step 1: Check Baseline
|
||||
|
||||
Check if a baseline exists:
|
||||
```bash
|
||||
ls <plugin-root>/reports/baselines/
|
||||
```
|
||||
|
||||
If no baselines exist, this is a first run — the scan will create the initial baseline.
|
||||
|
||||
## Step 2: Run Scan with Baseline
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/scan-orchestrator.mjs "<target>" --baseline --save-baseline
|
||||
```
|
||||
|
||||
Target = `$ARGUMENTS` or current working directory.
|
||||
|
||||
## Step 3: Display Results
|
||||
|
||||
**First run (no prior baseline):** Show baseline created summary with finding counts.
|
||||
|
||||
**Subsequent runs:** Parse diff output showing:
|
||||
- **New** findings (not in baseline)
|
||||
- **Resolved** findings (in baseline but not current)
|
||||
- **Moved** findings (same finding, different location)
|
||||
- **Unchanged** findings (same as baseline)
|
||||
|
||||
Format as tables for each category.
|
||||
|
||||
## Step 4: Advisory
|
||||
|
||||
- All resolved, no new: "Security improving."
|
||||
- New findings detected: "Regression detected. Review new findings."
|
||||
- No changes: "Stable. No security changes since last baseline."
|
||||
36
plugins/llm-security-copilot/skills/harden/SKILL.md
Normal file
36
plugins/llm-security-copilot/skills/harden/SKILL.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
name: security-harden
|
||||
description: Generate Grade A security configuration — settings, instructions, .gitignore additions
|
||||
---
|
||||
|
||||
# Security Harden
|
||||
|
||||
Generate reference security configuration based on current posture gaps.
|
||||
|
||||
## Step 1: Generate Recommendations
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/reference-config-generator.mjs $ARGUMENTS
|
||||
```
|
||||
|
||||
If `$ARGUMENTS` is empty, scan current working directory. Parse JSON output: `projectType`, `posture`, `recommendations[]`, `summary`.
|
||||
|
||||
## Step 2: Present Recommendations
|
||||
|
||||
Show table of recommended changes with: file, change description, current state, recommended state.
|
||||
|
||||
## Step 3: Apply (if confirmed)
|
||||
|
||||
If user confirms (or `--apply` flag is set):
|
||||
```bash
|
||||
node <plugin-root>/scanners/reference-config-generator.mjs $ARGUMENTS --apply
|
||||
```
|
||||
|
||||
## Step 4: Verify
|
||||
|
||||
Re-run posture scanner to verify grade improvement:
|
||||
```bash
|
||||
node <plugin-root>/scanners/posture-scanner.mjs $ARGUMENTS
|
||||
```
|
||||
|
||||
Report: Grade before → Grade after. If still below A, explain remaining gaps that require manual configuration (hooks, custom settings).
|
||||
47
plugins/llm-security-copilot/skills/mcp-audit/SKILL.md
Normal file
47
plugins/llm-security-copilot/skills/mcp-audit/SKILL.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
---
|
||||
name: security-mcp-audit
|
||||
description: Audit all installed MCP server configurations for security risks, trust verification, and permission analysis
|
||||
---
|
||||
|
||||
# MCP Audit
|
||||
|
||||
Comprehensive audit of all installed MCP server configurations.
|
||||
|
||||
## Step 1: Parse Arguments
|
||||
|
||||
Check for `--live` flag in `$ARGUMENTS`.
|
||||
|
||||
## Step 2: Discover MCP Configs
|
||||
|
||||
Search these locations for MCP server configurations:
|
||||
- `.mcp.json` in project root
|
||||
- `.vscode/mcp.json`
|
||||
- Settings files with `mcpServers` sections
|
||||
- Global MCP configuration files
|
||||
|
||||
## Step 3: Analyze Each Server
|
||||
|
||||
Read `<plugin-root>/knowledge/mcp-threat-patterns.md`.
|
||||
|
||||
For each discovered MCP server, perform 5-phase analysis:
|
||||
1. **Tool Description Analysis** — Check for hidden instructions, excessive length (>500 chars), Unicode anomalies, dynamic description loading
|
||||
2. **Source Code Analysis** — Code execution (eval/exec), network calls, file system access, credential access, time-conditional behavior
|
||||
3. **Dependency Analysis** — Run `npm audit` or `pip audit` as appropriate. Check for typosquatting, suspicious packages
|
||||
4. **Configuration Analysis** — Permission surface, declared vs actual scope, auth configuration
|
||||
5. **Rug Pull Detection** — Dynamic tool metadata, config self-modification, remote flag control, self-update mechanisms
|
||||
|
||||
Trust rating per server: Trusted / Cautious / Untrusted / Dangerous.
|
||||
|
||||
## Step 4: Live Inspection (if --live)
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/mcp-live-inspect.mjs
|
||||
```
|
||||
|
||||
Connect to running MCP servers, scan live tool descriptions, detect injection and shadowing.
|
||||
|
||||
## Step 5: Report
|
||||
|
||||
Output: MCP Landscape Summary table, per-server trust rating, findings grouped by severity. Group servers into: Keep (Trusted) / Review (Cautious) / Remove (Untrusted/Dangerous).
|
||||
|
||||
If static finding + live injection on same server = CRITICAL escalation.
|
||||
43
plugins/llm-security-copilot/skills/mcp-inspect/SKILL.md
Normal file
43
plugins/llm-security-copilot/skills/mcp-inspect/SKILL.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
name: security-mcp-inspect
|
||||
description: Connect to running MCP servers and scan live tool descriptions for injection, shadowing, and drift
|
||||
---
|
||||
|
||||
# MCP Inspect
|
||||
|
||||
Live MCP server inspection — connect to running servers, scan tool descriptions.
|
||||
|
||||
## Step 1: Run Inspector
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/mcp-live-inspect.mjs $ARGUMENTS
|
||||
```
|
||||
|
||||
Supports `--timeout <ms>` and `--skip-global` flags.
|
||||
|
||||
## Step 2: Format Results
|
||||
|
||||
Parse JSON output. Display:
|
||||
|
||||
```
|
||||
# MCP Live Inspection: [VERDICT]
|
||||
|
||||
Servers: N discovered, N contacted, N timed-out, N failed
|
||||
|
||||
## Server Details
|
||||
|
||||
| Server | Transport | Status | Tools | Findings |
|
||||
|--------|-----------|--------|-------|----------|
|
||||
[per-server row]
|
||||
|
||||
## Findings
|
||||
|
||||
[Findings table: severity, server, tool, description, evidence]
|
||||
```
|
||||
|
||||
## Step 3: Advisory
|
||||
|
||||
- Timeouts: "Server did not respond within timeout. May be SSE-based (unsupported for live inspection) or not running."
|
||||
- No servers found: "No MCP servers detected. Check configuration."
|
||||
- Clean: "All servers passed live inspection."
|
||||
- Findings: "Review findings. Combine with `mcp-audit` for static analysis."
|
||||
57
plugins/llm-security-copilot/skills/plugin-audit/SKILL.md
Normal file
57
plugins/llm-security-copilot/skills/plugin-audit/SKILL.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
---
|
||||
name: security-plugin-audit
|
||||
description: Audit a plugin for security risks, permission analysis, and trust assessment before installation
|
||||
---
|
||||
|
||||
# Plugin Audit
|
||||
|
||||
Dedicated plugin security audit with trust verdict. Accepts local paths or repository URLs.
|
||||
|
||||
## Step 1: Resolve Target
|
||||
|
||||
- If `$ARGUMENTS` starts with `https://` → clone the repo:
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/git-clone.mjs clone "<url>" [--branch <branch>]
|
||||
```
|
||||
Set `clone_path` and `target`.
|
||||
- Otherwise → `target = $ARGUMENTS` or current directory
|
||||
|
||||
## IMPORTANT: Cleanup Guarantee (remote)
|
||||
|
||||
If cloned, cleanup MUST run regardless of outcome:
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/git-clone.mjs cleanup "<clone_path>"
|
||||
```
|
||||
|
||||
## Step 2: Pre-extraction (remote only)
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/fs-utils.mjs tmppath "plugin-extract.json"
|
||||
node <plugin-root>/scanners/content-extractor.mjs "<target>" --output-file "<evidence_file>"
|
||||
```
|
||||
|
||||
## Step 3: Inventory
|
||||
|
||||
Read plugin manifest (plugin.json). Glob for all components: commands, agents, hooks, skills, MCP configs. Build permission matrix: what tools each component can access.
|
||||
|
||||
## Step 4: Security Analysis
|
||||
|
||||
Read `<plugin-root>/knowledge/skill-threat-patterns.md` and `<plugin-root>/knowledge/secrets-patterns.md`.
|
||||
|
||||
Analyze all files for 7 threat categories. If remote, analyze from evidence package (never read untrusted files directly). Check hook registration vs scripts (ghost hooks). Cross-reference description vs tools permissions.
|
||||
|
||||
## Step 5: Report
|
||||
|
||||
Output: Plugin metadata, component inventory, permission matrix, hook analysis, all findings by severity.
|
||||
|
||||
**Trust Verdict:**
|
||||
- **Install** — No critical or high findings, permissions appropriate
|
||||
- **Review** — High findings present but potentially justified, manual review recommended
|
||||
- **Do Not Install** — Critical findings, active exfiltration, or injection detected
|
||||
|
||||
## Step 6: Cleanup
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/git-clone.mjs cleanup "<clone_path>"
|
||||
node <plugin-root>/scanners/lib/fs-utils.mjs cleanup "<evidence_file>"
|
||||
```
|
||||
54
plugins/llm-security-copilot/skills/posture/SKILL.md
Normal file
54
plugins/llm-security-copilot/skills/posture/SKILL.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
name: security-posture
|
||||
description: Quick security posture assessment — scorecard with grade, coverage status, and top recommendations
|
||||
---
|
||||
|
||||
# Security Posture
|
||||
|
||||
Quick security scorecard — grade, coverage, top recommendations. Deterministic scanner, <2 sec.
|
||||
|
||||
## Step 1: Resolve Plugin Root
|
||||
|
||||
Plugin root = the directory containing `plugin.json`, found by searching up from this file's location.
|
||||
|
||||
## Step 2: Run Scanner
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/posture-scanner.mjs $ARGUMENTS
|
||||
```
|
||||
|
||||
If `$ARGUMENTS` is empty, scan the current working directory.
|
||||
|
||||
Parse the JSON output: `scoring.grade` (A-F), `scoring.pass_rate`, `risk.score` (0-100), `risk.band`, `risk.verdict`, `categories[]`, `findings[]`, `counts`.
|
||||
|
||||
## Step 3: Format Scorecard
|
||||
|
||||
```
|
||||
# Security Posture — [project name]
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Grade** | [A-F] |
|
||||
| **Risk Score** | [N]/100 ([band]) |
|
||||
| **Verdict** | [verdict] |
|
||||
|
||||
## Category Scorecard
|
||||
|
||||
| # | Category | Status | Findings |
|
||||
|---|----------|--------|----------|
|
||||
[one row per category, status as PASS/PARTIAL/FAIL/N-A]
|
||||
|
||||
## Top Findings
|
||||
|
||||
[List critical and high findings with title and recommendation]
|
||||
|
||||
## Quick Wins
|
||||
|
||||
[List low-effort fixes from findings]
|
||||
```
|
||||
|
||||
## Step 4: Closing
|
||||
|
||||
- Grade A/B: "Posture solid. Re-run after major changes."
|
||||
- Grade C: "Run the audit skill for detailed findings."
|
||||
- Grade D/F: "Significant exposure. Run audit before production use."
|
||||
38
plugins/llm-security-copilot/skills/pre-deploy/SKILL.md
Normal file
38
plugins/llm-security-copilot/skills/pre-deploy/SKILL.md
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
---
|
||||
name: security-pre-deploy
|
||||
description: Pre-deployment security checklist — verify controls, compliance, and production readiness
|
||||
---
|
||||
|
||||
# Pre-Deploy Checklist
|
||||
|
||||
10 automated checks + 3 manual verification questions.
|
||||
|
||||
## Automated Checks (PASS/FAIL/WARN/N-A)
|
||||
|
||||
Perform these checks using Read, Glob, and Grep:
|
||||
|
||||
1. **Deny-first permissions** — Settings contain restrictive default permissions
|
||||
2. **Secrets hook active** — pre-edit-secrets hook is registered and script exists
|
||||
3. **Path guard active** — pre-write-pathguard hook is registered and script exists
|
||||
4. **Destructive command guard** — pre-bash-destructive hook registered and script exists
|
||||
5. **MCP servers verified** — All configured MCP servers have been audited
|
||||
6. **No hardcoded secrets** — Grep for API keys, tokens, passwords in source files
|
||||
7. **Gitignore covers secrets** — .gitignore includes .env, credentials, keys
|
||||
8. **Security instructions present** — CLAUDE.md or copilot-instructions.md has security section
|
||||
9. **Sandbox enabled** — No `dangerouslySkipPermissions` or bypass flags
|
||||
10. **Audit logging** — Post-tool hooks are active for monitoring
|
||||
|
||||
## Manual Verification
|
||||
|
||||
Ask the user these questions:
|
||||
1. "Is this running on an Enterprise plan with audit logging?"
|
||||
2. "Has a Data Protection Impact Assessment (DPIA) been completed?"
|
||||
3. "Is there an incident response plan for AI security events?"
|
||||
|
||||
## Report
|
||||
|
||||
Present as checklist table. Verdict by pass count:
|
||||
- 10/10: Low risk — Ready to deploy
|
||||
- 7-9: Medium risk — Address gaps first
|
||||
- 4-6: High risk — Significant exposure
|
||||
- 0-3: Extreme risk — Do not deploy
|
||||
54
plugins/llm-security-copilot/skills/red-team/SKILL.md
Normal file
54
plugins/llm-security-copilot/skills/red-team/SKILL.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
name: security-red-team
|
||||
description: Attack simulation — test hook defenses with crafted payloads across 12 categories
|
||||
---
|
||||
|
||||
# Red Team
|
||||
|
||||
Attack simulation testing hook defenses with crafted payloads. 64 scenarios across 12 categories.
|
||||
|
||||
## Step 1: Parse Arguments
|
||||
|
||||
- `--category <name>` — Run only one category
|
||||
- `--verbose` — Show individual scenario results
|
||||
- `--adaptive` — Enable mutation-based evasion testing (5 rounds per passing scenario)
|
||||
- `--json` — Raw JSON output
|
||||
|
||||
## Step 2: Run Simulator
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/attack-simulator.mjs [--category <name>] [--verbose] [--adaptive] [--json]
|
||||
```
|
||||
|
||||
## Step 3: Narrative Report
|
||||
|
||||
For each category, explain: what was tested, how many scenarios passed (blocked correctly), what gaps exist.
|
||||
|
||||
**Categories (12):**
|
||||
|
||||
| Category | Hook Tested | Scenarios |
|
||||
|----------|------------|-----------|
|
||||
| secrets | pre-edit-secrets | Multiple |
|
||||
| destructive | pre-bash-destructive | Multiple |
|
||||
| supply-chain | pre-install-supply-chain | Multiple |
|
||||
| prompt-injection | pre-prompt-inject-scan | Multiple |
|
||||
| pathguard | pre-write-pathguard | Multiple |
|
||||
| mcp-output | post-mcp-verify | Multiple |
|
||||
| session-trifecta | post-session-guard | Multiple |
|
||||
| hybrid | Multiple hooks | Multiple |
|
||||
| unicode-evasion | pre-prompt-inject-scan | Multiple |
|
||||
| bash-evasion | pre-bash-destructive | Multiple |
|
||||
| hitl-traps | post-mcp-verify | Multiple |
|
||||
| long-horizon | post-session-guard | Multiple |
|
||||
|
||||
## Step 4: Defense Score
|
||||
|
||||
- 100%: All scenarios correctly blocked
|
||||
- 90-99%: Minor gaps, review failing scenarios
|
||||
- <90%: Significant gaps, prioritize fixes
|
||||
|
||||
## Step 5: Adaptive Results (if --adaptive)
|
||||
|
||||
Mutation types: homoglyph substitution, encoding variants, zero-width insertion, case alternation, synonym replacement. Expected bypass rate varies by category.
|
||||
|
||||
**Safety:** No real exploits executed. No network calls. No file modifications. All payloads are synthetic test data.
|
||||
53
plugins/llm-security-copilot/skills/registry/SKILL.md
Normal file
53
plugins/llm-security-copilot/skills/registry/SKILL.md
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
---
|
||||
name: security-registry
|
||||
description: Skill signature registry — view stats, scan and register skills, search known fingerprints
|
||||
---
|
||||
|
||||
# Skill Registry
|
||||
|
||||
Track scanned skills by fingerprint. Three sub-commands.
|
||||
|
||||
## Sub-command: Stats (default)
|
||||
|
||||
```bash
|
||||
node -e "
|
||||
import { getStats, listEntries } from '<plugin-root>/scanners/lib/skill-registry.mjs';
|
||||
const stats = getStats('<plugin-root>');
|
||||
const entries = listEntries('<plugin-root>', 5);
|
||||
console.log(JSON.stringify({ stats, entries }));
|
||||
" --input-type=module
|
||||
```
|
||||
|
||||
Show: total entries, last scan date, 5 most recent entries.
|
||||
|
||||
## Sub-command: Scan
|
||||
|
||||
```bash
|
||||
node -e "
|
||||
import { fingerprintSkill, checkRegistry } from '<plugin-root>/scanners/lib/skill-registry.mjs';
|
||||
const r = fingerprintSkill('<target>');
|
||||
const c = checkRegistry(r.fingerprint, '<plugin-root>');
|
||||
console.log(JSON.stringify({ fingerprint: r.fingerprint, name: r.name, files: r.files, ...c }));
|
||||
" --input-type=module
|
||||
```
|
||||
|
||||
If cached and not stale: show cached result. If miss or stale: run full scan, then register:
|
||||
|
||||
```bash
|
||||
node -e "
|
||||
import { registerScan } from '<plugin-root>/scanners/lib/skill-registry.mjs';
|
||||
registerScan({ skillPath: '<target>', fingerprint: '<fp>', name: '<name>', files: <files>, verdict: '<v>', risk_score: <n>, counts: <counts>, files_scanned: <n> }, '<plugin-root>');
|
||||
" --input-type=module
|
||||
```
|
||||
|
||||
## Sub-command: Search
|
||||
|
||||
```bash
|
||||
node -e "
|
||||
import { searchRegistry } from '<plugin-root>/scanners/lib/skill-registry.mjs';
|
||||
const results = searchRegistry('<pattern>', '<plugin-root>');
|
||||
console.log(JSON.stringify(results));
|
||||
" --input-type=module
|
||||
```
|
||||
|
||||
Show matching entries with verdict, risk score, last scanned date.
|
||||
80
plugins/llm-security-copilot/skills/scan/SKILL.md
Normal file
80
plugins/llm-security-copilot/skills/scan/SKILL.md
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
---
|
||||
name: security-scan
|
||||
description: Scan files, directories, or repos for security issues — secrets, injection vulnerabilities, supply chain risks, OWASP LLM patterns
|
||||
---
|
||||
|
||||
# Security Scan
|
||||
|
||||
Scan target for security issues. Accepts local paths or repository URLs.
|
||||
|
||||
## Step 1: Resolve Target
|
||||
|
||||
- If `$ARGUMENTS` contains `--deep` → strip it, set `run_deep_scan = true`
|
||||
- If `$ARGUMENTS` contains `--branch <name>` → strip it, set `branch = <name>`
|
||||
- If `$ARGUMENTS` is empty → target = current working directory
|
||||
- If `$ARGUMENTS` starts with `https://` → clone the repo:
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/git-clone.mjs clone "<url>" [--branch <branch>]
|
||||
```
|
||||
Set `clone_path` = stdout (trimmed), `target = clone_path`
|
||||
- Otherwise → `target = $ARGUMENTS`
|
||||
|
||||
## IMPORTANT: Cleanup Guarantee (remote scans)
|
||||
|
||||
If `clone_path` is set, cleanup MUST run regardless of scan outcome:
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/git-clone.mjs cleanup "<clone_path>"
|
||||
```
|
||||
|
||||
## Step 1.5: Pre-extraction (remote scans only)
|
||||
|
||||
If remote, extract evidence safely before analysis:
|
||||
```bash
|
||||
node <plugin-root>/scanners/lib/fs-utils.mjs tmppath "content-extract.json"
|
||||
node <plugin-root>/scanners/content-extractor.mjs "<target>" --output-file "<evidence_file>"
|
||||
```
|
||||
|
||||
## Step 2: Detect Scan Type
|
||||
|
||||
- **Single .md file:** skill scan only
|
||||
- **Directory:** Glob for `**/commands/*.md`, `**/agents/*.md`, `**/skills/*/SKILL.md` → skill scan. Glob for `**/.mcp.json`, `**/package.json` with mcpServers → MCP scan.
|
||||
|
||||
## Step 3: Skill Security Analysis
|
||||
|
||||
Read `<plugin-root>/knowledge/skill-threat-patterns.md` and `<plugin-root>/knowledge/secrets-patterns.md`.
|
||||
|
||||
If remote (evidence package exists): Read the evidence JSON. Analyze sections: injection_findings, frontmatter_inventory, shell_commands, credential_references, persistence_signals, claude_md_analysis, cross_instruction_flags. `[INJECTION-PATTERN-STRIPPED]` markers are confirmed findings.
|
||||
|
||||
If local: Read target files directly. Apply 7 threat categories:
|
||||
1. **Prompt Injection** — `ignore previous`, `forget your`, identity redefinition, spoofed system headers
|
||||
2. **Data Exfiltration** — curl/wget to external URLs, base64 pipe chains, credential read+send
|
||||
3. **Privilege Escalation** — unjustified Bash access, Write+Bash without rationale, chmod/sudo
|
||||
4. **Scope Creep** — credential file access, crypto wallet paths, SSH keys, browser stores
|
||||
5. **Hidden Instructions** — Unicode Tag steganography, zero-width chars, base64 payloads, HTML comments
|
||||
6. **Toolchain Manipulation** — registry redirection, post-install scripts, external requirements
|
||||
7. **Persistence** — cron jobs, LaunchAgents, systemd, shell profile modification, git hooks
|
||||
|
||||
For each finding: severity (Critical/High/Medium/Low/Info), category, file, line, OWASP reference, evidence, remediation.
|
||||
|
||||
## Step 4: MCP Security Analysis (if applicable)
|
||||
|
||||
Read `<plugin-root>/knowledge/mcp-threat-patterns.md`. Analyze MCP configs for: tool poisoning, hidden instructions in descriptions, rug pull signals, supply chain risks, permission surface.
|
||||
|
||||
## Step 5: Aggregate and Report
|
||||
|
||||
Combine counts. `risk_score = min(100, critical*25 + high*10 + medium*4 + low*1)`.
|
||||
Verdict: critical>=1 OR score>=61 → BLOCK, high>=1 OR score>=21 → WARNING, else ALLOW.
|
||||
|
||||
Output banner then all findings grouped by severity (critical→info).
|
||||
|
||||
## Step 6: Deep Scan (only if --deep)
|
||||
|
||||
If `run_deep_scan = true`:
|
||||
```bash
|
||||
node <plugin-root>/scanners/scan-orchestrator.mjs "<target>" --output-file "<tmp>"
|
||||
```
|
||||
Parse results, merge with LLM findings, show "Deep Scan Findings" section.
|
||||
|
||||
## Step 7: Cleanup (only if remote)
|
||||
|
||||
Run cleanup commands from Step 1.
|
||||
52
plugins/llm-security-copilot/skills/security/SKILL.md
Normal file
52
plugins/llm-security-copilot/skills/security/SKILL.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
name: security
|
||||
description: LLM Security — security scanning, auditing, and threat modeling for AI coding agent projects
|
||||
---
|
||||
|
||||
# LLM Security Plugin
|
||||
|
||||
Security scanning, auditing, and threat modeling. Based on OWASP LLM Top 10 (2025), Agentic AI Top 10, Skills Top 10, MCP Top 10, and DeepMind Agent Traps.
|
||||
|
||||
## Skills
|
||||
|
||||
| Skill | Description | When to use |
|
||||
|-------|-------------|-------------|
|
||||
| `scan [path\|url]` | Scan skills, MCP servers, directories, or repos | Before installing a new skill or MCP server |
|
||||
| `scan [path\|url] --deep` | Enhanced scan: LLM + 10 deterministic scanners | Deep analysis with entropy, taint, git forensics |
|
||||
| `deep-scan [path]` | 10 deterministic scanners only (no LLM agents) | Fast, reproducible, deterministic-only analysis |
|
||||
| `audit` | Full project security audit with A-F grading | Periodic review (monthly recommended) |
|
||||
| `plugin-audit [path\|url]` | Plugin security audit with trust verdict | Before installing a third-party plugin |
|
||||
| `mcp-audit [--live]` | MCP server config audit | After adding MCP servers or on suspicion |
|
||||
| `mcp-inspect` | Live MCP server inspection | Verify running servers have safe tool descriptions |
|
||||
| `posture` | Quick security posture scorecard | Daily/weekly health check |
|
||||
| `threat-model` | Interactive STRIDE/MAESTRO threat modeling | When designing new architecture |
|
||||
| `diff [path]` | Compare scan against stored baseline | Track security changes over time |
|
||||
| `watch [path] [--interval 6h]` | Continuous monitoring — diff on interval | Ongoing security monitoring |
|
||||
| `clean [path]` | Scan and remediate findings | Sanitizing third-party plugin before install |
|
||||
| `supply-check [path]` | Re-audit installed deps | Periodic dependency audit |
|
||||
| `dashboard` | Cross-project security dashboard | Machine-wide health check |
|
||||
| `harden [path]` | Generate Grade A security config | Hardening a new or existing project |
|
||||
| `red-team [--category]` | Attack simulation against hooks | Verify hooks block what they claim |
|
||||
| `pre-deploy` | Pre-deployment security checklist | Before pushing to production |
|
||||
| `registry [scan\|search]` | Skill signature registry | Track scanned skills |
|
||||
|
||||
## What's Protected (Hooks)
|
||||
|
||||
This plugin includes automated security hooks that run on every operation:
|
||||
|
||||
| Hook | Protects Against |
|
||||
|------|-----------------|
|
||||
| Secret detection | API keys, tokens, passwords in code |
|
||||
| Path guarding | Writes to .env, .ssh, credentials files |
|
||||
| Destructive command blocking | rm -rf, force push, pipe-to-shell |
|
||||
| Supply chain protection | Compromised packages, typosquats |
|
||||
| Prompt injection scanning | Injection attempts in user prompts |
|
||||
| MCP output verification | Data leakage in tool outputs |
|
||||
| Session monitoring | Lethal trifecta detection (untrusted input + sensitive data + exfil) |
|
||||
|
||||
## Quick Start
|
||||
|
||||
- **New to security?** Start with `posture` for a quick health check
|
||||
- **Evaluating a plugin?** Run `plugin-audit path/to/plugin`
|
||||
- **Full analysis?** Run `scan . --deep` to combine LLM + deterministic scanners
|
||||
- **Machine-wide overview?** Run `dashboard`
|
||||
34
plugins/llm-security-copilot/skills/supply-check/SKILL.md
Normal file
34
plugins/llm-security-copilot/skills/supply-check/SKILL.md
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
---
|
||||
name: security-supply-check
|
||||
description: Re-audit installed dependencies — check lockfiles against blocklists, OSV.dev CVEs, and typosquat detection
|
||||
---
|
||||
|
||||
# Supply Chain Check
|
||||
|
||||
Re-audit installed dependencies from lockfiles.
|
||||
|
||||
## Step 1: Run Scanner
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/supply-chain-recheck-cli.mjs $ARGUMENTS
|
||||
```
|
||||
|
||||
Target = `$ARGUMENTS` or current working directory. Checks: package-lock.json, yarn.lock, requirements.txt, Pipfile.lock against blocklists, OSV.dev batch API, and Levenshtein typosquat detection.
|
||||
|
||||
## Step 2: Format Results
|
||||
|
||||
Parse JSON output. Show:
|
||||
|
||||
```
|
||||
# Supply Chain Check: [STATUS]
|
||||
|
||||
Findings: XC XH XM XL | Lockfiles: N scanned
|
||||
|
||||
[If osv_offline: "OSV.dev unreachable — blocklist and typosquat checks ran, CVE checks skipped"]
|
||||
|
||||
## Findings
|
||||
|
||||
[Table: severity, package, ecosystem, type (blocklist/CVE/typosquat), description]
|
||||
```
|
||||
|
||||
If zero findings: "All dependencies pass supply chain checks."
|
||||
80
plugins/llm-security-copilot/skills/threat-model/SKILL.md
Normal file
80
plugins/llm-security-copilot/skills/threat-model/SKILL.md
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
---
|
||||
name: security-threat-model
|
||||
description: Interactive threat modeling using STRIDE and MAESTRO frameworks — guides architecture analysis and generates threat model document
|
||||
---
|
||||
|
||||
# Threat Model
|
||||
|
||||
Interactive STRIDE/MAESTRO threat modeling session. 15-30 minutes → complete threat model document.
|
||||
|
||||
## Principles
|
||||
|
||||
- Ask one question at a time
|
||||
- Not a rubber stamp — challenge assumptions
|
||||
- Cite OWASP IDs (LLM01-LLM10, ASI01-ASI10)
|
||||
- Distinguish theoretical threats from actively exploited ones
|
||||
- Advisory output only — no file modifications
|
||||
|
||||
## Phase 1: Architecture Discovery
|
||||
|
||||
Read knowledge files:
|
||||
- `<plugin-root>/knowledge/skill-threat-patterns.md`
|
||||
- `<plugin-root>/knowledge/mcp-threat-patterns.md`
|
||||
- `<plugin-root>/knowledge/mitigation-matrix.md`
|
||||
|
||||
Ask the user these questions (one at a time):
|
||||
1. What type of system is this? (plugin, MCP server, standalone agent, API service)
|
||||
2. What tools/MCP servers does the agent have access to? (file system, network, databases, external APIs)
|
||||
3. What data does the system handle? (credentials, PII, source code, business data)
|
||||
4. Who are the users and what's the trust model? (single developer, team, external users)
|
||||
5. How is it deployed? (local CLI, VS Code extension, cloud agent, CI/CD)
|
||||
|
||||
## Phase 2: Component Mapping
|
||||
|
||||
Map identified components to MAESTRO 7-Layer Model:
|
||||
- L1 Foundation Models — Base LLM capabilities
|
||||
- L2 Data Operations — RAG, embeddings, knowledge bases
|
||||
- L3 Agent Frameworks — Orchestration, tool routing
|
||||
- L4 Tool Ecosystem — MCP servers, API integrations
|
||||
- L5 Deployment — Runtime environment, containers
|
||||
- L6 Interaction — User interfaces, chat, CLI
|
||||
- L7 Ecosystem — Plugin marketplace, supply chain
|
||||
|
||||
Present the mapping for user confirmation.
|
||||
|
||||
## Phase 3: Threat Identification
|
||||
|
||||
Apply STRIDE (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege) to each relevant MAESTRO layer.
|
||||
|
||||
For each threat: state actor, method, asset at risk, impact, OWASP ID, whether it's known to be exploited in the wild.
|
||||
|
||||
## Phase 4: Risk Assessment
|
||||
|
||||
Rate each threat: Likelihood (1-5) x Impact (1-5) = Risk Score.
|
||||
- 20-25: Critical — Immediate action required
|
||||
- 12-19: High — Address within current sprint
|
||||
- 6-11: Medium — Plan remediation
|
||||
- 1-5: Low — Accept or monitor
|
||||
|
||||
Present the risk matrix for user validation.
|
||||
|
||||
## Phase 5: Mitigation Mapping
|
||||
|
||||
Read `<plugin-root>/knowledge/mitigation-matrix.md`. For each threat, classify the control status:
|
||||
- Already mitigated (existing hooks, settings)
|
||||
- Can be mitigated (configuration change or code fix)
|
||||
- Partially mitigated
|
||||
- Accepted risk (documented rationale)
|
||||
- External dependency
|
||||
|
||||
## Output: Threat Model Document
|
||||
|
||||
Produce an 8-section document:
|
||||
1. System Description
|
||||
2. Architecture Overview
|
||||
3. MAESTRO Layer Mapping
|
||||
4. Threat Catalog (per layer, with STRIDE/OWASP/likelihood/impact/scenario/control/recommendation)
|
||||
5. Risk Matrix
|
||||
6. Mitigation Plan (Critical+High actions, Already Mitigated, Accepted Risks)
|
||||
7. Residual Risk Summary
|
||||
8. Assumptions and Limitations
|
||||
39
plugins/llm-security-copilot/skills/watch/SKILL.md
Normal file
39
plugins/llm-security-copilot/skills/watch/SKILL.md
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
name: security-watch
|
||||
description: Continuous security monitoring — runs diff on a recurring interval
|
||||
---
|
||||
|
||||
# Security Watch
|
||||
|
||||
Set up continuous monitoring that runs security diffs on an interval.
|
||||
|
||||
## Step 1: Parse Arguments
|
||||
|
||||
- Target path = first positional argument or current directory
|
||||
- `--interval <value>` — monitoring interval (default: 6h). Supports: 30m, 1h, 6h, 12h, 24h.
|
||||
|
||||
## Step 2: Create Initial Baseline
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/scan-orchestrator.mjs "<target>" --save-baseline
|
||||
```
|
||||
|
||||
Display baseline summary: verdict, risk score, finding counts.
|
||||
|
||||
## Step 3: Start Monitoring
|
||||
|
||||
For continuous monitoring, use the watch-cron scanner:
|
||||
|
||||
```bash
|
||||
node <plugin-root>/scanners/watch-cron.mjs [--config <plugin-root>/reports/watch/config.json]
|
||||
```
|
||||
|
||||
Or set up a system cron job / scheduled task:
|
||||
- **Linux/macOS cron:** `0 */6 * * * node <plugin-root>/scanners/watch-cron.mjs`
|
||||
- **Windows Task Scheduler:** Create a task that runs the same command on your preferred interval
|
||||
|
||||
## Step 4: Advisory
|
||||
|
||||
- Results stored in `<plugin-root>/reports/watch/latest.json`
|
||||
- Use `security-diff` to manually compare against baseline at any time
|
||||
- Watch-cron overwrites latest.json on each run
|
||||
Loading…
Add table
Add a link
Reference in a new issue