Kjell Tore Guttormsen
965b1d9bca
The seven-category signal lists in agents/skill-scanner-agent.md move to knowledge/skill-threat-patterns.md § Detection Rules. The agent keeps its procedure, severity table, verdict logic and output format, plus a pointer, and now says to report an unreadable knowledge file instead of scanning from memory (a remembered subset reports clean for rules it never applied). Chose the existing file over a new one because every command that invokes the agent (scan, audit, clean, plugin-audit) already passes <plugin-root>/knowledge/skill-threat-patterns.md explicitly; a new file would have needed four command edits to reach the agent at all. Why (v8.1.0 AV surface): a clean SKILL.md was quarantined as Trojan:Script/Wacatac.H!ml; a quarantine on agents/*.md breaks the installed plugin, not just a clone. Move measured lossless: the only line-level differences are the two intended rewrites and headings. Also in this commit, rewritten as descriptions or with a <shell> placeholder for the interpreter (no technique removed): - posture-assessor-agent: hook-coverage item and override phrases - deep-scan-synthesizer-agent: example decoded message - commands/red-team.md: scenario table cell - knowledge/*.md: 15 runnable download-into-shell one-liners, the "Decodes to" line first; fenced examples keep their exact shape with <shell>, prose and tables become sentences. Probe (e): 8 -> 3 (the three left are hook-script lines behind the path guard). knowledge: 17 -> 2 (attack-scenarios.json, the red-team simulator's input; left on purpose). claude plugin validate . passes; agent frontmatter untouched, all six agents parse. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
115 lines
5.3 KiB
Markdown
115 lines
5.3 KiB
Markdown
---
|
|
name: llm-security:red-team
|
|
description: Attack simulation — test hook defenses with crafted payloads
|
|
allowed-tools: Bash, Read
|
|
model: sonnet
|
|
---
|
|
|
|
# Red Team — Attack Simulation
|
|
|
|
Run crafted attack payloads against the plugin's own hooks to verify defenses.
|
|
|
|
## What was requested
|
|
|
|
The user ran `/security red-team` to test their hook defenses.
|
|
|
|
## Arguments
|
|
|
|
Parse `$ARGUMENTS` for:
|
|
- `--category <name>` — filter: secrets, destructive, supply-chain, prompt-injection, pathguard, mcp-output, session-trifecta, hybrid, unicode-evasion, bash-evasion, hitl-traps, long-horizon, all
|
|
- `--json` — raw JSON output
|
|
- `--adaptive` — mutation-based evasion testing (5 mutation rounds per passing scenario)
|
|
|
|
Default: all categories, fixed mode.
|
|
|
|
## Steps
|
|
|
|
1. **Run the attack simulator:**
|
|
|
|
```
|
|
node scanners/attack-simulator.mjs [--category <name>] [--verbose] [--adaptive]
|
|
```
|
|
|
|
The simulator runs 64 attack scenarios across 12 categories against the plugin's hooks. Each scenario sends a crafted payload and verifies the hook blocks or detects it.
|
|
|
|
In **adaptive mode** (`--adaptive`), for each scenario that passes (attack blocked), the simulator applies 5 mutation rounds:
|
|
1. Homoglyph substitution (Latin chars replaced with Cyrillic lookalikes)
|
|
2. Encoding wrapping (URL-encoded keywords)
|
|
3. Zero-width character injection (ZW chars inserted between keyword letters)
|
|
4. Case alternation (aLtErNaTiNg case)
|
|
5. Synonym substitution (keyword replacement from synonym table)
|
|
|
|
Bypasses are reported as findings but not auto-fixed.
|
|
|
|
2. **Present the results as a narrative report:**
|
|
|
|
For each category, explain:
|
|
- What was tested (the attack type)
|
|
- How many attacks were blocked
|
|
- Whether defenses are adequate
|
|
|
|
If any scenarios fail, explain the gap and what hook needs attention.
|
|
|
|
In adaptive mode, also explain:
|
|
- How many mutations were tested
|
|
- Which mutations found bypasses
|
|
- That bypasses are expected for synonym and encoding mutations (deterministic hooks cannot catch all evasions)
|
|
|
|
3. **Defense Score interpretation:**
|
|
- **100%** — All hooks functioning correctly. No defense gaps.
|
|
- **90-99%** — Minor gaps. Review failed scenarios.
|
|
- **Below 90%** — Significant gaps. Hooks may be misconfigured or missing.
|
|
|
|
## Categories
|
|
|
|
| Category | Hook Tested | Scenarios |
|
|
|----------|------------|-----------|
|
|
| secrets | pre-edit-secrets.mjs | 7 secret types (AWS, GitHub, PEM, DB, Bearer, Azure, Slack) |
|
|
| destructive | pre-bash-destructive.mjs | 8 commands (rm -rf, chmod 777, pipe-to-shell, fork bomb, mkfs, dd, eval) |
|
|
| supply-chain | pre-install-supply-chain.mjs | 4 managers (npm, pip, cargo, gem) |
|
|
| prompt-injection | pre-prompt-inject-scan.mjs | 6 patterns (override, spoofed headers, identity, evasion) |
|
|
| pathguard | pre-write-pathguard.mjs | 6 paths (.env, .ssh, .aws, .npmrc, /etc, hooks) |
|
|
| mcp-output | post-mcp-verify.mjs | 4 threats (injection, secrets, HTML traps, MCP injection) |
|
|
| session-trifecta | post-session-guard.mjs | 3 patterns (classic trifecta, MCP-concentrated, volume) |
|
|
| hybrid | post-mcp-verify.mjs | 8 patterns (P2SQL, recursive injection, XSS variants) |
|
|
| unicode-evasion | pre-prompt-inject-scan.mjs | 6 patterns (Unicode Tags, ZW chars, homoglyphs, BIDI, HTML entities, multi-lang) |
|
|
| bash-evasion | pre-bash-destructive.mjs | 5 patterns (empty quotes, dollar expansion, backslash splitting, supply chain) |
|
|
| hitl-traps | post-mcp-verify.mjs | 4 patterns (approval urgency, summary suppression, scope minimization, cognitive load) |
|
|
| long-horizon | post-session-guard.mjs | 3 patterns (delegation-after-input, sensitive path, MCP-concentrated trifecta) |
|
|
|
|
## Mutation Types (Adaptive Mode)
|
|
|
|
| Mutation | Technique | Expected Bypass Rate |
|
|
|----------|-----------|---------------------|
|
|
| homoglyph | Cyrillic/Latin lookalike substitution | Low (MEDIUM patterns detect) |
|
|
| encoding | URL-encode keywords | High (hooks normalize some, not all) |
|
|
| zero_width | Insert zero-width chars in keywords | Low (normalizer strips these) |
|
|
| case_alternation | aLtErNaTiNg case | Low (regex uses /i flag) |
|
|
| synonym | Replace with semantic equivalents | Medium (novel synonyms evade patterns) |
|
|
|
|
## Important
|
|
|
|
- This tests the plugin's OWN hooks — it does not perform real exploits
|
|
- No network calls, no file modifications, no LLM invocations
|
|
- Safe to run repeatedly — all state is cleaned up after each run
|
|
- Adaptive mode bypasses are **expected** — they document evasion resistance limits
|
|
|
|
## HTML Report
|
|
|
|
After producing the markdown red-team narrative report above:
|
|
|
|
1. Compute a temp markdown path:
|
|
```bash
|
|
node -p "require('path').join(require('os').tmpdir(), 'sec-red-team-' + Date.now() + '.md')"
|
|
```
|
|
2. Use the Write tool to save the **entire markdown report you just produced** (per-category narrative + scenario pass/fail + defense score + adaptive-mode bypasses if `--adaptive`) to that temp path. Verbatim.
|
|
3. Run the renderer:
|
|
```bash
|
|
node <plugin-root>/scripts/render-report.mjs red-team --in "<temp-md-path>"
|
|
```
|
|
The CLI writes `reports/red-team-<YYYYMMDD-HHmmss>.html` relative to CWD and prints `file:///abs/path.html` on stdout.
|
|
4. Append to your response (markdown link, no bare URL):
|
|
|
|
> **HTML report:** [Open in browser](file:///abs/path.html)
|
|
|
|
If the CLI exits non-zero, mention the error but do not block — the markdown report above is the primary deliverable.
|