feat(llm-security): add 3 more runnable threat examples [skip-docs]
Three new self-contained, runnable threat demonstrations under
examples/, continuing the batch started in 583a78c. Each example
has README.md + run-*.mjs + expected-findings.md and uses
state-isolation discipline so the user's real cache/state files
are never polluted.
- examples/supply-chain-attack/ — two-layer demonstration:
pre-install-supply-chain (PreToolUse) blocks compromised
event-stream version 3.3.6 and emits a scope-hop advisory for
the @evilcorp scope; dep-auditor (DEP scanner, offline) flags
5 typosquat dependencies plus a curl-piped install-script
vector in the fixture package.json. Maps to LLM03/LLM05/ASI04.
- examples/poisoned-claude-md/ — all 6 memory-poisoning detectors
fire on a deliberately poisoned CLAUDE.md plus a fixture
agent file under .claude/agents (E15/v7.2.0 surface):
detectInjection, detectShellCommands, detectSuspiciousUrls,
detectCredentialPaths, detectPermissionExpansion,
detectEncodedPayloads. No agent runtime needed — scanner
imported directly. Maps to LLM01/LLM06/ASI04.
- examples/bash-evasion-gallery/ — one disguised variant per
T1 through T9 evasion technique fed through pre-bash-destructive,
verified BLOCK after bash-normalize strips the evasion. T8
base64-pipe-shell uses its own BLOCK_RULE. The canonical
destructive form uses a path token rather than the bare slash
(regex word-boundary requires it). Source-string fragmentation
pattern reused from the e2e attack-chain test. Maps to
LLM06/ASI01/LLM01.
Plugin README "Other runnable examples" section + plugin
CLAUDE.md "Examples" table + CHANGELOG Unreleased/Added
all updated. Marketplace root README unchanged
([skip-docs] for marketplace-level gate — plugin's outward
coverage is unchanged, only demonstrations were added).
This commit is contained in:
parent
8179415bc2
commit
ca5a8cec67
15 changed files with 1184 additions and 0 deletions
131
plugins/llm-security/examples/bash-evasion-gallery/README.md
Normal file
131
plugins/llm-security/examples/bash-evasion-gallery/README.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
# Bash Evasion Gallery (T1-T9)
|
||||
|
||||
> **WARNING: This is a demonstration fixture, NOT a real attack.**
|
||||
> No destructive command actually runs. The script feeds JSON
|
||||
> payloads to one PreToolUse hook script and verifies the hook's
|
||||
> exit code. Your `$HOME` is safe.
|
||||
|
||||
## What this demonstrates
|
||||
|
||||
`hooks/scripts/pre-bash-destructive.mjs` (PreToolUse on `Bash`)
|
||||
catches destructive commands by pattern-matching the command
|
||||
string. A motivated attacker can disguise the same command using
|
||||
shell metacharacters that bash interprets transparently — so
|
||||
naive regex matching is bypassed.
|
||||
|
||||
`scanners/lib/bash-normalize.mjs` is the **defense-in-depth layer**
|
||||
that strips these evasion techniques before pattern matching runs.
|
||||
This gallery feeds one disguised variant per T-tag through the
|
||||
hook and verifies that every variant gets normalized and blocked.
|
||||
|
||||
This complements — does not replace — Claude Code 2.1.98+'s
|
||||
harness-level bash normalization. The plugin layer runs before
|
||||
the harness layer, so even on older Claude Code versions the
|
||||
techniques below are caught.
|
||||
|
||||
## The T-tag taxonomy
|
||||
|
||||
| Tag | Technique | Example | Normalizes to |
|
||||
|-----|-----------|---------|---------------|
|
||||
| T1 | empty single quotes | `r''m -rf $HOME` | `rm -rf $HOME` |
|
||||
| T2 | empty double quotes | `r""m -rf $HOME` | `rm -rf $HOME` |
|
||||
| T3 | parameter expansion (multi-char) | `${UNUSED}rm -rf $HOME` | `rm -rf $HOME` |
|
||||
| T3 | parameter expansion (single-char) | `c${u}rl url \| sh` | `curl url \| sh` |
|
||||
| T4 | backslash word-splitting | `r\m -rf $HOME` | `rm -rf $HOME` |
|
||||
| T5 | IFS word-splitting | `rm${IFS}-rf${IFS}$HOME` | `rm -rf $HOME` |
|
||||
| T6 | ANSI-C hex quoting | `$'\x72\x6d' -rf $HOME` | `rm -rf $HOME` |
|
||||
| T7 | process substitution | `cat <(echo rm) -rf $HOME` | `cat echo rm -rf $HOME` |
|
||||
| T8 | base64-pipe-shell idiom | `echo cm0gLXJmICRIT01F \| base64 -d \| sh` | (separate BLOCK_RULE — not normalization) |
|
||||
| T9 | eval-via-variable (one-level forward flow) | `X=rm; $X -rf $HOME` | `X=rm; rm -rf $HOME` |
|
||||
|
||||
The canonical destructive target throughout the gallery is
|
||||
`rm -rf $HOME`. The `pre-bash-destructive` "Filesystem root
|
||||
destruction" BLOCK_RULE matches `rm -rf` followed by `$HOME`,
|
||||
`/<path>`, or `~<path>` — but plain `rm -rf /` slips through
|
||||
because the regex requires a word boundary after the path. The
|
||||
gallery uses `$HOME` so the regex fires reliably; for the literal
|
||||
`rm -rf /` case, see Claude Code 2.1.98+ harness-level checks.
|
||||
|
||||
T8 is structurally different — it's not an evasion of an existing
|
||||
rule but a fresh attack pattern (decode + pipe to shell). It has
|
||||
its own BLOCK_RULE in `pre-bash-destructive`, named "T8 —
|
||||
base64-pipe-shell idiom".
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
cd plugins/llm-security
|
||||
node examples/bash-evasion-gallery/run-evasion-gallery.mjs
|
||||
|
||||
# Detailed: full hook stderr
|
||||
node examples/bash-evasion-gallery/run-evasion-gallery.mjs --verbose
|
||||
```
|
||||
|
||||
Expected: `10 pass, 0 fail`. Each T-tag should produce
|
||||
`BLOCKED: Destructive command detected — Filesystem root
|
||||
destruction (rm -rf /)` on stderr (or `T8 — base64-pipe-shell
|
||||
idiom` for the T8 case).
|
||||
|
||||
## Hooks involved
|
||||
|
||||
- **`hooks/scripts/pre-bash-destructive.mjs`** — PreToolUse on
|
||||
`Bash`. Reads `tool_input.command`, applies `normalizeBashExpansion`
|
||||
(T1-T7, T9) then `normalizeCommand` (whitespace + ANSI), then
|
||||
iterates 8 BLOCK_RULES (rm-root, chmod 777, curl|sh, fork bomb,
|
||||
mkfs, dd, /dev/ writes, eval+expansion, T8 base64-pipe-shell).
|
||||
Exit 2 = block, exit 0 = allow (with optional WARN advisory).
|
||||
|
||||
- **`scanners/lib/bash-normalize.mjs`** — pure function module,
|
||||
shared with `pre-install-supply-chain.mjs`. Exports
|
||||
`normalizeBashExpansion(cmd)`. Test contract:
|
||||
`tests/lib/bash-normalize.test.mjs`.
|
||||
|
||||
## Source-string fragmentation
|
||||
|
||||
The `run-evasion-gallery.mjs` script never contains the literal
|
||||
string `rm -rf $HOME`. Each test command is built at runtime via
|
||||
concatenation (`'r' + 'm'`, `'-' + 'rf'`, `'$' + 'HOME'`). This
|
||||
follows the discipline from `tests/e2e/attack-chain.test.mjs` —
|
||||
secrets-shaped or destructive-command-shaped strings should never
|
||||
appear as literals in the source, because:
|
||||
|
||||
1. `pre-edit-secrets` would block writing the file
|
||||
2. Static scanners would treat the source as compromised
|
||||
3. Operators reading the diff would not be able to tell at a
|
||||
glance whether the literal was meant to be safe context or
|
||||
live payload
|
||||
|
||||
The `--verbose` mode prints the assembled command to stdout for
|
||||
inspection — that's a runtime print, not a source-file literal.
|
||||
|
||||
## OWASP / framework mapping
|
||||
|
||||
| Code | Framework | Why |
|
||||
|------|-----------|-----|
|
||||
| LLM06 | OWASP LLM Top 10 (2025) | Excessive Agency — destructive-command surface |
|
||||
| ASI01 | OWASP Agentic Top 10 | Excessive Agency / runtime command injection |
|
||||
| LLM01 | OWASP LLM Top 10 (2025) | Indirect prompt injection often delivers commands |
|
||||
|
||||
## Limitations
|
||||
|
||||
- T8 base64 decoding is detected pattern-only — it doesn't decode
|
||||
the blob to inspect its content. A blob that decodes to
|
||||
*non-destructive* content would still be flagged. This is the
|
||||
documented v5.0 honest-limitation: deterministic detection
|
||||
errs toward false-positive, not false-negative.
|
||||
- T9 only resolves one level of variable substitution forward.
|
||||
`X=rm; Y=$X; $Y -rf $HOME` is **not** caught — the second-level
|
||||
alias is too rare in real attacks to justify the runtime cost.
|
||||
- The hook is a defense-in-depth layer. Claude Code 2.1.98+
|
||||
includes harness-level bash normalization that catches the
|
||||
same techniques (and more, since the harness sees fully-
|
||||
expanded commands). This plugin's hook covers older Claude
|
||||
Code versions and runs before the harness layer either way.
|
||||
|
||||
## See also
|
||||
|
||||
- `docs/security-hardening-guide.md` §3 — bash evasion theory
|
||||
- `tests/lib/bash-normalize.test.mjs` — per-T-tag unit contract
|
||||
- `tests/lib/pre-bash-destructive.test.mjs` — block-rule contract
|
||||
- `examples/supply-chain-attack/` — adjacent layer (install gate)
|
||||
- `expected-findings.md` (in this folder) — testable contract
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
# Expected Findings — Bash Evasion Gallery
|
||||
|
||||
This is the testable contract. `run-evasion-gallery.mjs` exits 0
|
||||
only when every T-tag produces a BLOCK.
|
||||
|
||||
## Per-tag contract
|
||||
|
||||
| Tag | Hook stderr must contain |
|
||||
|-----|--------------------------|
|
||||
| baseline | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T1 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T2 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T3 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T4 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T5 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T6 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T7 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
| T8 | `BLOCKED: Destructive command detected — T8 — base64-pipe-shell idiom (echo BLOB \| base64 -d \| sh)` |
|
||||
| T9 | `BLOCKED: Destructive command detected — Filesystem root destruction (rm -rf /)` |
|
||||
|
||||
All cases must exit with code 2.
|
||||
|
||||
## Why $HOME instead of /
|
||||
|
||||
The "Filesystem root destruction" BLOCK_RULE regex requires a
|
||||
word-boundary anchor after the destination path:
|
||||
|
||||
```
|
||||
\brm\s+(?:-[a-zA-Z]*f[a-zA-Z]*\s+|--force\s+)*-[a-zA-Z]*r[a-zA-Z]*\s+(?:\/|~|\$HOME)\b
|
||||
```
|
||||
|
||||
`rm -rf /` ends with `/` followed by end-of-string; both `/` and
|
||||
EOL are non-word, so `\b` does not match. The variants
|
||||
`rm -rf /tmp`, `rm -rf $HOME`, and `rm -rf /etc` all match — the
|
||||
trailing word character provides the boundary.
|
||||
|
||||
This gallery uses `$HOME` because it is unambiguously destructive
|
||||
and the regex fires deterministically. The literal `rm -rf /`
|
||||
edge case is not part of this contract — it is covered by Claude
|
||||
Code 2.1.98+ harness-level checks.
|
||||
|
||||
## Side effects
|
||||
|
||||
- No file is modified
|
||||
- No real `bash` is invoked — only `node hooks/scripts/...`
|
||||
- Each hook spawn has `tool_input.command` set to the disguised
|
||||
variant — bash never sees these strings
|
||||
- No mutation of `$HOME`, `/`, `/tmp`, or anywhere else
|
||||
|
||||
## Notes for forks
|
||||
|
||||
- If `bash-normalize.mjs` adds new T-tags (T10+), add a new case
|
||||
to `CASES` and a corresponding row above
|
||||
- If a BLOCK_RULE in `pre-bash-destructive.mjs` is renamed,
|
||||
update the stderr-pattern column above (the assertion lives
|
||||
in `expected-findings.md` for documentation; the run script
|
||||
only checks exit code 2, so it continues to pass after a
|
||||
rename)
|
||||
- The base64 blob in T8 (`cm0gLXJmICRIT01F`) decodes to the
|
||||
literal command. If you change the canonical destructive
|
||||
target away from `$HOME`, regenerate the blob with
|
||||
`printf '<new-cmd>' | base64`
|
||||
|
|
@ -0,0 +1,186 @@
|
|||
#!/usr/bin/env node
|
||||
// run-evasion-gallery.mjs — Bash evasion technique gallery (T1-T9)
|
||||
// Feeds one disguised command per T-tag to pre-bash-destructive and
|
||||
// verifies that every variant is normalized + blocked. Demonstrates
|
||||
// why bash-normalize.mjs exists as a defense-in-depth layer above
|
||||
// Claude Code 2.1.98+ harness fixes.
|
||||
//
|
||||
// Each case carries:
|
||||
// - the disguised command (what an attacker might paste)
|
||||
// - the canonical form bash-normalize should produce
|
||||
// - the BLOCK_RULE in pre-bash-destructive that catches the
|
||||
// normalized form
|
||||
//
|
||||
// Usage:
|
||||
// cd plugins/llm-security
|
||||
// node examples/bash-evasion-gallery/run-evasion-gallery.mjs
|
||||
// node examples/bash-evasion-gallery/run-evasion-gallery.mjs --verbose
|
||||
|
||||
import { execFile } from 'node:child_process';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const PLUGIN_ROOT = resolve(__dirname, '../..');
|
||||
const HOOK = resolve(PLUGIN_ROOT, 'hooks/scripts/pre-bash-destructive.mjs');
|
||||
const VERBOSE = process.argv.includes('--verbose');
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cases
|
||||
// ---------------------------------------------------------------------------
|
||||
//
|
||||
// We pick `rm -rf /` (or close cousins) as the canonical destructive
|
||||
// command for T1-T7, T9. Pre-bash-destructive's BLOCK_RULES['Filesystem
|
||||
// root destruction'] catches it after normalization.
|
||||
//
|
||||
// T8 is a separate base64-pipe-shell BLOCK_RULE — we exercise that one
|
||||
// independently with the documented pattern.
|
||||
//
|
||||
// Each `command` is constructed at runtime from harmless fragments to
|
||||
// avoid triggering pre-edit-secrets / file-content scanners on this
|
||||
// source file. (Source-string fragmentation is the same pattern used
|
||||
// by tests/e2e/attack-chain.test.mjs.)
|
||||
|
||||
// Note: pre-bash-destructive's "Filesystem root destruction" BLOCK_RULE
|
||||
// requires a path token after `-rf` (the regex enforces a word boundary).
|
||||
// `rm -rf /` alone does not match — `rm -rf $HOME` does. We use $HOME
|
||||
// as the canonical destructive target so the regex fires after
|
||||
// normalization, mirroring the realistic case where an attacker
|
||||
// targets a populated home directory.
|
||||
const RM = 'r' + 'm';
|
||||
const RF = '-' + 'rf';
|
||||
const TARGET = '$' + 'HOME';
|
||||
const PIPE_SH = '| ' + 'sh';
|
||||
|
||||
const CASES = [
|
||||
{
|
||||
tag: 'baseline (no evasion)',
|
||||
command: `${RM} ${RF} ${TARGET}`,
|
||||
note: 'control: pre-bash-destructive blocks the canonical form directly',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T1 — empty single quotes',
|
||||
command: `r''m ${RF} ${TARGET}`,
|
||||
note: "r''m → rm after T1 normalization",
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T2 — empty double quotes',
|
||||
command: `r""m ${RF} ${TARGET}`,
|
||||
note: 'r""m → rm after T2 normalization',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T3 — parameter expansion',
|
||||
// bash-normalize's T3 has TWO branches:
|
||||
// ${x} (single char) → x e.g. c${u}rl → curl
|
||||
// ${FOO} (multi char) → '' e.g. ${FOO}rm → rm
|
||||
// We exercise the multi-char branch — the single-char branch is more
|
||||
// useful inside a name (curl letter substitution).
|
||||
command: `\${UNUSED}${RM} ${RF} ${TARGET}`,
|
||||
note: '${UNUSED} → "" (multi-char branch), so the prefix collapses to rm',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T4 — backslash word-splitting',
|
||||
command: `r\\m ${RF} ${TARGET}`,
|
||||
note: 'r\\m → rm after T4 normalization',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T5 — IFS word-splitting',
|
||||
command: `${RM}\${IFS}${RF}\${IFS}${TARGET}`,
|
||||
note: '${IFS} → space, so the command reads as `rm -rf $HOME`',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T6 — ANSI-C hex quoting',
|
||||
command: `$'\\x72\\x6d' ${RF} ${TARGET}`,
|
||||
note: "$'\\x72\\x6d' → rm after T6 hex decoding",
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T7 — process substitution',
|
||||
command: `cat <(echo ${RM}) ${RF} ${TARGET}`,
|
||||
note: '<( ... ) is stripped to expose the inner tokens',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T8 — base64-pipe-shell idiom',
|
||||
// echo cm0gLXJmICRIT01F | base64 -d | sh — base64 of "rm -rf $HOME"
|
||||
command: `echo cm0gLXJmICRIT01F | base64 -d ${PIPE_SH}`,
|
||||
note: 'separate BLOCK_RULE — not a normalization, but the same shape',
|
||||
expectBlock: true,
|
||||
},
|
||||
{
|
||||
tag: 'T9 — eval-via-variable (one-level forward flow)',
|
||||
command: `X=${RM}; $X ${RF} ${TARGET}`,
|
||||
note: 'one-level alias resolved during normalization',
|
||||
expectBlock: true,
|
||||
},
|
||||
];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Hook runner
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function runHook(command) {
|
||||
return new Promise((res) => {
|
||||
const child = execFile(
|
||||
'node',
|
||||
[HOOK],
|
||||
{ timeout: 5000 },
|
||||
(_err, stdout, stderr) => {
|
||||
res({ code: child.exitCode ?? 1, stdout: stdout || '', stderr: stderr || '' });
|
||||
},
|
||||
);
|
||||
child.stdin.end(JSON.stringify({
|
||||
tool_name: 'Bash',
|
||||
tool_input: { command },
|
||||
}));
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
console.log('BASH EVASION GALLERY (T1-T9)');
|
||||
console.log('============================\n');
|
||||
console.log('Hook: pre-bash-destructive (PreToolUse on Bash)');
|
||||
console.log('Normalize layer: scanners/lib/bash-normalize.mjs\n');
|
||||
|
||||
let pass = 0;
|
||||
let fail = 0;
|
||||
|
||||
for (const tc of CASES) {
|
||||
const result = await runHook(tc.command);
|
||||
const blocked = result.code === 2;
|
||||
const ok = blocked === tc.expectBlock;
|
||||
if (ok) pass++; else fail++;
|
||||
|
||||
console.log(`[${ok ? 'PASS' : 'FAIL'}] ${tc.tag}`);
|
||||
console.log(` command: ${tc.command}`);
|
||||
console.log(` expect: ${tc.expectBlock ? 'BLOCK (exit 2)' : 'allow (exit 0)'}`);
|
||||
console.log(` got: exit ${result.code}${blocked ? ' (blocked)' : ' (allowed)'}`);
|
||||
console.log(` why: ${tc.note}`);
|
||||
if (VERBOSE && result.stderr.trim()) {
|
||||
const head = result.stderr.trim().split('\n').slice(0, 2).join(' / ');
|
||||
console.log(` stderr: ${head.slice(0, 160)}`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
console.log('---');
|
||||
console.log(`Result: ${pass} pass, ${fail} fail`);
|
||||
|
||||
if (fail > 0) {
|
||||
console.log('\nFAILURE — at least one evasion variant slipped past the hook.');
|
||||
console.log('See expected-findings.md for the documented contract.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log('\nSUCCESS — all evasion variants normalized and blocked.');
|
||||
console.log('Read examples/bash-evasion-gallery/README.md for context.');
|
||||
process.exit(0);
|
||||
Loading…
Add table
Add a link
Reference in a new issue