open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

docs(readme): true self-scan number, AV false-positive section, demo form

Browse source 
.llm-security-ignore: MEM/SIG/UNI rules added only for by-design
findings, one explaining line each — the vendored conformance corpus,
the Cyrillic confusables in injection-patterns.mjs and the homoglyph
test inputs, the changelog's quoted homoglyph, the grade-F posture
fixture, and the miner names in the supply-chain blocklist. Not
suppressed on purpose: MEM on CLAUDE.md (the file a poisoning would
target) and AST/TRG (no by-design findings once the ast-scan fixtures
left the disk). Header prefix list completed (14 scanners) and the
"150" replaced by the measured count.

Self-scan, fresh clone of HEAD (node scanners/scan-orchestrator.mjs):
  before: BLOCK 80/100, 77 findings (1C 9H 29M 2L 36I), 362 suppressed
  after:  WARNING 61/100, 58 findings (0C 5H 15M 2L 36I), 381 suppressed
  without the ignore file: BLOCK 95/100, 439 findings (unchanged)
On `.` (working tree incl. local-only files): BLOCK 80/100, 80
findings -> WARNING 64/100, 61 findings.

README:
- Self-scan: the measured numbers replace "0 findings (ALLOW)".
- The demo commands materialize the tree to a temp dir (the form the
  demo README uses); no evil-project-health/ path left in README.
- New "Antivirus false positives" section + TOC entry: what the repo
  contains, why a classifier can still flag it, restore from
  quarantine, Microsoft's submission form, Microsoft's own advice
  against exclusions, a sparse-checkout recipe. Every Defender claim
  checked against learn.microsoft.com; the recipe run live against
  Forgejo (partial clone supported; 285 of 677 files on disk). The
  meaning of "!ml" is not claimed — Microsoft documents "!" suffixes
  only as internal indicators.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 14:24:05 +02:00
commit d9787aefb5
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 78 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

55
README.md
View file

@ -38,6 +38,7 @@ A Claude Code plugin that provides security scanning, auditing, and threat model
- [Migrating to v8.0.0](#migrating-to-v800)
- [Playground (v7.6.0)](#playground-v760)
- [Self-scan](#self-scan)
- [Antivirus false positives](#antivirus-false-positives)
- [Recent versions](#recent-versions)
- [Changelog](#changelog)
- [License & attribution](#license--attribution)
@ -660,15 +661,18 @@ you start:
npm test # node --test 'tests/**/*.test.mjs'
```
Running `node scanners/scan-orchestrator.mjs .` on this plugin produces **0 findings (ALLOW)** with ~190 suppressions via `.llm-security-ignore`. Every suppression is explained — a security plugin that documents attack patterns, ships a malicious demo fixture, and tests against deliberately evil code will trigger its own scanners. The entropy scanner flags regex patterns in `knowledge/secrets-patterns.md`. The taint scanner flags `eval(user_input)` in test fixtures. The toxic flow analyzer flags the plugin's own commands that use Read+Bash. Remove the ignore file and re-run to see the unsuppressed picture.
Running `node scanners/scan-orchestrator.mjs .` on a fresh clone produces **WARNING 61/100 with 58 findings** (0 critical, 5 high, 15 medium, 2 low, 36 info; measured 2026-09-22), after `.llm-security-ignore` suppresses 381. Without the ignore file the same run reports BLOCK 95/100 with 439 findings. A security plugin that documents attack patterns, ships a malicious demo, and tests against deliberately evil code triggers its own scanners; every suppression is explained in the file. What is left is reported on purpose: high-entropy strings in docs and the HTML report renderer, example and research domains in docs, three CLI tools that write to a path given on the command line, and memory findings on `CLAUDE.md` — it documents the hooks, but it is the file a poisoning would target, so it is not suppressed. Remove the ignore file and re-run to see the unsuppressed picture.
The `examples/malicious-skill-demo/` directory contains a deliberately malicious "Project Health Dashboard" plugin and a [full security assessment](examples/malicious-skill-demo/security-assessment.md). The combined LLM + deterministic pipeline produced **85 findings** (24 critical, 24 high, 20 medium, 6 low, 11 info) and verdict **BLOCK 100/100** — both layers independently maxed the risk score. A human reviewing the plugin's `README.md` and `SKILL.md` would likely miss most of them; the Unicode Tag steganography is literally invisible.
```bash
node scanners/scan-orchestrator.mjs examples/malicious-skill-demo/evil-project-health/ # ~5s
/security scan examples/malicious-skill-demo/evil-project-health/ --deep # full pipeline
DEMO="$(node examples/malicious-skill-demo/materialize.mjs)" # writes the demo to a temp dir, prints the path
node scanners/scan-orchestrator.mjs "$DEMO" # ~5s
rm -rf "$(dirname "$DEMO")" # when you are done
```
For the full pipeline, materialize the same way and run `/security scan <printed path> --deep` in Claude Code.
### Other runnable examples
The `examples/` directory contains additional self-contained
@ -726,6 +730,51 @@ demonstrations — each with `README.md`, fixture, run script, and
---
## Antivirus false positives
This repository is a security tool, so it contains detection rules and descriptions of attack
techniques: signature tables (regular expressions for known-bad code), knowledge files that explain
how prompt injection, exfiltration and persistence attacks work, and a demo of a malicious plugin.
Since v8.1.0 no payload sits on disk as a file that would run it. Test fixtures and the demo are
stored encoded or built from fragments, and exist only in a temp directory while a test or the demo
runs; `tests/av-surface.test.mjs` fails the suite if one comes back.
An antivirus product can still flag a file. Microsoft Defender Antivirus does not only match known
samples: it also uses local and cloud-based machine-learning models, behavior analysis and
heuristics ([Next-generation protection overview](https://learn.microsoft.com/defender-endpoint/next-generation-protection)).
A text file that explains many attack techniques in one place can be flagged although nothing in it
runs — a plain `SKILL.md` in another security repository was quarantined as
`Trojan:Script/Wacatac.H!ml`.
If Defender quarantines a file from this repository:
1. **Restore it:** Windows Security → Virus & threat protection → Protection history, select the
item, then **Restore** ([Restore quarantined files](https://learn.microsoft.com/defender-endpoint/restore-quarantined-files-microsoft-defender-antivirus)).
2. **Report the false positive** to Microsoft at <https://www.microsoft.com/wdsi/filesubmission>, so
Microsoft can review and correct the detection.
3. Open an issue in the marketplace repository with the file path and the detection name.
Excluding the folder is not recommended here, and Microsoft advises against it as a fix: "Every
exclusion is a protection gap that lowers your defenses, so use exclusions sparingly"
([Exclusions in Microsoft Defender Antivirus](https://learn.microsoft.com/defender-endpoint/microsoft-defender-antivirus-exclusions-overview)).
If you only want the scanners and hooks, a sparse checkout leaves knowledge files, tests, examples
and docs off disk (measured 2026-09-22: 285 of 677 files):
```bash
git clone --filter=blob:none --no-checkout https://git.fromaitochitta.com/open/llm-security.git
cd llm-security
git sparse-checkout set scanners hooks
git checkout main
```
Cone mode also checks out the files at the repository root. Some scanners read data from
`knowledge/` (the typosquat lists, the attack simulator's scenarios); add `knowledge` to the `set`
line if you run those. A sparse checkout of your own clone does not change what
`claude plugin install` puts in the plugin cache.
---
## Recent versions
| Version | Date | Highlights |