ktg-plugin-marketplace/plugins/llm-security/CHANGELOG.md

# Changelog

All notable changes to the LLM Security Plugin are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [6.1.0] - 2026-04-10

### Added
- **`--fail-on <severity>` flag** — CI-friendly exit codes: exit 1 when any finding at or above the specified severity exists (critical/high/medium/low). Configurable via `policy.json` `ci.failOn`
- **`--compact` output mode** — One-liner per finding format (`[SEVERITY] scanner: title (file:line)`), reduces CI log noise. Configurable via `policy.json` `ci.compact`
- **CI/CD pipeline templates** — Ready-to-use templates in `ci/`: GitHub Actions (`github-action.yml`), Azure DevOps (`azure-pipelines.yml`), GitLab CI (`gitlab-ci.yml`) with SARIF upload, Node 18 setup
- **CI/CD integration guide** — `docs/ci-cd-guide.md` with 5-minute setup per platform, Schrems II/NSM compliance documentation, exit code reference
- **npm publish preparation** — `files` whitelist in `package.json` (only `bin/` + `scanners/`), `.npmignore` safety net, `homepage` field
- **Policy `ci` section** — New `ci: { failOn, compact }` section in `.llm-security/policy.json` for distributable CI configuration

### Changed
- Version bump: 6.0.0 → 6.1.0 across all files

## [6.0.0] - 2026-04-10

### Added
- **Compliance mapping** — `knowledge/compliance-mapping.md` maps plugin capabilities to EU AI Act (Art. 9, 15, 17), NIST AI RMF (Map, Measure, Manage, Govern), ISO 42001 (Annex A), and MITRE ATLAS techniques (AML.T IDs)
- **Norwegian regulatory context** — `knowledge/norwegian-context.md` covers Datatilsynet (DPIA for AI), NSM (basic security principles), and Digitaliseringsdirektoratet guidance
- **SARIF 2.1.0 output** — `scanners/lib/sarif-formatter.mjs` converts scan output to OASIS SARIF standard format. Use `--format sarif` with scan/deep-scan commands
- **Structured audit trail** — `scanners/lib/audit-trail.mjs` writes JSONL audit events with ISO 8601 timestamps, OWASP category tags, and SIEM-ready schema. Configurable via `LLM_SECURITY_AUDIT_*` env vars
- **AI-BOM generator** — `scanners/ai-bom-generator.mjs` + `scanners/lib/bom-builder.mjs` produce CycloneDX 1.6 Bills of Materials for AI components (models, MCP servers, plugins, knowledge, hooks)
- **Policy-as-code** — `scanners/lib/policy-loader.mjs` reads `.llm-security/policy.json` for distributable hook configuration. Integrated into all 8 hooks. Env vars always take precedence
- **Standalone CLI** — `bin/llm-security.mjs` provides `npx llm-security` entry point. Subcommands: `scan`, `deep-scan`, `posture`, `audit-bom`, `benchmark`
- **Posture compliance categories** — 3 new posture categories (14: EU AI Act, 15: NIST AI RMF, 16: ISO 42001). Advisory only — do not affect Grade A threshold
- **Attack simulator benchmark mode** — `--benchmark` flag outputs structured pass/fail metrics for CI integration

### Changed
- Version bump: 5.1.0 → 6.0.0 across all files
- Knowledge base expanded from 13 to 15 files
- Scanner count: 15 → 16 (AI-BOM generator added)
- Posture scanner: 13 → 16 categories
- All hooks now read policy from `.llm-security/policy.json` (backward-compatible — defaults match hardcoded values)

## [5.1.0] - 2026-04-07

### Added
- **Sandboxed remote cloning** — `git clone` for remote scans is now hardened with two defense layers:
  1. Git config flags: `core.hooksPath=/dev/null`, `core.symlinks=false`, `core.fsmonitor=false`, all LFS filter drivers disabled, `protocol.file.allow=never`, `transfer.fsckObjects=true`. Environment: `GIT_CONFIG_NOSYSTEM=1`, `GIT_CONFIG_GLOBAL=/dev/null`, `GIT_ATTR_NOSYSTEM=1`, `GIT_TERMINAL_PROMPT=0`
  2. OS-level filesystem sandbox: macOS `sandbox-exec` and Linux `bubblewrap` (bwrap) restrict file writes to only the specific temp directory. Even if `.gitattributes` filter drivers bypass git config, they cannot write outside the clone dir. bwrap probe-tests availability before use (graceful fallback on Ubuntu 24.04+ where AppArmor blocks it). Graceful fallback on Windows (git config flags only, WARN logged)
- **Post-clone size check** — Repos exceeding 100MB after clone are rejected and cleaned up
- **UUID-unique evidence filenames** — `fs-utils.mjs tmppath` now generates unique filenames with `crypto.randomUUID()` suffix, preventing race conditions between concurrent scans
- **Evidence file cleanup** — `scan.md` and `plugin-audit.md` now clean up evidence files (content-extract, plugin-extract) after scanning
- **Cleanup guarantee** — Both `scan.md` and `plugin-audit.md` have explicit cleanup guarantee: temp dir + evidence file are removed even if scan fails or errors

### Changed
- `scanners/lib/git-clone.mjs` — complete rewrite of clone command with sandbox wrapping
- `scanners/lib/fs-utils.mjs` — tmppath uses `crypto.randomUUID()` for unique names

## [5.0.0] - 2026-04-06

### Added
- **Prompt Injection Hardening (v5.0)** — 8-session defense-in-depth overhaul driven by 7 research papers (2025-2026). Defense philosophy: broader detection + increased attack cost + longer monitoring windows + architectural constraints + honest documentation
- **MEDIUM advisory wiring** — `pre-prompt-inject-scan.mjs` emits advisory for MEDIUM-severity obfuscation signals (leetspeak, homoglyphs, zero-width, multi-language). Never blocks. `post-mcp-verify.mjs` includes MEDIUM in injection scan advisory
- **Unicode Tag steganography** — `string-utils.mjs` decodes U+E0001-E007F (invisible ASCII encoding). CRITICAL if decoded content matches injection patterns, HIGH for bare presence. Integrated into `normalizeForScan()` pipeline
- **BIDI override stripping** — Removes directional override characters before injection scanning
- **Bash expansion normalization** — New `bash-normalize.mjs` strips `${}`, empty quotes, backslash splits before command matching. Applied in `pre-bash-destructive.mjs` and `pre-install-supply-chain.mjs`
- **Rule of Two enforcement** — `post-session-guard.mjs` gains `LLM_SECURITY_TRIFECTA_MODE=block|warn|off` (default: warn). Block mode exits with code 2 for MCP-concentrated trifecta or sensitive path + exfiltration
- **100-call long-horizon monitoring** — Extended window alongside 20-call sliding window. Slow-burn trifecta detection (legs >50 calls apart = MEDIUM). Behavioral drift via Jensen-Shannon divergence on tool-class distribution
- **HITL trap detection** — HIGH patterns for approval urgency, summary suppression, scope minimization. MEDIUM for cognitive load (injection buried in verbose output)
- **Sub-agent delegation tracking** — `post-session-guard.mjs` tracks Task/Agent tool usage. Escalation-after-input advisory when delegation occurs within 5 calls of untrusted input (DeepMind Agent Traps kat. 4)
- **Natural language indirection** — MEDIUM patterns for "fetch this URL and execute", "send this data to", "read ~/.ssh". Strict false-positive tests for benign phrasing
- **Hybrid attack patterns** — P2SQL (SQL keywords in injection text), recursive injection (injection containing injection), XSS in agent context (`<script>`, `javascript:`, `onerror=`)
- **CaMeL-inspired data flow tagging** — SHA-256 provenance tracking in `post-session-guard.mjs`. Hash of tool output → match against subsequent tool input. Linked data flows elevate trifecta severity
- **Adaptive red-team** — `attack-simulator.mjs --adaptive` runs 5 mutation rounds per passing scenario: homoglyph substitution, encoding wrapping, zero-width injection, case alternation, synonym substitution. Rules in `knowledge/attack-mutations.json`
- **Knowledge base expansion** — `prompt-injection-research-2025-2026.md` (7 papers), `deepmind-agent-traps.md` (6 categories, 43 techniques), `attack-mutations.json` (synonym tables). Attack scenarios expanded from 38 to 64 across 12 categories
- **Posture scanner expanded to 13 categories** — New: Prompt Injection Hardening (cat 11), Rule of Two (cat 12), Long-Horizon Monitoring (cat 13). Checks for MEDIUM advisory, Unicode Tag detection, bash normalization, TRIFECTA_MODE, behavioral drift
- **Defense Philosophy section** in CLAUDE.md — honest documentation of what v5.0 can and cannot do, based on joint paper findings (95-100% ASR against all tested defenses)
- 8 new posture scanner tests (49 total for posture)

### Changed
- Posture scanner version updated to 5.0.0
- Dashboard aggregator version updated to 5.0.0
- Red-team scenarios expanded from 38 to 64 across 12 categories
- Knowledge files count: 10 -> 13

## [4.5.1] - 2026-04-04

### Fixed
- **Cross-platform support (Windows/Linux).** Replaced all Unix-only patterns: `fileURLToPath()` instead of `import.meta.url.replace('file://', '')`, `path.dirname()` instead of `lastIndexOf('/')`, native `fetch()` instead of `curl` subprocess (Node 18+), removed `2>/dev/null` from shell commands, fixed tilde expansion regex for Windows backslash paths. 11 files changed, 782 tests pass.

## [4.5.0] - 2026-04-04

### Added
- **Attack simulation / red-team mode** — `scanners/attack-simulator.mjs` runs 38 crafted attack scenarios across 7 categories against the plugin's own hooks. Data-driven: scenarios defined in `knowledge/attack-scenarios.json`, payloads assembled at runtime via fragment concatenation (avoids triggering hooks on source file). Categories: secrets (7), destructive (8), supply-chain (4), prompt-injection (6), pathguard (6), mcp-output (4), session-trifecta (3). CLI: `node scanners/attack-simulator.mjs [--category <name>] [--json] [--verbose]`. Library: `import { loadScenarios, runScenario, resolvePayloads }`
- **`/security red-team` command** — attack simulation with category filter (`--category secrets|destructive|...`). Narrative report with per-category breakdown and defense score
- **`knowledge/attack-scenarios.json`** — 38 red-team scenarios with placeholder payloads (`{{MARKER}}` syntax), resolved at runtime to actual attack strings
- 31 new tests for attack simulator (unit + integration + CLI)

## [4.4.0] - 2026-04-03

### Added
- **Cross-project security dashboard** — `scanners/dashboard-aggregator.mjs` discovers all Claude Code projects under ~/ (depth 3) and ~/.claude/plugins/, runs posture-scanner on each, aggregates results. Machine grade = weakest link across all projects. Cache in `~/.cache/llm-security/dashboard-latest.json` (24h staleness). CLI: `node scanners/dashboard-aggregator.mjs [--no-cache] [--max-depth N]`. Library: `import { aggregate, discoverProjects }`
- **`/security dashboard` command** — machine-wide security overview with per-project grade table, sorted by grade (worst first). Shows cache status, total findings, and recommendations based on machine grade
- 16 new tests for dashboard aggregator (discovery, aggregation, caching, grade logic)

## [4.3.0] - 2026-04-03

### Added
- **MCP description drift detection** — `scanners/lib/mcp-description-cache.mjs` caches MCP tool descriptions in `~/.cache/llm-security/mcp-descriptions.json` with 7-day TTL. Compares via Levenshtein distance — >10% change triggers advisory (OWASP MCP05 rug-pull). `extractMcpServer()` exported for server attribution
- **MCP-concentrated trifecta** — `post-session-guard.mjs` now detects when all 3 lethal trifecta legs (input + access + exfil) originate from the same MCP server, elevating severity. Single compromised server pattern
- **Cumulative data volume tracking** — `post-session-guard.mjs` tracks total output bytes per session, warns at 100KB (LOW), 500KB (MEDIUM), 1MB (HIGH) thresholds (OWASP ASI02)
- **Per-MCP-tool volume tracking** — `post-mcp-verify.mjs` tracks cumulative output per MCP tool, warns when a single tool exceeds 100KB (OWASP ASI02, MCP03)
- **MCP drift integration in post-mcp-verify** — checks MCP tool descriptions on every invocation against cached baseline, advisory on significant drift
- 35 new tests: 16 for mcp-description-cache, 5 for post-mcp-verify drift/volume, 14 for post-session-guard MCP features

## [4.2.0] - 2026-04-03

### Added
- **Supply chain re-check scanner** — `scanners/supply-chain-recheck.mjs` (prefix SCR) periodically re-audits installed dependencies by parsing lockfiles (package-lock.json, yarn.lock, requirements.txt, Pipfile.lock). Checks against curated blocklists, OSV.dev batch API (`/v1/querybatch`) for known CVEs, and Levenshtein-based typosquat detection against top-packages knowledge base. Offline fallback: blocklist + typosquat checks run without network, INFO finding notes skipped CVE check. OWASP: LLM03, ASI04, AST06, MCP04
- **Shared supply chain data module** — `scanners/lib/supply-chain-data.mjs` extracts blocklists (NPM/PIP/Cargo/Gem), helper functions, and OSV.dev API calls shared between the hook (`pre-install-supply-chain.mjs`) and the new scanner
- **`/security supply-check` command** — standalone dependency re-audit with focused output. CLI wrapper: `node scanners/supply-chain-recheck-cli.mjs <path>`
- SCR prefix added to all 4 OWASP maps (LLM, ASI, AST, MCP) in severity.mjs
- Supply chain scanner integrated into scan-orchestrator (10th scanner, runs before toxic-flow)
- Test fixtures: `tests/fixtures/supply-chain/` with compromised and clean lockfiles for npm, pip, yarn, Pipfile
- 30 new tests for supply-chain-recheck scanner and shared module

### Changed
- `pre-install-supply-chain.mjs` hook refactored to import blocklists and helpers from shared `supply-chain-data.mjs` module (reduced duplication by ~160 lines)

## [4.1.0] - 2026-04-03

### Added
- **Reference configuration generator** — `scanners/reference-config-generator.mjs` generates Grade A security configuration based on posture scanner gaps. Detects project type (plugin/monorepo/standalone). Templates in `templates/reference-config/`. CLI: `node scanners/reference-config-generator.mjs [path] [--apply]`. Library: `import { generate } from './reference-config-generator.mjs'`
- **`/security harden` command** — runs posture scanner, identifies gaps, generates settings.json (deny-first), CLAUDE.md security section, and .gitignore additions. Supports `--dry-run` (default) and `--apply` (writes with backup). Post-apply verification re-runs posture scanner to confirm improvement
- Reference config templates: `settings-deny-first.json`, `claude-md-security-section.md`, `gitignore-security.txt`
- 23 new tests for reference-config-generator (grade-a, grade-f, apply mode, project type detection)

## [4.0.0] - 2026-04-03

### Added
- **Deterministic posture scanner** — `posture-scanner.mjs` replaces the Opus-based posture-assessor-agent for `/security posture`. 10 categories assessed in <50ms (was ~6 min with agent). Scanner prefix PST. Standalone CLI: `node scanners/posture-scanner.mjs [path]` → JSON stdout. Categories: Deny-First, Secrets, Path Guarding, MCP Trust, Destructive Blocking, Sandbox, Human Review, Plugin Sources, Session Isolation, Cognitive State Security. Reuses `scanForInjection()` and `gradeFromPassRate()` from shared libraries. Grade A/B/C/D/F with risk score, risk band, and verdict
- PST prefix added to all 4 OWASP maps (LLM, ASI, AST, MCP) in severity.mjs
- Test fixtures: `tests/fixtures/posture-scan/grade-a-project/` (Grade A) and `grade-f-project/` (Grade F)
- 41 new tests for posture scanner (interface, grade-a, grade-f)

### Changed
- `/security posture` now uses deterministic scanner via Bash instead of spawning posture-assessor-agent. Instant results, zero token cost
- `/security audit` runs posture scanner first for instant category data, then agents for narrative and skill/MCP analysis
- Posture-assessor-agent retained for full audit narrative only

## [3.1.1] - 2026-04-03

Audit remediation: 6 findings fixed, global settings hardened.

## [3.0.0] - 2026-04-03

Public release. 8 development sessions from v2.5 to v3.0.

### Added
- **Toxic flow analysis** (v2.7.0) — 8th orchestrated scanner (`toxic-flow-analyzer.mjs`, prefix TFA) detecting lethal trifecta patterns: untrusted input + sensitive data access + exfiltration sink. Post-processing correlator consuming output from all prior scanners. Direct, cross-component, and project-level detection with mitigation downgrades. OWASP: ASI01, ASI02, ASI05
- **Runtime session guard** (v2.7.1) — PostToolUse hook monitoring tool call sequences for lethal trifecta forming during a session. Sliding window (20 calls), per-session JSONL state in `/tmp/`, advisory warning (never blocks). Auto-cleanup after 24h
- **MCP runtime inspection** (v2.8.0) — Standalone scanner (`mcp-live-inspect.mjs`, prefix MCI) connecting to running MCP stdio servers via JSON-RPC 2.0. Fetches live tool/prompt/resource lists, scans descriptions for injection patterns, detects tool shadowing across servers. 10s timeout per server. New `/security mcp-inspect` command. `/security mcp-audit --live` flag for combined static + live analysis
- **Auto update notifications** (v2.8.1) — UserPromptSubmit hook checking for newer plugin versions against the public Forgejo repo (max 1x/24h, cached in `~/.cache/llm-security/`). Disable: `LLM_SECURITY_UPDATE_CHECK=off`
- **Report diffing & baseline** (v2.9.0) — `diff-engine.mjs` library for finding fingerprinting, fuzzy line matching (+-3), and diff categorization (new/resolved/unchanged/moved). Scan orchestrator gains `--baseline` and `--save-baseline` flags. Baselines stored per target hash in `reports/baselines/`. New `/security diff` command
- **Continuous scanning** (v2.9.1) — `/security watch [path] [--interval 6h]` using built-in /loop for recurring diff scanning. `watch-cron.mjs` standalone script for system cron/launchd with multi-target config and exit codes
- **Skill signature registry** (v2.9.2) — `skill-registry.mjs` library for SHA-256 fingerprinting of normalized skill content, scan result caching (7-day staleness), and pattern search. New `/security registry` command. `/security scan` checks registry before full scan for instant results on known fingerprints
- **OWASP Skills Top 10** (v2.6.0) — New knowledge file `owasp-skills-top10.md` (AST01-AST10) with skill-specific threat definitions and mitigations
- **MEDIUM injection patterns** (v2.6.0) — ~15 new patterns: base64 payloads, leetspeak, homoglyphs, multi-language mixing, markdown/HTML comment injection
- **4-framework OWASP mapping** (v2.6.0) — Full coverage of LLM Top 10, Agentic AI Top 10 (ASI), Skills Top 10 (AST), MCP Top 10 in severity.mjs
- Architecture diagram (mermaid) in README
- CHANGELOG.md

### Changed
- Scan orchestrator now runs 8 scanners (was 7) with TFA running last
- Agent prompts updated with ASI/AST/MCP OWASP references
- `scanForInjection()` returns `{ found, severity, patterns }` instead of boolean
- Self-scan suppressions updated from ~150 to ~190 (TFA self-referential findings added)
- Plugin description updated to reference all 4 OWASP frameworks

### Fixed
- package.json version sync with plugin.json

## [2.5.0] - 2026-04-02

### Added
- Pre-extraction indirection layer for remote scan defense. Remote scans pre-extract structured evidence via `content-extractor.mjs` and strip injection patterns BEFORE LLM agents see the content

## [2.4.0] - 2026-04-01

### Added
- GitHub repo URL support for `scan` and `plugin-audit`. Clone to temp dir via `git-clone.mjs`, scan locally, clean up. `--branch <name>` flag for non-default branches

## [2.3.0] - 2026-04-01

### Added
- PostToolUse expanded to ALL tools (was Bash-only). Scans Read, WebFetch, MCP, and all other tool output for indirect prompt injection
- `LLM_SECURITY_INJECTION_MODE` env var: `block` (default), `warn`, `off`
- Complementary Tools section in README (parry-guard, Lasso, Snyk)
- CLAUDE.md poisoning documented as known limitation

### Changed
- Short output skip (<100 chars) for PostToolUse performance

## [2.2.0] - 2026-04-01

### Added
- UserPromptSubmit hook blocking prompt injection in user input
- Obfuscation decoding: unicode-escape, hex-escape, URL-encoding, base64 normalization
- Shared `injection-patterns.mjs` module (21 critical + 8 high patterns)
- PostToolUse indirect injection scanning in tool output (LLM01)

### Changed
- LLM01 coverage 83% -> 95%, LLM05 80% -> 83%

## [2.1.0] - 2026-04-01

### Added
- 383 tests (was 177): full hook coverage (66 tests), auto-cleaner coverage (140 tests)
- HTTPS install URL under fromaitochitta org

### Fixed
- Auto-cleaner import guard
- Solo project setup (CONTRIBUTING.md removed)

### Changed
- Model defaults set to sonnet

## [2.0.0] - 2026-03-31

### Added
- Open-source release: MIT LICENSE, SECURITY.md
- Test suite (`node:test`, 177 tests)
- `pre-write-pathguard.mjs` hook (8 path categories)
- `.gitignore`, `.editorconfig`

## [1.4.0] - 2026-02-21

### Added
- Unified risk scoring formula (25/10/4/1 weights)
- Score-based verdicts and risk bands (Low -> Extreme)
- OWASP categorization and A-F grading
- Single `unified-report.md` template replacing 9 separate templates

## [1.3.0] - 2026-02-21

### Added
- `/security clean` command with 3-tier remediation (auto/semi-auto/manual)
- `auto-cleaner.mjs` engine (16 fix operations, atomic writes, post-fix validation)
- `cleaner-agent` for semi-auto proposals
- `--dry-run` flag

## [1.2.0] - 2026-02-19

### Added
- 7 deterministic Node.js scanners (unicode, entropy, permissions, dependencies, taint, git forensics, network)
- `/security deep-scan` command and `--deep` flag
- Synthesizer agent for scanner JSON interpretation
- Shared scanner library (`scanners/lib/`)
- Demo fixture with 85-finding security assessment

### Changed
- OWASP coverage: LLM01 70->85%, LLM02 90->95%, LLM03 80->90%, LLM06 85->95%

## [1.1.0] - 2026-02-19

### Added
- `/security plugin-audit` command
- `/security mcp-audit` command
- `/security pre-deploy` command
- 3 new report templates

### Changed
- OWASP coverage: LLM03 75% -> 80%

## [1.0.0] - 2026-02-19

### Added
- Initial release
- 4 agents: skill-scanner, mcp-scanner, posture-assessor, threat-modeler
- 4 hooks: secret detection, destructive commands, supply chain, output verification
- 6 knowledge files (2,771 lines)
- 8 commands: security, scan, audit, posture, threat-model, plugin-audit, mcp-audit, pre-deploy
- 7 report templates
- OWASP LLM Top 10 + Agentic AI Top 10 coverage