feat(ci): add CI/CD integration — --fail-on, --compact, pipeline templates
Add threshold-based exit codes (--fail-on <severity>) and compact output mode (--compact) to scan-orchestrator and CLI. Pipeline templates for GitHub Actions, Azure DevOps, GitLab CI with SARIF upload. CI/CD guide with Schrems II/NSM compliance documentation. npm publish preparation (files whitelist, .npmignore). Policy ci section for distributable CI defaults. Version 6.1.0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen
parent
d642203991
commit
2c33e9cc64
15 changed files with 599 additions and 17 deletions
165
plugins/llm-security/docs/ci-cd-guide.md
Normal file
165
plugins/llm-security/docs/ci-cd-guide.md
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
# CI/CD Integration Guide
|
||||
|
||||
Integrate llm-security into your CI/CD pipeline for automated security scanning of AI/LLM projects. The standalone CLI runs 10 deterministic Node.js scanners — no AI models, no external API calls, no data leaves your pipeline environment.
|
||||
|
||||
## Data Sovereignty
|
||||
|
||||
**The standalone CLI makes zero network calls by default.** All 10 scanners operate locally on your source code using Shannon entropy analysis, regex pattern matching, AST traversal, and git log parsing. No data is transmitted to any external service.
|
||||
|
||||
**Exception: supply-chain-recheck** — When scanning lockfiles for known vulnerabilities, this scanner optionally queries the [OSV.dev](https://osv.dev/) batch API. This sends only package names and versions (not source code) over HTTPS. To disable: set `LLM_SECURITY_SCR_OFFLINE=1`.
|
||||
|
||||
**What about Claude Code integration?** The Claude Code plugin (hooks, agents, commands) uses AI models and sends data to Anthropic. These components are **not included** in the standalone CLI. When you run `npx llm-security scan`, only deterministic scanners execute.
|
||||
|
||||
### Schrems II / NSM Compliance
|
||||
|
||||
- Standalone CLI: **fully compliant** — no cross-border data transfer
|
||||
- OSV.dev queries (opt-in): sends package metadata to Google-operated API — evaluate per your organization's data classification
|
||||
- Claude Code plugin: sends code context to Anthropic (US) — requires data processing agreement for regulated environments
|
||||
|
||||
### Norwegian Regulatory Context
|
||||
|
||||
- **NSM Grunnprinsipper:** Automated security scanning fulfills GP 3.1 (vulnerability management) and GP 2.4 (secure development)
|
||||
- **Digitaliseringsdirektoratet:** Aligns with recommended practices for AI system development lifecycle security
|
||||
- **EU AI Act (expected Aug 2026):** Directly supports Art. 9 (risk management) and Art. 15 (cybersecurity) requirements
|
||||
|
||||
## 5-Minute Setup
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
Copy `ci/github-action.yml` to `.github/workflows/llm-security.yml`:
|
||||
|
||||
```yaml
|
||||
name: LLM Security Scan
|
||||
on: [push, pull_request]
|
||||
jobs:
|
||||
security-scan:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
security-events: write
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '18'
|
||||
- run: npx llm-security scan . --fail-on high --format sarif --output-file results.sarif
|
||||
- uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: results.sarif
|
||||
```
|
||||
|
||||
SARIF results appear in the repository's **Security** tab under **Code scanning alerts**.
|
||||
|
||||
### Azure DevOps
|
||||
|
||||
Copy `ci/azure-pipelines.yml` to your pipeline, or include the scan step in an existing pipeline:
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- task: NodeTool@0
|
||||
inputs:
|
||||
versionSpec: '18.x'
|
||||
- script: npx llm-security scan . --fail-on high --format sarif --output-file $(Build.ArtifactStagingDirectory)/results.sarif
|
||||
displayName: Run llm-security scan
|
||||
- task: PublishBuildArtifacts@1
|
||||
condition: always()
|
||||
inputs:
|
||||
pathToPublish: $(Build.ArtifactStagingDirectory)/results.sarif
|
||||
artifactName: llm-security-scan
|
||||
```
|
||||
|
||||
For Azure DevOps Advanced Security, replace `PublishBuildArtifacts@1` with `AdvancedSecurity-Publish@1`.
|
||||
|
||||
### GitLab CI
|
||||
|
||||
Add to `.gitlab-ci.yml`:
|
||||
|
||||
```yaml
|
||||
llm-security-scan:
|
||||
image: node:18-alpine
|
||||
stage: test
|
||||
script:
|
||||
- npx llm-security scan . --fail-on high --format sarif --output-file results.sarif
|
||||
artifacts:
|
||||
paths:
|
||||
- results.sarif
|
||||
reports:
|
||||
sast: results.sarif
|
||||
when: always
|
||||
```
|
||||
|
||||
SAST report parsing requires GitLab Ultimate. On Free/Premium tiers, download the SARIF artifact manually.
|
||||
|
||||
## Configuration
|
||||
|
||||
### CLI Flags
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--fail-on <severity>` | Exit 1 if any finding at or above severity exists. Values: `critical`, `high`, `medium`, `low` |
|
||||
| `--compact` | One-liner per finding format. Reduces CI log noise |
|
||||
| `--format sarif` | Output OASIS SARIF 2.1.0 (default: JSON) |
|
||||
| `--output-file <path>` | Write full results to file. Stdout gets compact aggregate |
|
||||
| `--baseline` | Diff against stored baseline (show new/resolved findings) |
|
||||
| `--save-baseline` | Save current results as baseline for future diffs |
|
||||
|
||||
### Policy File
|
||||
|
||||
Configure defaults in `.llm-security/policy.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"ci": {
|
||||
"failOn": "high",
|
||||
"compact": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
CLI flags always take precedence over policy file values.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `LLM_SECURITY_SCR_OFFLINE=1` | Disable OSV.dev network calls in supply-chain-recheck |
|
||||
| `LLM_SECURITY_AUDIT_LOG=<path>` | Write structured JSONL audit trail (SIEM-ready) |
|
||||
|
||||
## Exit Codes
|
||||
|
||||
| Code | Meaning | When |
|
||||
|------|---------|------|
|
||||
| `0` | Clean / below threshold | No findings at or above `--fail-on` level, or ALLOW verdict |
|
||||
| `1` | Threshold exceeded | Findings at or above `--fail-on` level, or WARNING verdict (without `--fail-on`) |
|
||||
| `2` | Block | BLOCK verdict (only without `--fail-on`) |
|
||||
|
||||
With `--fail-on`, exit codes are binary: 0 (clean) or 1 (threshold exceeded). Without `--fail-on`, the legacy tri-state (0/1/2) is preserved.
|
||||
|
||||
## What Gets Scanned
|
||||
|
||||
The 10 deterministic scanners cover:
|
||||
|
||||
| Scanner | Detects |
|
||||
|---------|---------|
|
||||
| Unicode | Zero-width characters, homoglyphs, Unicode Tag steganography |
|
||||
| Entropy | High-entropy strings (potential secrets/tokens) |
|
||||
| Permission | Overly broad permissions, missing tool justification |
|
||||
| Dependency | Known vulnerable packages, typosquats |
|
||||
| Taint | Untrusted input flows to sensitive operations |
|
||||
| Git forensics | Force pushes, sensitive file history, author anomalies |
|
||||
| Network | Suspicious URLs, exfiltration endpoints, C2 patterns |
|
||||
| Memory poisoning | Injection patterns in CLAUDE.md, memory files, rules |
|
||||
| Supply chain | Lockfile audit, blocklists, OSV.dev (opt-in) |
|
||||
| Toxic flow | Lethal trifecta correlation (input + access + exfil) |
|
||||
|
||||
## Local Testing
|
||||
|
||||
Test the exact same command locally before adding to CI:
|
||||
|
||||
```bash
|
||||
# With npx (requires npm publish)
|
||||
npx llm-security scan . --fail-on high --compact
|
||||
|
||||
# With local clone
|
||||
node bin/llm-security.mjs scan . --fail-on high --compact
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue