Add threshold-based exit codes (--fail-on <severity>) and compact output mode (--compact) to scan-orchestrator and CLI. Pipeline templates for GitHub Actions, Azure DevOps, GitLab CI with SARIF upload. CI/CD guide with Schrems II/NSM compliance documentation. npm publish preparation (files whitelist, .npmignore). Policy ci section for distributable CI defaults. Version 6.1.0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
165 lines
6 KiB
Markdown
165 lines
6 KiB
Markdown
# CI/CD Integration Guide
|
|
|
|
Integrate llm-security into your CI/CD pipeline for automated security scanning of AI/LLM projects. The standalone CLI runs 10 deterministic Node.js scanners — no AI models, no external API calls, no data leaves your pipeline environment.
|
|
|
|
## Data Sovereignty
|
|
|
|
**The standalone CLI makes zero network calls by default.** All 10 scanners operate locally on your source code using Shannon entropy analysis, regex pattern matching, AST traversal, and git log parsing. No data is transmitted to any external service.
|
|
|
|
**Exception: supply-chain-recheck** — When scanning lockfiles for known vulnerabilities, this scanner optionally queries the [OSV.dev](https://osv.dev/) batch API. This sends only package names and versions (not source code) over HTTPS. To disable: set `LLM_SECURITY_SCR_OFFLINE=1`.
|
|
|
|
**What about Claude Code integration?** The Claude Code plugin (hooks, agents, commands) uses AI models and sends data to Anthropic. These components are **not included** in the standalone CLI. When you run `npx llm-security scan`, only deterministic scanners execute.
|
|
|
|
### Schrems II / NSM Compliance
|
|
|
|
- Standalone CLI: **fully compliant** — no cross-border data transfer
|
|
- OSV.dev queries (opt-in): sends package metadata to Google-operated API — evaluate per your organization's data classification
|
|
- Claude Code plugin: sends code context to Anthropic (US) — requires data processing agreement for regulated environments
|
|
|
|
### Norwegian Regulatory Context
|
|
|
|
- **NSM Grunnprinsipper:** Automated security scanning fulfills GP 3.1 (vulnerability management) and GP 2.4 (secure development)
|
|
- **Digitaliseringsdirektoratet:** Aligns with recommended practices for AI system development lifecycle security
|
|
- **EU AI Act (expected Aug 2026):** Directly supports Art. 9 (risk management) and Art. 15 (cybersecurity) requirements
|
|
|
|
## 5-Minute Setup
|
|
|
|
### GitHub Actions
|
|
|
|
Copy `ci/github-action.yml` to `.github/workflows/llm-security.yml`:
|
|
|
|
```yaml
|
|
name: LLM Security Scan
|
|
on: [push, pull_request]
|
|
jobs:
|
|
security-scan:
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
security-events: write
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '18'
|
|
- run: npx llm-security scan . --fail-on high --format sarif --output-file results.sarif
|
|
- uses: github/codeql-action/upload-sarif@v3
|
|
if: always()
|
|
with:
|
|
sarif_file: results.sarif
|
|
```
|
|
|
|
SARIF results appear in the repository's **Security** tab under **Code scanning alerts**.
|
|
|
|
### Azure DevOps
|
|
|
|
Copy `ci/azure-pipelines.yml` to your pipeline, or include the scan step in an existing pipeline:
|
|
|
|
```yaml
|
|
steps:
|
|
- task: NodeTool@0
|
|
inputs:
|
|
versionSpec: '18.x'
|
|
- script: npx llm-security scan . --fail-on high --format sarif --output-file $(Build.ArtifactStagingDirectory)/results.sarif
|
|
displayName: Run llm-security scan
|
|
- task: PublishBuildArtifacts@1
|
|
condition: always()
|
|
inputs:
|
|
pathToPublish: $(Build.ArtifactStagingDirectory)/results.sarif
|
|
artifactName: llm-security-scan
|
|
```
|
|
|
|
For Azure DevOps Advanced Security, replace `PublishBuildArtifacts@1` with `AdvancedSecurity-Publish@1`.
|
|
|
|
### GitLab CI
|
|
|
|
Add to `.gitlab-ci.yml`:
|
|
|
|
```yaml
|
|
llm-security-scan:
|
|
image: node:18-alpine
|
|
stage: test
|
|
script:
|
|
- npx llm-security scan . --fail-on high --format sarif --output-file results.sarif
|
|
artifacts:
|
|
paths:
|
|
- results.sarif
|
|
reports:
|
|
sast: results.sarif
|
|
when: always
|
|
```
|
|
|
|
SAST report parsing requires GitLab Ultimate. On Free/Premium tiers, download the SARIF artifact manually.
|
|
|
|
## Configuration
|
|
|
|
### CLI Flags
|
|
|
|
| Flag | Description |
|
|
|------|-------------|
|
|
| `--fail-on <severity>` | Exit 1 if any finding at or above severity exists. Values: `critical`, `high`, `medium`, `low` |
|
|
| `--compact` | One-liner per finding format. Reduces CI log noise |
|
|
| `--format sarif` | Output OASIS SARIF 2.1.0 (default: JSON) |
|
|
| `--output-file <path>` | Write full results to file. Stdout gets compact aggregate |
|
|
| `--baseline` | Diff against stored baseline (show new/resolved findings) |
|
|
| `--save-baseline` | Save current results as baseline for future diffs |
|
|
|
|
### Policy File
|
|
|
|
Configure defaults in `.llm-security/policy.json`:
|
|
|
|
```json
|
|
{
|
|
"ci": {
|
|
"failOn": "high",
|
|
"compact": true
|
|
}
|
|
}
|
|
```
|
|
|
|
CLI flags always take precedence over policy file values.
|
|
|
|
### Environment Variables
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `LLM_SECURITY_SCR_OFFLINE=1` | Disable OSV.dev network calls in supply-chain-recheck |
|
|
| `LLM_SECURITY_AUDIT_LOG=<path>` | Write structured JSONL audit trail (SIEM-ready) |
|
|
|
|
## Exit Codes
|
|
|
|
| Code | Meaning | When |
|
|
|------|---------|------|
|
|
| `0` | Clean / below threshold | No findings at or above `--fail-on` level, or ALLOW verdict |
|
|
| `1` | Threshold exceeded | Findings at or above `--fail-on` level, or WARNING verdict (without `--fail-on`) |
|
|
| `2` | Block | BLOCK verdict (only without `--fail-on`) |
|
|
|
|
With `--fail-on`, exit codes are binary: 0 (clean) or 1 (threshold exceeded). Without `--fail-on`, the legacy tri-state (0/1/2) is preserved.
|
|
|
|
## What Gets Scanned
|
|
|
|
The 10 deterministic scanners cover:
|
|
|
|
| Scanner | Detects |
|
|
|---------|---------|
|
|
| Unicode | Zero-width characters, homoglyphs, Unicode Tag steganography |
|
|
| Entropy | High-entropy strings (potential secrets/tokens) |
|
|
| Permission | Overly broad permissions, missing tool justification |
|
|
| Dependency | Known vulnerable packages, typosquats |
|
|
| Taint | Untrusted input flows to sensitive operations |
|
|
| Git forensics | Force pushes, sensitive file history, author anomalies |
|
|
| Network | Suspicious URLs, exfiltration endpoints, C2 patterns |
|
|
| Memory poisoning | Injection patterns in CLAUDE.md, memory files, rules |
|
|
| Supply chain | Lockfile audit, blocklists, OSV.dev (opt-in) |
|
|
| Toxic flow | Lethal trifecta correlation (input + access + exfil) |
|
|
|
|
## Local Testing
|
|
|
|
Test the exact same command locally before adding to CI:
|
|
|
|
```bash
|
|
# With npx (requires npm publish)
|
|
npx llm-security scan . --fail-on high --compact
|
|
|
|
# With local clone
|
|
node bin/llm-security.mjs scan . --fail-on high --compact
|
|
```
|