ktg-plugin-marketplace/plugins/llm-security/docs/security-hardening-guide.md
Kjell Tore Guttormsen 6f86de937a feat(llm-security)!: v7.0.0 commit 6 — tests, docs, version bump
Final commit in the trustworthy-scoring series. Bundles verdict cutoff
alignment, the last suite of tests, and all documentation touch-points
that quote version numbers or describe v7.0.0 behaviour.

Verdict/band co-monotonicity
- `scanners/lib/severity.mjs` — verdict cutoffs moved from 61/21 to 65/15
  so `BLOCK >= 65`, `WARNING >= 15` locks onto the v2 riskBand() boundaries.
  Prevents "BLOCK / Medium band" contradictions under the v2 formula.

Scanner hardening (bug fixes from v7.0.0 testing)
- `scanners/entropy-scanner.mjs` — `policy_source` now uses
  `existsSync('.llm-security/policy.json')` instead of value-based check.
  Old heuristic always reported 'policy.json' because DEFAULT_POLICY now
  carries an `entropy.thresholds` section.
- `scanners/lib/file-discovery.mjs` — `.sass` and GPU shader extensions
  (`.glsl, .frag, .vert, .shader, .wgsl`) added to TEXT_EXTENSIONS. Without
  this, shader files were invisible to file-discovery, so they were never
  counted as skipped by the entropy-scanner extension filter.

Tests
- `tests/scanners/entropy-context.test.mjs` (new, 24 tests) — A. File-ext
  skip (4), B. Line-level rules 11-17 (8), C. Policy overrides (3).
  Fixtures generate 80-char base64 payloads at runtime via
  `crypto.randomBytes` to dodge the plugin's own pre-edit credential hook
  on the test source.
- `tests/lib/severity.test.mjs` — rewritten with v2 scoring table (70
  tests total, was 52).
- `tests/lib/output.test.mjs:243` — "1 critical = score 80" under v2
  (was 25 under v1).
- Full suite: 1485/1485 green (was 1461).

Docs
- `CHANGELOG.md` — v7.0.0 entry with BREAKING CHANGES section.
- `README.md` (plugin + marketplace root) — version badge, history table,
  plugin-card version string, test count.
- `CLAUDE.md` — header version, "v7.0.0 — Trustworthy scoring" summary
  paragraph at the top.
- `docs/security-hardening-guide.md` — new section 6 "Calibration & false
  positives" documenting v2 formula, context-aware entropy scanner,
  typosquat allowlist, and §6.4 tuning workflow. Existing "Recommended
  baseline" section renumbered to §7.

Version bump
- `6.6.0 -> 7.0.0` across package.json, .claude-plugin/plugin.json,
  scanners/ide-extension-scanner.mjs VERSION const, README badge,
  CLAUDE.md header, marketplace root README card.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 22:26:35 +02:00

11 KiB
Raw Blame History

Security Hardening Guide

This guide documents the environment variables, sandboxing mechanisms, and hook modes available in llm-security, and how to align them with the capabilities of Opus 4.7 and Claude Code 2.1.112.

The guide is opinionated: it describes the configurations the plugin authors run in production. Deviations are fine, but the defaults here are the tested path.


1. Environment variables

1.1 Harness-level (Claude Code)

Variable Values Effect
CLAUDE_CODE_EFFORT_LEVEL low | medium | high | xhigh Tunes how aggressively the model spends compute per turn. xhigh is recommended for security-sensitive planning and audits.
ENABLE_PROMPT_CACHING_1H 1 | unset Enables 1-hour prompt cache TTL. Reduces cost and latency for repeated context; cache hits do not weaken scanning.
CLAUDE_CODE_SCRIPT_CAPS JSON blob Declares maximum capabilities Claude Code can grant scripts it spawns. Use to lock down hook and command execution.

1.2 Plugin-specific hook modes

Variable Default Modes
LLM_SECURITY_INJECTION_MODE block block — exit 2 on critical/high injection patterns. warn — advisory via systemMessage. off — disables scan.
LLM_SECURITY_TRIFECTA_MODE warn block — exit 2 when lethal trifecta (untrusted input + sensitive data + exfiltration sink) detected. warn — advisory. off — disables.
LLM_SECURITY_PRECOMPACT_MODE warn block — exit 2 on findings during PreCompact. warn — advisory via systemMessage. off — disables scan.
LLM_SECURITY_PRECOMPACT_MAX_BYTES 512000 Tail size in bytes read from transcript for scanning. Higher values increase coverage at the cost of latency.
LLM_SECURITY_UPDATE_CHECK on off disables the daily update-check HTTP call.
LLM_SECURITY_AUDIT_* unset Audit trail configuration (destination, format, etc.) for SIEM-ready JSONL output.

Apply env vars via shell profile, .envrc, or the host MDM. Do not write them into the repository.


2. Sandboxing

2.1 macOS — sandbox-exec

scanners/lib/git-clone.mjs wraps remote clones in a sandbox-exec profile that restricts file writes to the specific temp directory. This defends against malicious .gitattributes filter/smudge drivers. The plugin uses this path by default on Darwin.

2.2 Linux — bubblewrap (bwrap)

On Linux, the same flow uses bwrap to accomplish equivalent isolation. Works on Fedora and Arch without configuration. Ubuntu 24.04+ may require a permissive AppArmor profile (administrator privileges); fallback is git-config flags only, with a WARN logged in the clone audit trail.

2.3 Windows

Windows has no equivalent OS sandbox available in default installs. The plugin falls back to hardened git-config flags (core.hooksPath=/dev/null, core.symlinks=false, disabled LFS drivers, protocol.file.allow=never, transfer.fsckObjects=true) and environment isolation (GIT_CONFIG_NOSYSTEM=1, GIT_CONFIG_GLOBAL=/dev/null, GIT_ATTR_NOSYSTEM=1). A WARN is logged so the caller can weigh the residual risk.

2.4 PID-namespace considerations

On Linux hosts with user namespaces disabled (some hardened kernels), bwrap may fail to create the PID namespace. Prefer running scans from a normal user shell; avoid root, which disables user-namespace confinement.


3. Hook modes in practice

3.1 Start in warn mode

Every new integration of llm-security should begin with all modes set to warn. This yields advisories without breaking workflow, and lets the team calibrate false-positive rates against their actual repositories.

3.2 Promote to block after baselining

After a baseline period (typically 1-2 weeks), flip each mode to block in this order: LLM_SECURITY_INJECTION_MODE, LLM_SECURITY_TRIFECTA_MODE, LLM_SECURITY_PRECOMPACT_MODE. The injection hook is first because false positives there are the most visible; blocking comes last because the others build confidence.

3.3 Off mode is a deliberate choice

Use off only when you explicitly need to disable a layer (e.g., during performance profiling). Prefer warn in all other cases — the signal is still recorded in the audit trail.


4. Bash normalization (T1-T6) as defense-in-depth

scanners/lib/bash-normalize.mjs collapses six known bash obfuscation techniques before the denylist gate runs. These are defense-in-depth layers that complement the Claude Code 2.1.98+ harness-level fixes, not a replacement.

Layer Technique Example Normalization
T1 Empty quotes rm''-rf / strip '' / "" between tokens
T2 ${} expansion r${x}m -rf / drop ${VAR} where VAR is unset in scan context
T3 Backslash continuation rm\<newline>-rf / collapse backslash-newline pairs
T4 Tab/whitespace splitting rm\t-rf / normalize whitespace to single space
T5 ${IFS} word-splitting rm${IFS}-rf${IFS}/ replace ${IFS} with space
T6 ANSI-C hex quoting $'\x72\x6d' -rf / decode $'\xHH' to ASCII byte

See CLAUDE.md §Defense Philosophy for the broader framing.


5. Alignment with Opus 4.7 (system card references)

5.1 Agent safety evaluations (§5.2.1)

The Opus 4.7 system card §5.2.1 documents agentic safety evaluations and notes that multi-layer defenses outperform single-layer defenses against adaptive attacks. llm-security implements this posture: prompt-scan + pathguard + trifecta-guard + pre-compact-scan operate in depth. A single layer failing does not compromise the defense.

5.2 Instruction following and hierarchy (§6.3.1.1)

The Opus 4.7 system card §6.3.1.1 describes tighter adherence to the declared instruction hierarchy and more literal interpretation of agent instructions. Consequently:

  • Stacked imperatives (e.g., "NEVER do X / MUST NOT do X") are less useful than tool-level enforcement. Prefer tools: frontmatter to restrict capabilities at the platform level, so the agent simply does not have the unsafe tool.
  • Agent instructions should mark speculation as speculation, and cite evidence (path, line number) rather than generalizing from one observation. See the "Step 0 Generaliseringsgrense" note added to skill-scanner-agent.md and mcp-scanner-agent.md.
  • Parallel Read calls are preferred for independent file reads, documented in the same Step 0 notes. This reduces latency and aligns with the model's improved parallel-tool-use behavior.

5.3 Known limitations (system card §6.3)

Prompt injection is structurally unsolvable in the current architecture. The system card acknowledges this; so does CLAUDE.md §Defense Philosophy. The hardening described here reduces the attack surface and raises the cost of attacks but does not eliminate them.


6. Calibration & false positives (v7.0.0+)

Security scanners live or die by their signal-to-noise ratio. A scanner that cries "extreme" on every project destroys its own credibility — users learn to ignore findings, and genuine threats slip past. v7.0.0 ships three calibration layers to keep that from happening.

6.1 Risk-score v2 formula

The v1 formula was a sum-and-cap: critical*25 + high*10 + medium*4 + low*1, capped at 100. Every non-trivial scan collapsed to 100/Extreme regardless of actual distribution. A codebase with 2 mediums and 100 lows scored the same as a codebase with 5 criticals.

v2 (scanners/lib/severity.mjs) is severity-dominated and log-scaled within tier:

Finding mix Score range Band
Critical present 7095 (1=80, 2=86, 4=90, 10=95) Critical/Extreme
High only 4065 (1=48, 5=60, 17=65) High
Medium only 1535 (1=20, 5=28, 50=33) Medium
Low only 111 (1=4, 10=11) Low
None 0 Low

Verdict cutoffs (BLOCK ≥65, WARNING ≥15) are locked to the riskBand() boundaries so you can't get a "BLOCK / Medium band" contradiction. The legacy formula is kept as riskScoreV1() for reference only.

CI impact: Pipelines with --fail-on high keep working (the severity gate is unaffected). Pipelines with score-based thresholds need recalibration — old score >= 21 corresponds roughly to new score >= 15.

6.2 Context-aware entropy scanner

The entropy scanner flags high-Shannon-entropy strings as possible credentials. On codebases heavy with shader code, bundled JS, CSS-in-JS or SQL it produced astronomical false-positive rates. v7.0.0 adds three suppression layers:

  1. File-extension skip — whole files with these extensions are never inspected for entropy findings: .glsl, .frag, .vert, .shader, .wgsl, .css, .scss, .sass, .less, .svg + compound .min.js, .min.css, .map. A skip counter (calibration.files_skipped_by_extension) is reported in the scanner envelope.
  2. Line-level rules 1117 — applied when a line contains any of: GLSL keywords (uniform, vec3, texture2D…), CSS-in-JS templates (styled.…), inline <svg> markup, ffmpeg filter_complex syntax, browser User-Agent strings, SQL DDL on a dedicated line (^\s*(SELECT|INSERT|…)), or throw new Error(\…`)` templates.
  3. Per-project policy override.llm-security/policy.json entropy section supports:
{
  "entropy": {
    "thresholds": {
      "critical": { "entropy": 5.4, "minLen": 128 },
      "high":     { "entropy": 5.1, "minLen": 64 },
      "medium":   { "entropy": 4.7, "minLen": 40 }
    },
    "suppress_extensions": [".custom"],
    "suppress_line_patterns": ["MY_VENDOR_MARKER"],
    "suppress_paths": ["vendored/", "generated/"]
  }
}

The synthesizer agent reports calibration prominently if >80 % of files were skipped (signals a policy so aggressive the scan is effectively bypassed) and omits it silently if <5 % were skipped.

6.3 Typosquat allowlist

The DEP scanner flags Levenshtein-close package names against a top-N list to catch typosquats (lod-ash, expres). On real codebases this tripped on short-name tools like knip, nx, tsx, uv, ruff. v7.0.0 extends knowledge/typosquat-allowlist.json with 22 npm + 5 PyPI entries for modern tools.

6.4 Tuning workflow

  1. Run /security deep-scan on a representative codebase.
  2. Read calibration.files_skipped_by_extension and files_skipped_by_path from the envelope — are they reasonable?
  3. Review the top 10 findings. For each false positive, pick the narrowest suppression that catches it:
    • Whole extension noisy → suppress_extensions
    • One line pattern recurring → suppress_line_patterns
    • Whole directory vendored → suppress_paths
  4. Raise thresholds only as a last resort — you're hiding real signal.
  5. Re-scan and verify verdict/band/score make sense relative to the finding set.

  1. Set CLAUDE_CODE_EFFORT_LEVEL=xhigh for audit and planning sessions.
  2. Set ENABLE_PROMPT_CACHING_1H=1 globally — reduces cost, does not weaken scanning.
  3. All three plugin hook modes: start at warn, promote to block after baselining.
  4. Keep sandbox wrappers enabled (default on macOS / Linux).
  5. Periodically run /security posture (16-category scorecard) and /security dashboard (cross-project view) to catch drift.
  6. After first /security deep-scan, run the §6.4 tuning workflow once to calibrate the noise floor for your codebase.

Last updated: 2026-04-19 for v7.0.0.