open/llm-security
1
0
Fork 0
Code Issues Releases 1 Activity

feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs

Browse source 
Closes Phase 3 (B11) of the v8.0.0 plan. Three parts, all with the failing
test written first.

riskScoreV1 removed. scanners/lib/severity.mjs drops riskScoreV1() and its
SEVERITY_WEIGHTS_V1 table - @deprecated since v7.0.0, kept for diff/comparison,
zero callers in code or tests (re-verified, not taken from the plan). The v1
weights are recorded in CHANGELOG so an old score stays re-derivable. riskScore
(v2) is untouched; a test pins that one critical still lands in the 70-95 tier
and that 50 lows score below it, which is exactly the case v1 collapsed to 100.

Posture category 12 no longer keys off an identifier name. The check was
/TRIFECTA_MODE/i over the session-guard source, which measured what a constant
was CALLED rather than whether enforcement was configurable. With the env-var
gone, that regex would have dropped every correctly-migrated project from PASS
to PARTIAL - the gate punishing the migration it exists to encourage. It now
matches getPolicyValue('trifecta', 'mode', ...) and still accepts a pre-v8
vendored guard reading the old env-var, because a third-party project carries
its own hook copy and is equally configurable either way; the evidence line
says which of the two was found. The PARTIAL finding recommended setting an
env-var that v8 ignores; it now names the policy key. The grade-a fixture hook
moves to the policy-era form.

Two never-implemented env-vars deleted from the docs. LLM_SECURITY_SCR_OFFLINE
(ci-cd-guide) and LLM_SECURITY_OFFLINE (supply-chain-attack example) were
documented as OSV.dev / npm-audit kill-switches. No code has ever read either -
verified by grep across scanners, hooks and scripts, which finds them only in
markdown. A promised kill-switch that does nothing is worse than a documented
absence: it is trusted precisely when the run is meant to be air-gapped. The
docs now say there is none and that egress must be blocked at the network
layer. The LLM_SECURITY_AUDIT_* wildcard is narrowed to the one real key.

Docs. Migration section in README + CHANGELOG with the env-var -> policy-key
table, the detection commands (env + shell rc + .envrc + workflows), and the
explicit warning that a removed variable is now INERT rather than an error -
which is the failure mode that loses a project its configuration silently. The
hardening-guide env table splits into surviving vars and a removed-vars
migration table; its "promote to block" runbook named two variables that no
longer exist. Also swept: CLAUDE.md hook table, scanner-reference, ci-cd-guide,
both lethal-trifecta example docs, mitigation-matrix, injection-research.

Test counts in README/CLAUDE.md synced 2034 -> 2045.

Suite 2045 tests, 0 fail (2039 + 4 posture-trifecta + 2 riskScoreV1). The two
known parallel-load flakes did not recur this run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 10:25:03 +02:00
commit fdec4b36ad
16 changed files with 333 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

21
scanners/lib/severity.mjs
View file

@ -9,9 +9,6 @@ export const SEVERITY = Object.freeze({
INFO: 'info',
});
// Legacy weights — used only by riskScoreV1() for backwards-compat reference.
const SEVERITY_WEIGHTS_V1 = { critical: 25, high: 10, medium: 4, low: 1, info: 0 };
/**
* Calculate aggregate risk score from severity counts (v2 model — v7.0.0+).
*
@ -56,24 +53,6 @@ export function riskScore(counts) {
return Math.round(Math.min(100, base));
}
/**
* Legacy v1 risk score formula — kept for diff/comparison only.
* Not exported in production paths; reference for CI re-calibration.
*
* @deprecated Since v7.0.0. Use riskScore() instead. Kept for diff/comparison only — not used in production paths.
* @param {{ critical: number, high: number, medium: number, low: number, info: number }} counts
* @returns {number} 0-100 capped score (sum-and-cap model)
*/
export function riskScoreV1(counts) {
const raw =
(counts.critical || 0) * SEVERITY_WEIGHTS_V1.critical +
(counts.high || 0) * SEVERITY_WEIGHTS_V1.high +
(counts.medium || 0) * SEVERITY_WEIGHTS_V1.medium +
(counts.low || 0) * SEVERITY_WEIGHTS_V1.low +
(counts.info || 0) * SEVERITY_WEIGHTS_V1.info;
return Math.min(raw, 100);
}
/**
* Derive verdict from severity counts and risk score (v7.0.0 thresholds).
* Aligned to v2 riskBand cutoffs so verdict and band are co-monotonic:

21
scanners/posture-scanner.mjs
View file

@ -1122,9 +1122,22 @@ async function checkRuleOfTwo(projectRoot, hooksJson) {
if (hookActive) {
const scriptPath = join(projectRoot, 'hooks', 'scripts', 'post-session-guard.mjs');
const content = await readText(scriptPath);
if (content && /TRIFECTA_MODE/i.test(content)) {
// What this category measures is whether enforcement is *configurable*, not
// what the constant holding it is called. v8.0.0 moved the mode to the
// policy.json key `trifecta.mode`; a pre-v8 vendored guard still resolving
// LLM_SECURITY_TRIFECTA_MODE is equally configurable and still passes.
const readsPolicyMode =
content && /getPolicyValue\w*\(\s*['"]trifecta['"]\s*,\s*['"]mode['"]/.test(content);
const readsLegacyEnv = content && /LLM_SECURITY_TRIFECTA_MODE/.test(content);
if (readsPolicyMode) {
hasTrifectaMode = true;
evidence.push('TRIFECTA_MODE configurable: yes');
evidence.push('Enforcement mode configurable: policy.json trifecta.mode');
} else if (readsLegacyEnv) {
hasTrifectaMode = true;
evidence.push(
'Enforcement mode configurable: LLM_SECURITY_TRIFECTA_MODE (pre-v8.0.0 env-var; ' +
'migrate to the policy.json key trifecta.mode)'
);
}
}
@ -1146,10 +1159,10 @@ async function checkRuleOfTwo(projectRoot, hooksJson) {
scanner: 'PST',
severity: SEVERITY.MEDIUM,
title: 'Session guard lacks configurable trifecta mode',
description: 'post-session-guard does not support LLM_SECURITY_TRIFECTA_MODE (block/warn/off).',
description: 'post-session-guard resolves no enforcement mode (block/warn/off) from the policy.json key trifecta.mode.',
file: 'hooks/scripts/post-session-guard.mjs',
owasp: 'ASI02',
recommendation: 'Upgrade to v5.0 session guard with configurable enforcement mode.',
recommendation: 'Upgrade to a session guard that reads trifecta.mode from .llm-security/policy.json.',
}));
return { status: STATUS.PARTIAL, findings, evidence };
}