refactor(llm-security): build the SIG ruleset from vendored commons (malware-signatures 0.1.0)

Fifth and last consumer swap of v8 Phase 5 step 4. The seven known-bad-identity
signatures stop living in knowledge/signatures.json and are built from the
vendored commons artifact signatures/malware-signatures.json instead.

Measured before the swap over all seven positions -- id, family, severity,
pattern, description, provenance, key order, and recompilation identity under
the engine's unconditional `i` flag: zero divergences over 56 checks, in order.
The commons copy was extracted from this repository's own file at b0de0ca and
had not drifted.

knowledge/signatures.json is REMOVED rather than left in place. Keeping it would
have left two files spelling one table with nothing gating the drift, and its
golden `file:` pin would have gone on passing while pinning bytes no scanner
reads -- a gate reporting success without running. The pin is replaced by a
walked-module anchor over SIGNATURE_RULES, which is strictly stronger: the pin
covered the bytes on disk, the walk covers what `new RegExp` made of them.
Golden diff was exactly that and nothing else: 7 ADDED, 1 REMOVED, 0 CHANGED
(102/7/5 -> 109/7/4), each added source verified equal to the recompiled commons
pattern.

compileRules() moves into the new lib module and is exported, so the built-in
ruleset and the operator's sig.custom_rules_path path keep one implementation
rather than two copies of the defaulting logic.

Coverage by construction, not by memory: the probe table in the scanner test is
asserted against the LOADED ruleset, so a rule commons adds cannot arrive
without an end-to-end probe. Mutation of the vendored JSON fires in three
directions -- under-match (xmrig alternative dropped) reddens two scanner tests
plus golden; over-match (webshell rule widened to a bare `shell`) reddens the
clean-fixture false-positive probe plus golden; reorder reddens the declared-
order test plus golden.

Loud failure is contract: an unresolvable commons writes one line to stderr
rather than silently disabling known-malware detection, and never throws.

Suite 2247 / 2241 pass / 6 skipped / 0 fail. suite-counts.json untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0151x4FVg9Mn55C2LvHLpHKo

scanners/lib/commons-loader.mjs

@@ -11,8 +11,9 @@
 // fresh zero-dep processes, so resolution must be a fast synchronous read of
 // the vendored copy, never network. Cached once per process. A
 // missing/unvendored commons dir degrades to the caller's fallback rather
-// than crashing a hook — the same graceful-empty contract loadRules() uses
-// for a missing knowledge/signatures.json.
+// than crashing a hook — the same graceful-empty contract the SIG scanner's
+// loadRules() used for a missing knowledge/signatures.json before that
+// ruleset itself moved here in Phase 5 step 4.
 //
 // Deliberately NOT policy-driven, unlike loadCustomRules()'s
 // sig.custom_rules_path: this plugin scans untrusted cloned repos, and

scanners/lib/malware-signatures.mjs

@@ -0,0 +1,118 @@
+// malware-signatures.mjs — SIG known-bad-identity ruleset, built from vendored commons.
+//
+// v8 Phase 5 step 4, fifth and last consumer swap. The seven signatures lived
+// in `knowledge/signatures.json`; they are now built once, here, from
+// `signatures/malware-signatures.json` in the vendored llm-security-commons
+// subtree. The commons copy was extracted from this repository's own file at
+// commit b0de0ca; verified before the swap by a differential over all seven
+// positions — id, family, severity, pattern, description, provenance, key
+// order, and recompilation identity under the engine's unconditional `i` flag
+// — against commons malware-signatures 0.1.0: zero divergences, in order.
+//
+// `knowledge/signatures.json` is REMOVED rather than left in place. Keeping it
+// would leave two files spelling one table with nothing gating the drift, and
+// its golden `file:` pin would have gone on passing while pinning bytes no
+// scanner reads — a gate reporting success without running. The pin is
+// replaced by a walked-module anchor over `SIGNATURE_RULES`, which covers what
+// `new RegExp` made of the patterns rather than the bytes they arrived as.
+//
+// Three properties carried over from the earlier swaps, holding for the same
+// reasons:
+//
+//   1. FAILURE IS LOUD. An empty ruleset means `scan()` returns status `ok`
+//      with zero findings for every file it is handed — a malware gate that
+//      reports success without running, which is the v7.8.2 defect class. So
+//      an unresolvable commons writes one line to stderr. It still does not
+//      throw: this module is imported by a scanner that runs inside the deep
+//      scan orchestrator, and a module-load throw aborts that whole run rather
+//      than degrading one scanner. The warning fires only for the DEFAULT
+//      root — an explicit `commonsRoot` is a test or a dev checkout pointing
+//      elsewhere on purpose, and warning there trains the reader to ignore the
+//      line.
+//
+//   2. RULES ARE COMPILED DEFENSIVELY. commons is vendored data, not code:
+//      `new RegExp` throws on an uncompilable pattern, and it would throw at
+//      module load. A malformed rule is dropped rather than published.
+//
+//   3. CASE-INSENSITIVITY IS ENGINE BEHAVIOUR, NOT DATA. The artifact carries
+//      no `flags` field on any rule and says so explicitly: every pattern is
+//      compiled with `i`, unconditionally. A consumer that compiled these
+//      case-sensitively would silently under-match all seven.
+//
+// Zero external dependencies — Node.js builtins only.
+
+import { loadArtifact } from './commons-loader.mjs';
+
+/**
+ * Compile a parsed ruleset object (`{ rules: [...] }`) into executable rules.
+ *
+ * Exported because the built-in ruleset and the operator's
+ * `sig.custom_rules_path` ruleset must share ONE implementation: two copies of
+ * this defaulting logic would drift, and the custom path is the one an
+ * operator can get wrong.
+ *
+ * Each rule's `pattern` is compiled case-insensitively; rules lacking `id` or
+ * `pattern`, and rules whose pattern fails to compile, are dropped. The
+ * defaults below are loader tolerance, recorded in the artifact under
+ * `missing-field-defaults` — not an optional-field contract.
+ *
+ * @param {object} parsed
+ * @returns {Array<{id: string, family: string, severity: string, re: RegExp,
+ *   description: string, provenance: string|null}>}
+ */
+export function compileRules(parsed) {
+  const compiled = [];
+  for (const rule of parsed?.rules || []) {
+    if (!rule || !rule.id || !rule.pattern) continue;
+    let re;
+    try {
+      re = new RegExp(rule.pattern, 'i');
+    } catch {
+      continue; // skip uncompilable patterns
+    }
+    compiled.push({
+      id: rule.id,
+      family: rule.family || 'unknown',
+      severity: rule.severity || 'high',
+      re,
+      description: rule.description || rule.id,
+      provenance: rule.provenance || null,
+    });
+  }
+  return compiled;
+}
+
+/**
+ * Build the built-in signature ruleset from a commons root.
+ *
+ * @param {object} [opts]
+ * @param {string} [opts.commonsRoot] - explicit commons root (tests, dev
+ *   checkout). Suppresses the unresolvable-commons warning, which is meant for
+ *   the default root only.
+ * @returns {ReadonlyArray<object>} Always an array, so a lost commons yields an
+ *   empty ruleset rather than an undefined one the scanner would spread and
+ *   throw on.
+ */
+export function buildSignatureRules(opts = {}) {
+  const artifact = loadArtifact('signatures/malware-signatures', {
+    fallback: null,
+    commonsRoot: opts.commonsRoot,
+  });
+
+  if (artifact === null && opts.commonsRoot === undefined) {
+    // See note 1 above: silent is the one thing this failure must not be.
+    process.stderr.write(
+      '[llm-security] malware-signatures ruleset unresolvable at '
+      + 'scanners/commons/signatures/malware-signatures.json — known-malware detection is '
+      + 'DISABLED for this process. Reinstall the plugin or re-vendor the commons subtree.\n',
+    );
+  }
+
+  // Array order is the declared order: unlike signatures/secret-egress.json
+  // there is no `order` field, and no rule's report depends on another rule
+  // having matched first — each fires at most once per file, independently.
+  return Object.freeze(compileRules(artifact ?? {}).map((r) => Object.freeze(r)));
+}
+
+/** The known-bad-identity signatures the SIG scanner matches files against. */
+export const SIGNATURE_RULES = buildSignatureRules();

scanners/signature-scanner.mjs

@@ -13,71 +13,22 @@
 // Zero external dependencies — Node.js builtins only.
 
 import { readFile } from 'node:fs/promises';
-import { join, dirname, isAbsolute, resolve } from 'node:path';
-import { fileURLToPath } from 'node:url';
+import { isAbsolute, resolve } from 'node:path';
 import { finding, scannerResult } from './lib/output.mjs';
 import { readTextFile } from './lib/file-discovery.mjs';
 import { normalizeForScan, foldHomoglyphs, rot13 } from './lib/string-utils.mjs';
 import { getPolicyValue } from './lib/policy-loader.mjs';
-
-const __dirname = dirname(fileURLToPath(import.meta.url));
+import { SIGNATURE_RULES, compileRules } from './lib/malware-signatures.mjs';
 
 // Paths excluded from signature scanning when present under the scan root:
-// our own ruleset, test fixtures, and docs all legitimately contain patterns
-// that would otherwise self-flag.
+// test fixtures and docs legitimately contain patterns that would otherwise
+// self-flag. `knowledge/` is kept even though the ruleset itself moved to the
+// vendored commons in v8 Phase 5 — a scanned target's own knowledge/ directory
+// is as likely to hold ruleset-shaped prose as ours was.
 const EXCLUDED_PATH_RE = /(^|\/)(knowledge|tests|docs|node_modules)\//i;
 
 const DEFAULT_FAMILIES = ['webshell', 'reverse_shell', 'cryptominer', 'hacktool'];
 
-// Cached, compiled ruleset (loaded once per process).
-let _rules = null;
-
-/**
- * Compile a parsed ruleset object ({ rules: [...] }) into executable rules.
- * Each rule's `pattern` is compiled to a case-insensitive RegExp; rules whose
- * pattern fails to compile (or that lack id/pattern) are dropped.
- * @param {object} parsed
- * @returns {Array<{id,family,severity,re,description,provenance}>}
- */
-function compileRules(parsed) {
-  const compiled = [];
-  for (const rule of parsed.rules || []) {
-    if (!rule || !rule.id || !rule.pattern) continue;
-    let re;
-    try {
-      re = new RegExp(rule.pattern, 'i');
-    } catch {
-      continue; // skip uncompilable patterns
-    }
-    compiled.push({
-      id: rule.id,
-      family: rule.family || 'unknown',
-      severity: rule.severity || 'high',
-      re,
-      description: rule.description || rule.id,
-      provenance: rule.provenance || null,
-    });
-  }
-  return compiled;
-}
-
-/**
- * Load and compile signatures.json.
- * Graceful fallback to an empty ruleset on any load/parse error.
- * @returns {Promise<Array<{id,family,severity,re,description,provenance}>>}
- */
-async function loadRules() {
-  if (_rules) return _rules;
-  const rulesetPath = join(__dirname, '..', 'knowledge', 'signatures.json');
-  try {
-    const raw = await readFile(rulesetPath, 'utf8');
-    _rules = compileRules(JSON.parse(raw));
-  } catch {
-    _rules = []; // graceful: no ruleset -> no findings
-  }
-  return _rules;
-}
-
 /**
  * v7.8.3 (#36): load operator-supplied rules from the documented
  * `sig.custom_rules_path` policy option. Relative paths resolve against the
@@ -127,7 +78,7 @@ export async function scan(targetPath, discovery) {
   let filesScanned = 0;
 
   try {
-    const rules = [...await loadRules(), ...await loadCustomRules(targetPath)];
+    const rules = [...SIGNATURE_RULES, ...await loadCustomRules(targetPath)];
     const enabledFamilies = new Set(
       (getPolicyValue('sig', 'enabled_families', DEFAULT_FAMILIES, targetPath) || []).map(f => String(f)),
     );