open/llm-security
1
0
Fork 0
Code Issues Releases Activity

docs(llm-security): v8 Phase 2 — B10 docs consistency, counts pinned by test

Browse source 
Extends tests/lib/doc-consistency.test.mjs with 15 cases that derive every
inventory count from source instead of trusting prose. Each count has one
stated derivation; a doc surface that disagrees now fails the suite.

Counts corrected (all were wrong before the test existed):
- orchestrated scanners: docs said 10 (README, ci-cd-guide), CLAUDE.md said 12,
  the synthesizer agent said 9 — scan-orchestrator registers 14
- total scanners: README badge + 3 prose sites said 23; the counting rule in
  docs/scanner-reference.md (14 orchestrated + 8 standalone) yields 22
- knowledge files: README badge + prose said 22; knowledge/ holds 23
- output.mjs finding() prefix JSDoc listed 10 of the 17 prefixes actually
  passed to it (missing IDE, MCI, MEM, PST, SCR, TFA, WFL)
- norwegian-context.md said "8 hooks, 10 scanners" -> 9 and 14
- ci-cd-guide "what gets scanned" table listed 10 of 14 rows; adds workflow,
  trigger abuse, signature, AST taint

Two plan items changed after verifying against ground truth:
- CLAUDE.md's synthesizer "(12 scanners)" was not a deliberate subset; the
  agent file itself claimed 9. Both bumped to 14.
- compliance-mapping.md's "13 posture categories" is substantively correct —
  its matrix has exactly 13 data rows, and categories 14-16 are governance
  consumers of the file, not rows in it. The planned 13->16 bump would have
  made the document false. Wording clarified to "code-level" instead, and the
  test now pins row count against the stated claim.

Framework currency (both verified against primary reporting):
- EU AI Act: Digital Omnibus (EP 2026-06-16, Council 2026-06-29) deferred the
  high-risk obligations behind Art. 9/15 to 2027-12-02 (Annex III) and
  2028-08-02 (Annex I); transparency still applies from 2026-08-02
- OWASP Agentic AI Top 10 labelled as the 2026 edition

Also: CLAUDE.md Distribution section rewritten monorepo -> polyrepo (each
plugin is its own repo; the catalog pins url + ref per plugin), and
current-state test counts synced 2013 -> 2034. Release-note paragraphs keep
their historical numbers.

No scanner, hook, or command behaviour changes. Suite 2034/2034.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wt4YQGoXwRja5K2Zmv8RZE
This commit is contained in:
Kjell Tore Guttormsen 2026-08-02 21:22:20 +02:00
commit 0f1be986d0
8 changed files with 287 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

4
scanners/lib/output.mjs
View file

@ -16,7 +16,9 @@ export function resetCounter() {
/**
* Create a finding object.
* @param {object} opts
* @param {string} opts.scanner - Scanner prefix (UNI, ENT, PRM, DEP, TNT, GIT, NET, TRG, SIG, AST)
* Every prefix below is pinned by tests/lib/doc-consistency.test.mjs against the
* scanners that actually call this function — orchestrated first, then standalone.
* @param {string} opts.scanner - Scanner prefix (UNI, ENT, PRM, DEP, TNT, GIT, NET, MEM, SCR, WFL, TRG, SIG, AST, TFA, IDE, MCI, PST)
* @param {string} opts.severity - From SEVERITY constants
* @param {string} opts.title - Short finding title
* @param {string} opts.description - Detailed description