open/llm-security
1
0
Fork 0
Code Issues Releases Activity

docs(llm-security): v8 Phase 2 — B10 docs consistency, counts pinned by test

Browse source 
Extends tests/lib/doc-consistency.test.mjs with 15 cases that derive every
inventory count from source instead of trusting prose. Each count has one
stated derivation; a doc surface that disagrees now fails the suite.

Counts corrected (all were wrong before the test existed):
- orchestrated scanners: docs said 10 (README, ci-cd-guide), CLAUDE.md said 12,
  the synthesizer agent said 9 — scan-orchestrator registers 14
- total scanners: README badge + 3 prose sites said 23; the counting rule in
  docs/scanner-reference.md (14 orchestrated + 8 standalone) yields 22
- knowledge files: README badge + prose said 22; knowledge/ holds 23
- output.mjs finding() prefix JSDoc listed 10 of the 17 prefixes actually
  passed to it (missing IDE, MCI, MEM, PST, SCR, TFA, WFL)
- norwegian-context.md said "8 hooks, 10 scanners" -> 9 and 14
- ci-cd-guide "what gets scanned" table listed 10 of 14 rows; adds workflow,
  trigger abuse, signature, AST taint

Two plan items changed after verifying against ground truth:
- CLAUDE.md's synthesizer "(12 scanners)" was not a deliberate subset; the
  agent file itself claimed 9. Both bumped to 14.
- compliance-mapping.md's "13 posture categories" is substantively correct —
  its matrix has exactly 13 data rows, and categories 14-16 are governance
  consumers of the file, not rows in it. The planned 13->16 bump would have
  made the document false. Wording clarified to "code-level" instead, and the
  test now pins row count against the stated claim.

Framework currency (both verified against primary reporting):
- EU AI Act: Digital Omnibus (EP 2026-06-16, Council 2026-06-29) deferred the
  high-risk obligations behind Art. 9/15 to 2027-12-02 (Annex III) and
  2028-08-02 (Annex I); transparency still applies from 2026-08-02
- OWASP Agentic AI Top 10 labelled as the 2026 edition

Also: CLAUDE.md Distribution section rewritten monorepo -> polyrepo (each
plugin is its own repo; the catalog pins url + ref per plugin), and
current-state test counts synced 2013 -> 2034. Release-note paragraphs keep
their historical numbers.

No scanner, hook, or command behaviour changes. Suite 2034/2034.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wt4YQGoXwRja5K2Zmv8RZE
This commit is contained in:
Kjell Tore Guttormsen 2026-08-02 21:22:20 +02:00
commit 0f1be986d0
8 changed files with 287 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

4
agents/deep-scan-synthesizer-agent.md
View file

@ -2,7 +2,7 @@
name: deep-scan-synthesizer-agent
description: |
Synthesizes deterministic deep-scan JSON results into a human-readable security report.
Takes raw scanner output (9 scanners, structured findings) and produces an executive summary,
Takes raw scanner output (14 scanners, structured findings) and produces an executive summary,
prioritized recommendations, and per-scanner analysis.
Use when /security deep-scan or /security scan --deep has completed scanner execution.
model: opus
@ -17,7 +17,7 @@ You are a security report synthesizer for the llm-security plugin's deterministi
## Input
You receive:
1. **Raw JSON output** from `scan-orchestrator.mjs` — contains findings from 9 scanners (including TFA toxic flow analysis)
1. **Raw JSON output** from `scan-orchestrator.mjs` — contains findings from 14 scanners (including TFA toxic flow analysis)
2. **Path to the report template** at `templates/unified-report.md` (ANALYSIS_TYPE: deep-scan)
3. **Knowledge base paths** for OWASP context