Commit graph

3 commits

Author SHA1 Message Date
0f1be986d0 docs(llm-security): v8 Phase 2 — B10 docs consistency, counts pinned by test
Extends tests/lib/doc-consistency.test.mjs with 15 cases that derive every
inventory count from source instead of trusting prose. Each count has one
stated derivation; a doc surface that disagrees now fails the suite.

Counts corrected (all were wrong before the test existed):
- orchestrated scanners: docs said 10 (README, ci-cd-guide), CLAUDE.md said 12,
  the synthesizer agent said 9 — scan-orchestrator registers 14
- total scanners: README badge + 3 prose sites said 23; the counting rule in
  docs/scanner-reference.md (14 orchestrated + 8 standalone) yields 22
- knowledge files: README badge + prose said 22; knowledge/ holds 23
- output.mjs finding() prefix JSDoc listed 10 of the 17 prefixes actually
  passed to it (missing IDE, MCI, MEM, PST, SCR, TFA, WFL)
- norwegian-context.md said "8 hooks, 10 scanners" -> 9 and 14
- ci-cd-guide "what gets scanned" table listed 10 of 14 rows; adds workflow,
  trigger abuse, signature, AST taint

Two plan items changed after verifying against ground truth:
- CLAUDE.md's synthesizer "(12 scanners)" was not a deliberate subset; the
  agent file itself claimed 9. Both bumped to 14.
- compliance-mapping.md's "13 posture categories" is substantively correct —
  its matrix has exactly 13 data rows, and categories 14-16 are governance
  consumers of the file, not rows in it. The planned 13->16 bump would have
  made the document false. Wording clarified to "code-level" instead, and the
  test now pins row count against the stated claim.

Framework currency (both verified against primary reporting):
- EU AI Act: Digital Omnibus (EP 2026-06-16, Council 2026-06-29) deferred the
  high-risk obligations behind Art. 9/15 to 2027-12-02 (Annex III) and
  2028-08-02 (Annex I); transparency still applies from 2026-08-02
- OWASP Agentic AI Top 10 labelled as the 2026 edition

Also: CLAUDE.md Distribution section rewritten monorepo -> polyrepo (each
plugin is its own repo; the catalog pins url + ref per plugin), and
current-state test counts synced 2013 -> 2034. Release-note paragraphs keep
their historical numbers.

No scanner, hook, or command behaviour changes. Suite 2034/2034.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wt4YQGoXwRja5K2Zmv8RZE
2026-08-02 21:22:20 +02:00
Kjell Tore Guttormsen
f49f9a7e4f docs(claude-md): 8.10 — fix hooks count + add doc-consistency test for hook-table sync 2026-04-30 17:12:49 +02:00
Kjell Tore Guttormsen
137057dd02 docs(scoring): unify scan/audit/mcp-scanner/posture-assessor to v2 formula
Closes the v7.1.1 out-of-scope item: commands/scan.md:113-114 retained
the v1 formula. Exploration found two more v1 surfaces that v7.1.1
missed: commands/audit.md:46 and agents/mcp-scanner-agent.md:419, plus
agents/posture-assessor-agent.md:376 (caught by the new doc-consistency
test). Four files unified to v2 in one atomic commit.

Three-way → four-way verdict-divergence is now closed:
- scanners/lib/severity.mjs (v2, BLOCK ≥65, WARNING ≥15) — authoritative
- agents/skill-scanner-agent.md (v2 since v7.1.1)
- templates/unified-report.md (v2 since v7.1.1)
- commands/scan.md (v2 — this commit)
- commands/audit.md (v2 — this commit)
- agents/mcp-scanner-agent.md (v2 — this commit)
- agents/posture-assessor-agent.md (v2 — this commit)

New: tests/lib/doc-consistency.test.mjs walks commands/ + agents/ and
asserts NO file contains v1 formula tokens. Pinned regex set:
  - score >= 61, score >= 21, score ≥ 61, score ≥ 21
  - critical * 25, Critical × 25
  - min(100, critical*25 ...)

Plus three v2-cutoff anchors asserting commands/scan.md, commands/audit.md,
and agents/mcp-scanner-agent.md document the v2 BLOCK ≥65 cutoff (or
reference riskScore() helper).

Tests: 1523 → 1551 (+28 from doc-consistency: 25 file walks + 3 anchors).
All green.
2026-04-29 13:58:25 +02:00