Commit graph

6 commits

Author SHA1 Message Date
b4d9f83521
docs: knowledge-file count from source, no offline claim for the CLI
PLAN § v8.1.3 tillegg b and c.
- b: scanner-reference.md said Knowledge Files (20); knowledge/ holds
  22. Added typosquat-allowlist.json and workflow-injection-patterns.md;
  the test pins header and table to the directory.
- c: ci-cd-guide.md claimed zero network calls, OSV opt-in and "no
  cross-border data transfer". Measured in the code: dep runs npm audit
  (package.json) and pip-audit (requirements.txt, if installed), network
  resolves found domains over DNS, supply-chain queries OSV.dev, none
  with a switch. The guide now says so. scanner-reference.md carried the
  same claim plus a `--online` flag that does not exist (0 hits in
  scanners/); fixed in the same commit because the same gate covers it
  (chosen over leaving a known-false line in a file already edited here).
  That extra check was red on fd7de23's text, verified.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:37:58 +02:00
b62c3e60f5
test(v8.1.3): red tests for plan points 1, 2, 3, 5 and addenda a-c
Each test fails on fd7de23:
- own-working-tree: venv site-packages, vendor/, config-dir skills/
  (punkt 1); ~/.claude/plugins next to $CLAUDE_CONFIG_DIR and a leading
  ~ in the variable (punkt 2); NODE_MODULES case variant and a
  case-mismatched parent segment (punkt 3, closes v8.1.2 punkt 4).
- watch-cron-scope: a watched project's own ignore file is honored
  (punkt 5).
- av-surface (b2): no runnable base64-to-shell line with a short
  command blob (addendum a; 3 hits today).
- doc-consistency: scanner-reference Knowledge Files matches knowledge/
  (addendum b); ci-cd-guide makes no offline claim (addendum c).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:29:22 +02:00
c4dfee5762 docs(llm-security): trim README badge row 8 -> 4 (repo-standard BADGE-COUNT)
repo-standard v0.2.0 flagged two README findings via coord. Verified both
against ground truth before acting:

- BADGE-COUNT (WARN, real): 8 badges confirmed by manual count. All eight
  numbers were accurate -- commands 20, agents 6, hooks 9, knowledge 23,
  scanners 22 (27 top-level .mjs minus the 5 non-scanner modules, per the
  counting rule in docs/scanner-reference.md). The finding is about clutter,
  not staleness. Operator chose to keep Version, Platform, Scanners, License
  and drop the four inventory badges (Commands/Agents/Hooks/Knowledge) --
  each of those repeats a table further down the same page and adds a sync
  obligation to every version bump.

- README-H1 (WARN, operator's call): no change. The H1 is the human-readable
  product title; llm-security is the package id in plugin.json and the
  catalog. Renaming the H1 to the slug would be a readability regression.

doc-consistency.test.mjs pinned two of the removed/kept badges. The scanners
assertion still holds; the knowledge_docs badge assertion is dropped and the
test narrowed to the README prose claims, which remain derived from
knowledge/ contents. Full suite 2034/2034.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016h37aUdBLVDT9xrvG9osA9
2026-08-04 11:50:12 +02:00
0f1be986d0 docs(llm-security): v8 Phase 2 — B10 docs consistency, counts pinned by test
Extends tests/lib/doc-consistency.test.mjs with 15 cases that derive every
inventory count from source instead of trusting prose. Each count has one
stated derivation; a doc surface that disagrees now fails the suite.

Counts corrected (all were wrong before the test existed):
- orchestrated scanners: docs said 10 (README, ci-cd-guide), CLAUDE.md said 12,
  the synthesizer agent said 9 — scan-orchestrator registers 14
- total scanners: README badge + 3 prose sites said 23; the counting rule in
  docs/scanner-reference.md (14 orchestrated + 8 standalone) yields 22
- knowledge files: README badge + prose said 22; knowledge/ holds 23
- output.mjs finding() prefix JSDoc listed 10 of the 17 prefixes actually
  passed to it (missing IDE, MCI, MEM, PST, SCR, TFA, WFL)
- norwegian-context.md said "8 hooks, 10 scanners" -> 9 and 14
- ci-cd-guide "what gets scanned" table listed 10 of 14 rows; adds workflow,
  trigger abuse, signature, AST taint

Two plan items changed after verifying against ground truth:
- CLAUDE.md's synthesizer "(12 scanners)" was not a deliberate subset; the
  agent file itself claimed 9. Both bumped to 14.
- compliance-mapping.md's "13 posture categories" is substantively correct —
  its matrix has exactly 13 data rows, and categories 14-16 are governance
  consumers of the file, not rows in it. The planned 13->16 bump would have
  made the document false. Wording clarified to "code-level" instead, and the
  test now pins row count against the stated claim.

Framework currency (both verified against primary reporting):
- EU AI Act: Digital Omnibus (EP 2026-06-16, Council 2026-06-29) deferred the
  high-risk obligations behind Art. 9/15 to 2027-12-02 (Annex III) and
  2028-08-02 (Annex I); transparency still applies from 2026-08-02
- OWASP Agentic AI Top 10 labelled as the 2026 edition

Also: CLAUDE.md Distribution section rewritten monorepo -> polyrepo (each
plugin is its own repo; the catalog pins url + ref per plugin), and
current-state test counts synced 2013 -> 2034. Release-note paragraphs keep
their historical numbers.

No scanner, hook, or command behaviour changes. Suite 2034/2034.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wt4YQGoXwRja5K2Zmv8RZE
2026-08-02 21:22:20 +02:00
Kjell Tore Guttormsen
f49f9a7e4f docs(claude-md): 8.10 — fix hooks count + add doc-consistency test for hook-table sync 2026-04-30 17:12:49 +02:00
Kjell Tore Guttormsen
137057dd02 docs(scoring): unify scan/audit/mcp-scanner/posture-assessor to v2 formula
Closes the v7.1.1 out-of-scope item: commands/scan.md:113-114 retained
the v1 formula. Exploration found two more v1 surfaces that v7.1.1
missed: commands/audit.md:46 and agents/mcp-scanner-agent.md:419, plus
agents/posture-assessor-agent.md:376 (caught by the new doc-consistency
test). Four files unified to v2 in one atomic commit.

Three-way → four-way verdict-divergence is now closed:
- scanners/lib/severity.mjs (v2, BLOCK ≥65, WARNING ≥15) — authoritative
- agents/skill-scanner-agent.md (v2 since v7.1.1)
- templates/unified-report.md (v2 since v7.1.1)
- commands/scan.md (v2 — this commit)
- commands/audit.md (v2 — this commit)
- agents/mcp-scanner-agent.md (v2 — this commit)
- agents/posture-assessor-agent.md (v2 — this commit)

New: tests/lib/doc-consistency.test.mjs walks commands/ + agents/ and
asserts NO file contains v1 formula tokens. Pinned regex set:
  - score >= 61, score >= 21, score ≥ 61, score ≥ 21
  - critical * 25, Critical × 25
  - min(100, critical*25 ...)

Plus three v2-cutoff anchors asserting commands/scan.md, commands/audit.md,
and agents/mcp-scanner-agent.md document the v2 BLOCK ≥65 cutoff (or
reference riskScore() helper).

Tests: 1523 → 1551 (+28 from doc-consistency: 25 file walks + 3 anchors).
All green.
2026-04-29 13:58:25 +02:00