Commit graph

6 commits

Author SHA1 Message Date
Kjell Tore Guttormsen
d3b1157a08 docs(scoring): unify scan/audit/mcp-scanner/posture-assessor to v2 formula
Closes the v7.1.1 out-of-scope item: commands/scan.md:113-114 retained
the v1 formula. Exploration found two more v1 surfaces that v7.1.1
missed: commands/audit.md:46 and agents/mcp-scanner-agent.md:419, plus
agents/posture-assessor-agent.md:376 (caught by the new doc-consistency
test). Four files unified to v2 in one atomic commit.

Three-way → four-way verdict-divergence is now closed:
- scanners/lib/severity.mjs (v2, BLOCK ≥65, WARNING ≥15) — authoritative
- agents/skill-scanner-agent.md (v2 since v7.1.1)
- templates/unified-report.md (v2 since v7.1.1)
- commands/scan.md (v2 — this commit)
- commands/audit.md (v2 — this commit)
- agents/mcp-scanner-agent.md (v2 — this commit)
- agents/posture-assessor-agent.md (v2 — this commit)

New: tests/lib/doc-consistency.test.mjs walks commands/ + agents/ and
asserts NO file contains v1 formula tokens. Pinned regex set:
  - score >= 61, score >= 21, score ≥ 61, score ≥ 21
  - critical * 25, Critical × 25
  - min(100, critical*25 ...)

Plus three v2-cutoff anchors asserting commands/scan.md, commands/audit.md,
and agents/mcp-scanner-agent.md document the v2 BLOCK ≥65 cutoff (or
reference riskScore() helper).

Tests: 1523 → 1551 (+28 from doc-consistency: 25 file walks + 3 anchors).
All green.
2026-04-29 13:58:25 +02:00
Kjell Tore Guttormsen
67ffff13a4 fix(llm-security): skill-scanner-agent — context-first severity, v2 alignment, Suppressed Signals section
Five coordinated edits to address scan-rapport whiplash at the agent
prompt level:

- Step 2.5 (NEW): Context-First Severity Assignment. Every signal has
  exactly one disposition — suppressed (counted only) or reported (full
  finding). The split happens BEFORE severity is assigned. Forbids
  'false positive', 'legitimate framework', 'no action required' in
  finding-body text; reserves them for the Suppressed Signals section.
- Verdict Logic: replaces stale v1 sum-and-cap formula (BLOCK >=61) with
  v2 reference (severity-dominated, BLOCK >=65) matching severity.mjs
  since v7.0.0. Documents that severity counts MUST exclude suppressed
  signals; introduces verdict_rationale field for descriptive context
  when suppressed >= 5 AND reported <= 1 high.
- Output Format: adds Suppressed Signals as required section #4 with
  category-level bullet format. Documents the trailing JSON shape
  including summary.narrative_audit.suppressed_findings.{count,
  by_category} and verdict_rationale fields.
- Comment block before Category 2 suppression rules clarifies that
  'false positive' as taxonomy language is OK; only finding-body
  description fields are forbidden from using the phrase.
- Step 0 (Norwegian generaliseringsgrense) preserved unchanged.

Part of v7.1.1 narrative-coherence patch (plan: .claude/plans/ultraplan-2026-04-29-report-coherence.md).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 12:47:58 +02:00
Kjell Tore Guttormsen
915aca69e4 feat(llm-security): v7.0.0 commit 5 — synthesizer scan calibration section
Makes suppression stats visible in the deep-scan report so users can
audit why the scanner produced the counts it did. Before: synthesizer
would acknowledge "true risk is High, not Extreme" in prose while
verdict stayed BLOCK/Extreme — inconsistent. After Commit 1 the
orchestrator verdict is coherent on its own; synthesizer's job shrinks
to transparency.

- Adds 'Scan Calibration' section instruction consuming
  scanner.calibration.* fields (entropy files_skipped_by_extension,
  policy_source, thresholds).
- Heuristic: omit the section if < 5% of files skipped (no signal).
  Flag the section if > 80% skipped (policy may be too aggressive).
- Explicit 'Don't override verdict' directive in DON'T DO list.
  Discrepancy goes in calibration, not in a rewritten dashboard.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 22:04:29 +02:00
Kjell Tore Guttormsen
931ad388e7 refactor(agents): add Step 0 + parallel-read hint to mcp-scanner-agent 2026-04-17 14:46:30 +02:00
Kjell Tore Guttormsen
b55c21bcaf refactor(agents): reduce stacked imperatives in skill-scanner-agent for Opus 4.7 2026-04-17 14:46:06 +02:00
Kjell Tore Guttormsen
f93d6abdae feat: initial open marketplace with llm-security, config-audit, ultraplan-local 2026-04-06 18:47:49 +02:00