Commit graph

2 commits

Author SHA1 Message Date
14070160ba test(llm-security): cover TRG/SIG/AST in e2e pipeline + SIG miner/hacktool/rot13 families (#58,#59)
#58 the e2e scan-pipeline suite omitted trg/sig/ast from EXPECTED_SCANNERS and no fixture carried trigger-abuse/known-signature/python-taint content, so the three v7.8.0 scanners' surfacing through the aggregate was never asserted; added a fixture (shadowing command name → TRG, webshell → SIG, tainted os.environ→requests.post → AST, python3-guarded) that asserts each appears in the envelope and the rolled-up counts/owasp_breakdown. #59 the SIG cryptominer (SIG-MINER-001/002) and hacktool (SIG-HACKTOOL-001) families and non-base64 decode variants had no test; added family-attribution tests and a rot13-decode assertion. Test-only, suite 2013/0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01TcQyMTQfyrsAapaCMPxTtQ
2026-07-18 10:46:19 +02:00
b5d29f8cd6 test(llm-security): add e2e suite proving framework works as coordinated system
Three new files in tests/e2e/ (45 tests, 1777 -> 1822):

- attack-chain.test.mjs (17): full hook stack against attack payloads in
  sequence -- prompt injection at the gate; T1/T5/T8 bash evasions;
  pathguard on .env / .ssh; secrets hook on AWS-shaped keys and PEM
  headers; markdown link-title and HTML-comment poisoning in tool
  output; trifecta accumulation over a single session with dedup on
  the next benign call.

- multi-session.test.mjs (9): state persistence across simulated
  session boundaries. Uses the fact that a hook child's process.ppid
  equals the test runner's process.pid, so writing the session state
  file directly simulates "previous session" history. Covers slow-burn
  trifecta (legs spread >50 calls), MCP cumulative description drift
  via LLM_SECURITY_MCP_CACHE_FILE override, and pre-compact transcript
  poisoning in warn / block / clean / missing-file modes.

- scan-pipeline.test.mjs (19): scan-orchestrator + all 10 scanners +
  toxic-flow correlator against poisoned-project (BLOCK / 95 / Extreme)
  and grade-a-project (WARNING / 48 / High). Asserts envelope shape,
  verdict, risk_score, severity counts, OWASP coverage, scanner
  enumeration, and a narrative-coherence cross-check that the BLOCK
  scan strictly outranks the WARNING scan along every axis.

Test files build credential-shaped payloads at runtime via concatenation
so they contain no literal matches for the pre-edit-secrets regexes
(memory rule feedback_secrets_hook_test_fixtures.md).

Doc updates in same commit per marketplace policy:
- CLAUDE.md header: 1777+ -> 1822+ tests, mentions tests/e2e/
- README.md badge tests-1777 -> tests-1822, body text updated
- CHANGELOG.md: new [Unreleased] Added section describing scope

No version bump. No behavior changes outside tests/.
2026-05-05 12:06:57 +02:00