test(coverage): runnable threat-coverage matrix (real-case validation gate)
Add a single declarative manifest proving, in one place, every vulnerability class the guard stops — and the documented gaps it does not. This is the real-case validation gate ahead of any v1.0 freeze (v1.0 stays parked until verified on real cases). - src/llm_ingestion_guard/coverage.py: stdlib-only manifest (CORE_CASES) + narrated runner. `python -m llm_ingestion_guard.coverage` prints class -> OWASP -> expected -> observed -> verdict; exit 0 iff every caught class is caught and every documented gap holds. 126 caught classes + 4 gaps. Lexicon cases are generated from load_lexicon() via a payload dict, so a pattern with no payload fails loudly at import (self-verifying). - tests/test_coverage_matrix.py: asserts total recall, that every documented gap still holds, and completeness (every lexicon id + every OWASP anchor has a case). Adds the full 25-pattern LLM02 secret-egress set (fixtures assembled from split tokens so no secret shape sits in source) and the container-layer front-end classes (CSV formula-injection, zip-slip, zip-bomb, symlink). - README + CHANGELOG: point to the runnable matrix. +165 tests (357 -> 522). No core dependency added.
This commit is contained in:
parent
66f3cbf4f5
commit
f5eae9a16e
4 changed files with 853 additions and 0 deletions
21
CHANGELOG.md
21
CHANGELOG.md
|
|
@ -5,6 +5,27 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added — runnable threat-coverage matrix
|
||||
|
||||
A single declarative manifest (`llm_ingestion_guard.coverage`) that proves, in one
|
||||
place, every vulnerability class the guard stops — and the documented gaps it does
|
||||
not. Two consumers of the same source of truth:
|
||||
|
||||
- `python -m llm_ingestion_guard.coverage` — a narrated matrix
|
||||
(`class -> OWASP -> expected -> observed -> verdict`); exit 0 iff every caught
|
||||
class is caught and every documented gap holds. Stdlib-only, CI-usable.
|
||||
- `tests/test_coverage_matrix.py` — asserts total recall over the core matrix
|
||||
(carriers, all 83 lexicon patterns, entropy/decode-rescan, active content, the
|
||||
contract asserters, the disposition engine, OKF T1–T7), asserts every documented
|
||||
gap still holds, and guards completeness (every lexicon pattern id, and every
|
||||
OWASP anchor claimed, has a case). Adds the full 25-pattern LLM02 secret-egress
|
||||
set and the container-layer front-end classes (CSV formula-injection, zip-slip,
|
||||
zip-bomb, symlink). +165 tests (357 → 522), no core dependency added.
|
||||
|
||||
This is the real-case validation gate ahead of a v1.0 freeze.
|
||||
|
||||
## [0.2.0] — 2026-07-06
|
||||
|
||||
### Added — OKF adapter (stream 1)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue