1
0
Fork 0
llm-ingestion-pipeline-secu.../tests
Kjell Tore Guttormsen 1c6816cdfe
fix(owasp): qualify every catalog number with the 2026 edition
OWASP's Top 10 for LLM Applications 2026 renumbered the list; only LLM01
and LLM02 kept their meaning. Every finding's `owasp` was written against
2025 with no year, so EchoLeak findings said LLM05, which in 2026 means
Data and Model Poisoning. Row 2 of the 2026 done-criterion counted 95
such sites at 6702d7a (82 old meaning, 13 wrong); it is now 0.

- Producers name the risk, never the number: one table (`_owasp.py`)
  owns `LLMnn:2026`. The vendored lexicon stays byte-equal with commons
  and gets its year where the finding is made; a commons number whose
  meaning moved between editions raises instead of being guessed.
- Mechanical map from the 2025 meaning (verified against the publisher
  in session 1): LLM05 -> LLM10:2026, LLM06 -> LLM03:2026,
  LLM10 -> LLM06:2026, LLM08 -> LLM09:2026, LLM04 -> LLM05:2026.

The 13 wrong-meaning sites, judged:
- Factual poisoning (PoisonedRAG gap, and the test that follows it):
  LLM05:2026. The 2026 text names "RAG Knowledge Base Poisoning" as its
  example 6. It said LLM09, which was Misinformation.
- Fail-secure (README row, matrix row): no number. It is the guard's own
  failure semantics; no edition describes it. Was LLM10.
- Parser-surface refusals (nested mapping, block scalar, quoted flow
  element, off-allowlist key): no number. Refusing a shape the parser
  would have to interpret is not resource consumption. Was LLM10. The
  anchor and merge-key rows stay LLM06:2026 (alias expansion is DoS).
- `resource` URL and cross-link scheme validation (4): no number. They
  validate received bundle metadata; LLM10:2026 is about model output
  and says input validation belongs to LLM01. Was LLM05.
- `_raise_case` default (OKF path traversal, absolute path, reserved
  name, non-.md, bundle escape): no number. None is prompt injection.
  Was LLM01.
A doubtful number would be the misreading the year exists to prevent,
so `Case.owasp` is now Optional and the matrix prints "-".

Self-safety (input caps, ReDoS) keeps LLM06:2026: the 2026 text, read at
the publisher, covers "input flood ... exploiting processing
inefficiencies" and application-side rechunking. The README's LLM09
anchor (Misinformation) is dropped; it was the factual-poisoning row.

The permanent test (tests/test_catalog_numbers.py) fails on an old or
wrong number, on a number without the year outside the vendored
lexicon, and on any finding the coverage matrix drives whose `owasp`
lacks the newest year; "newest" is read from the catalog register. The
census holds its wrong-rules to their counts at the frozen revision
only, so the frozen listing (da1aa0fa...6f) is unchanged.
2026-10-31 14:20:35 +01:00
..
inbox_frontend.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
redos_clock.py test(redos): the dead row had the wrong payload, and the last wall clock is retired 2026-08-13 21:47:43 +02:00
test_active_content.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_attacks_2026.py fix(sanitize): catch every invisible codepoint, exempt only on context 2026-10-29 08:31:43 +01:00
test_calibration.py feat(active-content): raw HTML graded on carrier, and a tag naming no target is inert 2026-08-12 00:42:44 +02:00
test_catalog_census.py test(threat-2026): freeze the catalog-number census at v1.4.1 2026-10-28 21:27:31 +01:00
test_catalog_numbers.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_contract.py fix(security): harden 5 adversarial-review findings (M1/M2/M3 + m4/m6) via TDD 2026-07-05 10:45:05 +02:00
test_corpus.py measure(fp): the upload door costs a human on 16-53% of benign documents 2026-08-10 21:32:56 +02:00
test_coverage_matrix.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_disposition.py docs(plan): kill the pointer that sent this session to the wrong section 2026-08-10 21:01:19 +02:00
test_docs_measurement_scripts.py fix(sanitize): catch every invisible codepoint, exempt only on context 2026-10-29 08:31:43 +01:00
test_entropy.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_fence.py feat(fence): randomized unspoofable delimiter + attacker marker-strip (TDD) [skip-docs] 2026-07-04 18:23:35 +02:00
test_grounding.py feat(grounding): SourceGroundingCheck protocol + pass-through default — the semantic-poisoning seam (TDD) [skip-docs] 2026-07-04 22:44:41 +02:00
test_input_cap.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_invisible.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_lexicon.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_neutralize.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_okf.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_okf_inbox_uploads.py feat(inbox): .xlsx extraction — formula gate, hidden sheets, cell comments (stage 2h) 2026-07-07 07:41:00 +02:00
test_okf_showcase.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_output.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_report.py feat: scaffold package + report and sanitize modules (TDD) 2026-07-04 09:24:20 +02:00
test_sanitize.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_showcase.py fix(owasp): qualify every catalog number with the 2026 edition 2026-10-31 14:20:35 +01:00
test_url_shape_doc.py docs(url-shape): make the rule reconstructable, and record what three corpora measured 2026-07-27 08:56:24 +02:00
test_wiring.py feat(active-content): raw HTML graded on carrier, and a tag naming no target is inert 2026-08-12 00:42:44 +02:00