1
0
Fork 0

fix(owasp): qualify every catalog number with the 2026 edition

OWASP's Top 10 for LLM Applications 2026 renumbered the list; only LLM01
and LLM02 kept their meaning. Every finding's `owasp` was written against
2025 with no year, so EchoLeak findings said LLM05, which in 2026 means
Data and Model Poisoning. Row 2 of the 2026 done-criterion counted 95
such sites at 6702d7a (82 old meaning, 13 wrong); it is now 0.

- Producers name the risk, never the number: one table (`_owasp.py`)
  owns `LLMnn:2026`. The vendored lexicon stays byte-equal with commons
  and gets its year where the finding is made; a commons number whose
  meaning moved between editions raises instead of being guessed.
- Mechanical map from the 2025 meaning (verified against the publisher
  in session 1): LLM05 -> LLM10:2026, LLM06 -> LLM03:2026,
  LLM10 -> LLM06:2026, LLM08 -> LLM09:2026, LLM04 -> LLM05:2026.

The 13 wrong-meaning sites, judged:
- Factual poisoning (PoisonedRAG gap, and the test that follows it):
  LLM05:2026. The 2026 text names "RAG Knowledge Base Poisoning" as its
  example 6. It said LLM09, which was Misinformation.
- Fail-secure (README row, matrix row): no number. It is the guard's own
  failure semantics; no edition describes it. Was LLM10.
- Parser-surface refusals (nested mapping, block scalar, quoted flow
  element, off-allowlist key): no number. Refusing a shape the parser
  would have to interpret is not resource consumption. Was LLM10. The
  anchor and merge-key rows stay LLM06:2026 (alias expansion is DoS).
- `resource` URL and cross-link scheme validation (4): no number. They
  validate received bundle metadata; LLM10:2026 is about model output
  and says input validation belongs to LLM01. Was LLM05.
- `_raise_case` default (OKF path traversal, absolute path, reserved
  name, non-.md, bundle escape): no number. None is prompt injection.
  Was LLM01.
A doubtful number would be the misreading the year exists to prevent,
so `Case.owasp` is now Optional and the matrix prints "-".

Self-safety (input caps, ReDoS) keeps LLM06:2026: the 2026 text, read at
the publisher, covers "input flood ... exploiting processing
inefficiencies" and application-side rechunking. The README's LLM09
anchor (Misinformation) is dropped; it was the factual-poisoning row.

The permanent test (tests/test_catalog_numbers.py) fails on an old or
wrong number, on a number without the year outside the vendored
lexicon, and on any finding the coverage matrix drives whose `owasp`
lacks the newest year; "newest" is read from the catalog register. The
census holds its wrong-rules to their counts at the frozen revision
only, so the frozen listing (da1aa0fa...6f) is unchanged.
This commit is contained in:
Kjell Tore Guttormsen 2026-10-31 14:20:35 +01:00
commit 1c6816cdfe
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
32 changed files with 405 additions and 145 deletions

View file

@ -5,7 +5,7 @@ active-content classes ``neutralize`` can defang — markdown images/links,
reference-link definitions, angle-bracket autolinks, raw active HTML, ``data:``
URIs — must also surface as *findings* on the standard gate, so
``screen_output`` and ``okf.import_bundle`` dispose of them instead of admitting
them silently (OWASP LLM05 — Improper Output Handling).
them silently (OWASP LLM10:2026 — Improper Output Handling).
Report-only twin of ``neutralize`` (design principles 3 & 4): it never mutates,
and severities mirror the defanger's (image / raw-html / data-uri HIGH, links
@ -41,7 +41,7 @@ def test_markdown_image_is_reported():
assert len(img) == 1
assert img[0].severity is Severity.HIGH
assert img[0].detector == "active_content"
assert img[0].owasp == "LLM05"
assert img[0].owasp == "LLM10:2026"
def test_scan_output_includes_active_content():
@ -433,7 +433,7 @@ def test_raw_html_no_longer_counts_end_tags():
assert pair[0].count == 1, f"end tag still counted: {pair[0].count}"
# --- self-safety (OWASP LLM10): the long-attribute arm -----------------------
# --- self-safety (OWASP LLM06:2026): the long-attribute arm -----------------------
# The `_REDOS_PAYLOADS` rows in test_output.py attack tags that never CLOSE, so
# `HTML_TAG_RE` fails and the tag body is never handed on. This arm is the
# opposite: the tag closes, and its body is long. `_tag` then runs
@ -483,7 +483,7 @@ def test_url_defanging_survives_the_redos_fix():
assert "http://" not in evidence and "https://" not in evidence, raw
# --- self-safety (OWASP LLM10) ----------------------------------------------
# --- self-safety (OWASP LLM06:2026) ----------------------------------------------
#
# Reached through `scan_output` this detector inherits that surface's cap. Called
# directly — the shape an adapter reaches for when it wants the active-content
@ -497,7 +497,7 @@ def test_oversize_input_is_capped_and_flagged():
oversize = [f for f in report.findings if "oversize" in f.label]
assert len(oversize) == 1
assert oversize[0].owasp == "LLM10"
assert oversize[0].owasp == "LLM06:2026"
assert oversize[0].count == len(big)
# Prefix only: the construct past the cap is not reported. This is the cost
# the flag exists to announce, so assert it rather than assume it.