1
0
Fork 0

fix(owasp): qualify every catalog number with the 2026 edition

OWASP's Top 10 for LLM Applications 2026 renumbered the list; only LLM01
and LLM02 kept their meaning. Every finding's `owasp` was written against
2025 with no year, so EchoLeak findings said LLM05, which in 2026 means
Data and Model Poisoning. Row 2 of the 2026 done-criterion counted 95
such sites at 6702d7a (82 old meaning, 13 wrong); it is now 0.

- Producers name the risk, never the number: one table (`_owasp.py`)
  owns `LLMnn:2026`. The vendored lexicon stays byte-equal with commons
  and gets its year where the finding is made; a commons number whose
  meaning moved between editions raises instead of being guessed.
- Mechanical map from the 2025 meaning (verified against the publisher
  in session 1): LLM05 -> LLM10:2026, LLM06 -> LLM03:2026,
  LLM10 -> LLM06:2026, LLM08 -> LLM09:2026, LLM04 -> LLM05:2026.

The 13 wrong-meaning sites, judged:
- Factual poisoning (PoisonedRAG gap, and the test that follows it):
  LLM05:2026. The 2026 text names "RAG Knowledge Base Poisoning" as its
  example 6. It said LLM09, which was Misinformation.
- Fail-secure (README row, matrix row): no number. It is the guard's own
  failure semantics; no edition describes it. Was LLM10.
- Parser-surface refusals (nested mapping, block scalar, quoted flow
  element, off-allowlist key): no number. Refusing a shape the parser
  would have to interpret is not resource consumption. Was LLM10. The
  anchor and merge-key rows stay LLM06:2026 (alias expansion is DoS).
- `resource` URL and cross-link scheme validation (4): no number. They
  validate received bundle metadata; LLM10:2026 is about model output
  and says input validation belongs to LLM01. Was LLM05.
- `_raise_case` default (OKF path traversal, absolute path, reserved
  name, non-.md, bundle escape): no number. None is prompt injection.
  Was LLM01.
A doubtful number would be the misreading the year exists to prevent,
so `Case.owasp` is now Optional and the matrix prints "-".

Self-safety (input caps, ReDoS) keeps LLM06:2026: the 2026 text, read at
the publisher, covers "input flood ... exploiting processing
inefficiencies" and application-side rechunking. The README's LLM09
anchor (Misinformation) is dropped; it was the factual-poisoning row.

The permanent test (tests/test_catalog_numbers.py) fails on an old or
wrong number, on a number without the year outside the vendored
lexicon, and on any finding the coverage matrix drives whose `owasp`
lacks the newest year; "newest" is read from the catalog register. The
census holds its wrong-rules to their counts at the frozen revision
only, so the frozen listing (da1aa0fa...6f) is unchanged.
This commit is contained in:
Kjell Tore Guttormsen 2026-10-31 14:20:35 +01:00
commit 1c6816cdfe
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
32 changed files with 405 additions and 145 deletions

View file

@ -146,7 +146,7 @@ As of `v1.4.2`: **136 / 136 defended classes demonstrated (recall 100%)** and **
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
update). The matrix is the single source of truth for the test suite (**1049
passing**), which also asserts total recall, that every lexicon pattern has a
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
case (so the matrix cannot fall behind the lexicon), the full LLM02:2026 secret-egress
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,
symlink). Run it once; it tells you exactly what assurance you are buying.
@ -233,7 +233,7 @@ Conceding these is itself a control. The full list is in the guard's `README.md`
`resource`, cross-link graph, reserved names, provenance) the adapter implements.
- `python -m llm_ingestion_guard.coverage` — the runnable "verify what it stops".
Threat-model anchors: OWASP LLM Top-10 2025 (LLM01/02/04/05/06 strongest, LLM08
boundary), PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak
Threat-model anchors: OWASP Top 10 for LLM Applications 2026 (LLM01:2026,
LLM02:2026, LLM03:2026, LLM05:2026, LLM10:2026 strongest; LLM09:2026 boundary), PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak
(CVE-2025-32711). OKF: Google Cloud Open Knowledge Format v0.1 (announced
2026-06-12) — `GoogleCloudPlatform/knowledge-catalog/okf/SPEC.md`.

View file

@ -206,7 +206,7 @@ The actual product is this checklist, encoded as code a pipeline wires in order:
## 10. Threat-model grounding
- OWASP LLM Top 10: **LLM01 Prompt Injection**, output-handling, and the RAG /
- OWASP LLM Top 10: **LLM01:2026 Prompt Injection**, output-handling, and the RAG /
knowledge-poisoning vectors.
- Google DeepMind "AI Agent Traps" (Franklin et al., 2025): Latent Memory
Poisoning and RAG Knowledge Poisoning map directly onto the write-time artifact.

View file

@ -218,7 +218,7 @@ items; this is the full list, each with the mechanism.
(≥32 chars), or Shannon entropy ≥4.4 at ≥24 chars. A shorter payload segment —
`https://evil.test/aGVsbG8gd29ybGQ` — cannot be told from a name, because entropy
is bounded by `log2(length)` at short lengths. Mitigation in depth, not in this
detector: a literal credential in a URL is still caught by the LLM02 egress
detector: a literal credential in a URL is still caught by the LLM02:2026 egress
patterns in the same `scan_output` pass, whatever severity the carrier gets.
- **Percent-escapes count as data-carrying — a `%20` in a path is a false positive.**
An ordinary link with an encoded space grades as carrying and reaches
@ -707,7 +707,7 @@ items; this is the full list, each with the mechanism.
`okf.link_graph`, whose cost was a bundle-wide `findall` over every document
body. **What truncation costs is worth naming: past the cap, "no finding" means
"not looked at".** Each says so rather than staying silent — the scanners emit
an `oversize-input` finding (`active:oversize-input`, OWASP LLM10), and
an `oversize-input` finding (`active:oversize-input`, OWASP LLM06:2026), and
`link_graph` records `(from_id, body_length)` in `LinkGraphResult.truncated`,
which is what lets a caller tell "no links past here" apart from "no links
*read* past here". **The split itself is conceded for 1.x, not deferred.** It is
@ -841,5 +841,5 @@ fails the test, forcing this doc to be updated:
## Out-of-scope (documented boundary)
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
Embedding/vector-layer defenses (OWASP LLM09:2026, downstream of persist); multimodal
steganography; query-time / runtime guardrails; semantic factuality verification.

View file

@ -206,6 +206,30 @@ CHANGELOG for 1.4.2.
årstall som ikke er nyeste, i sporede filer utenom historikk.
- CHANGELOG-merknaden sier rett ut at en bruker som filtrerer på `"LLM05"`, mister
treff. Konsumentene varsles.
*Gjort 31.10 (1.5.0):* **rad 2 står på 0** (var 95). Den frosne nevneren er uendret
ved `6702d7a` (95, `da1aa0fa…6f`), og tellingen over arbeidstreet gir 0 gammel og 0
feil. Den permanente testen er `tests/test_catalog_numbers.py`. Den feiler på
nummer med gammel eller feil betydning, på nummer uten årstall (unntatt
leksikondataene fra commons) og på funn fra matrisen som mangler nyeste årstall.
Hva som er «nyeste», leses fra katalogregisteret. Produsentene navngir risikoen,
ikke nummeret: én tabell (`_owasp.py`) eier numrene. Leksikonet er byte-likt med
commons og får årstallet når det lastes. Et nummer derfra som har flyttet seg
mellom utgavene, avvises i stedet for å gjettes.
De 13 stedene med feil betydning er vurdert slik:
- Faktaforgiftning (PoisonedRAG) er **LLM05:2026** (2 steder: matriseraden og
testen som følger den). Det er 2026-tekstens eget eksempel 6, «RAG Knowledge
Base Poisoning».
- De andre 11 stedene er fail-secure (README og matrisen), parserflate (4),
URL-validering av mottatt metadata (4) og standardverdien som merket
OKF-stiavvisninger som prompt injection (1 sted, 5 rader). De får **ikke noe
nummer**, fordi ingen utgave beskriver dem. Et tvilsomt nummer ville vært den
samme feilen på nytt. I matrisen er det 14 rader uten nummer, og hver av dem er
festet i testen.
Selvsikkerhet beholder **LLM06:2026**. 2026-teksten er lest hos utgiveren: eksempel
1 («input flood … exploiting processing inefficiencies … render the system
unresponsive») og eksempel 3 (applikasjonssidig re-chunking) dekker inputtak og
ReDoS. README-ankeret «LLM09» (Misinformation) er tatt ut. Det var
faktaforgiftnings-raden, og grounding-kroken er for den.
**Økt 4: katalogregisteret i bruk (rad 1). Ren dokumentasjon, gis ut med neste
release.**

View file

@ -82,7 +82,11 @@ HISTORY_FILES = {
"docs/2026-09-07-limitations-44-maaling.md": "published measurement",
}
SELF_PREFIXES = ("docs/catalog-census.py", "docs/threat-2026/")
SELF_PREFIXES = ("docs/catalog-census.py", "docs/threat-2026/", "tests/test_catalog_numbers.py")
# The newest edition. `tests/test_catalog_numbers.py` asserts it equals the
# register's `latest`, so the two cannot drift apart.
CURRENT_YEAR = "2026"
# `docs/PLAN.md` is split by its top-level headings.
PLAN = "docs/PLAN.md"
@ -179,7 +183,7 @@ def census(rev: str | None) -> list[tuple[str, ...]]:
cls, reason = "history", HISTORY_FILES[path]
elif sections[i]:
cls = sections[i]
elif year == "2026":
elif year == CURRENT_YEAR:
cls = "current"
else:
cls = "same" if nn in ("01", "02") else "old"
@ -195,7 +199,12 @@ def census(rev: str | None) -> list[tuple[str, ...]]:
rows.append((cls, path, str(i + 1), f"LLM{nn}", year,
TITLES_2025[nn], f"LLM{MAP_2025_TO_2026[nn]}:2026", reason))
stale = [(r[0], r[1], r[3], hits) for r, hits in zip(WRONG_RULES, rule_hits) if hits != r[3]]
if stale:
# The rules judge the sites of the FROZEN denominator, so they are held to
# their counts there. In a later tree a judged site is meant to be fixed --
# it then carries the year and never reaches a rule -- and a site a rule no
# longer reaches still counts as `old` or `same`, which the permanent test
# refuses as well. A stale rule there changes a label, never the total.
if stale and rev is not None:
raise SystemExit(f"stale wrong-rules (path, substring, expected, matched): {stale}")
return rows