fix(owasp): qualify every catalog number with the 2026 edition
OWASP's Top 10 for LLM Applications 2026 renumbered the list; only LLM01
and LLM02 kept their meaning. Every finding's `owasp` was written against
2025 with no year, so EchoLeak findings said LLM05, which in 2026 means
Data and Model Poisoning. Row 2 of the 2026 done-criterion counted 95
such sites at 6702d7a (82 old meaning, 13 wrong); it is now 0.
- Producers name the risk, never the number: one table (`_owasp.py`)
owns `LLMnn:2026`. The vendored lexicon stays byte-equal with commons
and gets its year where the finding is made; a commons number whose
meaning moved between editions raises instead of being guessed.
- Mechanical map from the 2025 meaning (verified against the publisher
in session 1): LLM05 -> LLM10:2026, LLM06 -> LLM03:2026,
LLM10 -> LLM06:2026, LLM08 -> LLM09:2026, LLM04 -> LLM05:2026.
The 13 wrong-meaning sites, judged:
- Factual poisoning (PoisonedRAG gap, and the test that follows it):
LLM05:2026. The 2026 text names "RAG Knowledge Base Poisoning" as its
example 6. It said LLM09, which was Misinformation.
- Fail-secure (README row, matrix row): no number. It is the guard's own
failure semantics; no edition describes it. Was LLM10.
- Parser-surface refusals (nested mapping, block scalar, quoted flow
element, off-allowlist key): no number. Refusing a shape the parser
would have to interpret is not resource consumption. Was LLM10. The
anchor and merge-key rows stay LLM06:2026 (alias expansion is DoS).
- `resource` URL and cross-link scheme validation (4): no number. They
validate received bundle metadata; LLM10:2026 is about model output
and says input validation belongs to LLM01. Was LLM05.
- `_raise_case` default (OKF path traversal, absolute path, reserved
name, non-.md, bundle escape): no number. None is prompt injection.
Was LLM01.
A doubtful number would be the misreading the year exists to prevent,
so `Case.owasp` is now Optional and the matrix prints "-".
Self-safety (input caps, ReDoS) keeps LLM06:2026: the 2026 text, read at
the publisher, covers "input flood ... exploiting processing
inefficiencies" and application-side rechunking. The README's LLM09
anchor (Misinformation) is dropped; it was the factual-poisoning row.
The permanent test (tests/test_catalog_numbers.py) fails on an old or
wrong number, on a number without the year outside the vendored
lexicon, and on any finding the coverage matrix drives whose `owasp`
lacks the newest year; "newest" is read from the catalog register. The
census holds its wrong-rules to their counts at the frozen revision
only, so the frozen listing (da1aa0fa...6f) is unchanged.
This commit is contained in:
parent
1241075ef8
commit
1c6816cdfe
32 changed files with 405 additions and 145 deletions
|
|
@ -146,7 +146,7 @@ As of `v1.4.2`: **136 / 136 defended classes demonstrated (recall 100%)** and **
|
|||
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
|
||||
update). The matrix is the single source of truth for the test suite (**1049
|
||||
passing**), which also asserts total recall, that every lexicon pattern has a
|
||||
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
|
||||
case (so the matrix cannot fall behind the lexicon), the full LLM02:2026 secret-egress
|
||||
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,
|
||||
symlink). Run it once; it tells you exactly what assurance you are buying.
|
||||
|
||||
|
|
@ -233,7 +233,7 @@ Conceding these is itself a control. The full list is in the guard's `README.md`
|
|||
`resource`, cross-link graph, reserved names, provenance) the adapter implements.
|
||||
- `python -m llm_ingestion_guard.coverage` — the runnable "verify what it stops".
|
||||
|
||||
Threat-model anchors: OWASP LLM Top-10 2025 (LLM01/02/04/05/06 strongest, LLM08
|
||||
boundary), PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak
|
||||
Threat-model anchors: OWASP Top 10 for LLM Applications 2026 (LLM01:2026,
|
||||
LLM02:2026, LLM03:2026, LLM05:2026, LLM10:2026 strongest; LLM09:2026 boundary), PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak
|
||||
(CVE-2025-32711). OKF: Google Cloud Open Knowledge Format v0.1 (announced
|
||||
2026-06-12) — `GoogleCloudPlatform/knowledge-catalog/okf/SPEC.md`.
|
||||
|
|
|
|||
|
|
@ -206,7 +206,7 @@ The actual product is this checklist, encoded as code a pipeline wires in order:
|
|||
|
||||
## 10. Threat-model grounding
|
||||
|
||||
- OWASP LLM Top 10: **LLM01 Prompt Injection**, output-handling, and the RAG /
|
||||
- OWASP LLM Top 10: **LLM01:2026 Prompt Injection**, output-handling, and the RAG /
|
||||
knowledge-poisoning vectors.
|
||||
- Google DeepMind "AI Agent Traps" (Franklin et al., 2025): Latent Memory
|
||||
Poisoning and RAG Knowledge Poisoning map directly onto the write-time artifact.
|
||||
|
|
|
|||
|
|
@ -218,7 +218,7 @@ items; this is the full list, each with the mechanism.
|
|||
(≥32 chars), or Shannon entropy ≥4.4 at ≥24 chars. A shorter payload segment —
|
||||
`https://evil.test/aGVsbG8gd29ybGQ` — cannot be told from a name, because entropy
|
||||
is bounded by `log2(length)` at short lengths. Mitigation in depth, not in this
|
||||
detector: a literal credential in a URL is still caught by the LLM02 egress
|
||||
detector: a literal credential in a URL is still caught by the LLM02:2026 egress
|
||||
patterns in the same `scan_output` pass, whatever severity the carrier gets.
|
||||
- **Percent-escapes count as data-carrying — a `%20` in a path is a false positive.**
|
||||
An ordinary link with an encoded space grades as carrying and reaches
|
||||
|
|
@ -707,7 +707,7 @@ items; this is the full list, each with the mechanism.
|
|||
`okf.link_graph`, whose cost was a bundle-wide `findall` over every document
|
||||
body. **What truncation costs is worth naming: past the cap, "no finding" means
|
||||
"not looked at".** Each says so rather than staying silent — the scanners emit
|
||||
an `oversize-input` finding (`active:oversize-input`, OWASP LLM10), and
|
||||
an `oversize-input` finding (`active:oversize-input`, OWASP LLM06:2026), and
|
||||
`link_graph` records `(from_id, body_length)` in `LinkGraphResult.truncated`,
|
||||
which is what lets a caller tell "no links past here" apart from "no links
|
||||
*read* past here". **The split itself is conceded for 1.x, not deferred.** It is
|
||||
|
|
@ -841,5 +841,5 @@ fails the test, forcing this doc to be updated:
|
|||
|
||||
## Out-of-scope (documented boundary)
|
||||
|
||||
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
||||
Embedding/vector-layer defenses (OWASP LLM09:2026, downstream of persist); multimodal
|
||||
steganography; query-time / runtime guardrails; semantic factuality verification.
|
||||
|
|
|
|||
24
docs/PLAN.md
24
docs/PLAN.md
|
|
@ -206,6 +206,30 @@ CHANGELOG for 1.4.2.
|
|||
årstall som ikke er nyeste, i sporede filer utenom historikk.
|
||||
- CHANGELOG-merknaden sier rett ut at en bruker som filtrerer på `"LLM05"`, mister
|
||||
treff. Konsumentene varsles.
|
||||
*Gjort 31.10 (1.5.0):* **rad 2 står på 0** (var 95). Den frosne nevneren er uendret
|
||||
ved `6702d7a` (95, `da1aa0fa…6f`), og tellingen over arbeidstreet gir 0 gammel og 0
|
||||
feil. Den permanente testen er `tests/test_catalog_numbers.py`. Den feiler på
|
||||
nummer med gammel eller feil betydning, på nummer uten årstall (unntatt
|
||||
leksikondataene fra commons) og på funn fra matrisen som mangler nyeste årstall.
|
||||
Hva som er «nyeste», leses fra katalogregisteret. Produsentene navngir risikoen,
|
||||
ikke nummeret: én tabell (`_owasp.py`) eier numrene. Leksikonet er byte-likt med
|
||||
commons og får årstallet når det lastes. Et nummer derfra som har flyttet seg
|
||||
mellom utgavene, avvises i stedet for å gjettes.
|
||||
De 13 stedene med feil betydning er vurdert slik:
|
||||
- Faktaforgiftning (PoisonedRAG) er **LLM05:2026** (2 steder: matriseraden og
|
||||
testen som følger den). Det er 2026-tekstens eget eksempel 6, «RAG Knowledge
|
||||
Base Poisoning».
|
||||
- De andre 11 stedene er fail-secure (README og matrisen), parserflate (4),
|
||||
URL-validering av mottatt metadata (4) og standardverdien som merket
|
||||
OKF-stiavvisninger som prompt injection (1 sted, 5 rader). De får **ikke noe
|
||||
nummer**, fordi ingen utgave beskriver dem. Et tvilsomt nummer ville vært den
|
||||
samme feilen på nytt. I matrisen er det 14 rader uten nummer, og hver av dem er
|
||||
festet i testen.
|
||||
Selvsikkerhet beholder **LLM06:2026**. 2026-teksten er lest hos utgiveren: eksempel
|
||||
1 («input flood … exploiting processing inefficiencies … render the system
|
||||
unresponsive») og eksempel 3 (applikasjonssidig re-chunking) dekker inputtak og
|
||||
ReDoS. README-ankeret «LLM09» (Misinformation) er tatt ut. Det var
|
||||
faktaforgiftnings-raden, og grounding-kroken er for den.
|
||||
|
||||
**Økt 4: katalogregisteret i bruk (rad 1). Ren dokumentasjon, gis ut med neste
|
||||
release.**
|
||||
|
|
|
|||
|
|
@ -82,7 +82,11 @@ HISTORY_FILES = {
|
|||
"docs/2026-09-07-limitations-44-maaling.md": "published measurement",
|
||||
}
|
||||
|
||||
SELF_PREFIXES = ("docs/catalog-census.py", "docs/threat-2026/")
|
||||
SELF_PREFIXES = ("docs/catalog-census.py", "docs/threat-2026/", "tests/test_catalog_numbers.py")
|
||||
|
||||
# The newest edition. `tests/test_catalog_numbers.py` asserts it equals the
|
||||
# register's `latest`, so the two cannot drift apart.
|
||||
CURRENT_YEAR = "2026"
|
||||
|
||||
# `docs/PLAN.md` is split by its top-level headings.
|
||||
PLAN = "docs/PLAN.md"
|
||||
|
|
@ -179,7 +183,7 @@ def census(rev: str | None) -> list[tuple[str, ...]]:
|
|||
cls, reason = "history", HISTORY_FILES[path]
|
||||
elif sections[i]:
|
||||
cls = sections[i]
|
||||
elif year == "2026":
|
||||
elif year == CURRENT_YEAR:
|
||||
cls = "current"
|
||||
else:
|
||||
cls = "same" if nn in ("01", "02") else "old"
|
||||
|
|
@ -195,7 +199,12 @@ def census(rev: str | None) -> list[tuple[str, ...]]:
|
|||
rows.append((cls, path, str(i + 1), f"LLM{nn}", year,
|
||||
TITLES_2025[nn], f"LLM{MAP_2025_TO_2026[nn]}:2026", reason))
|
||||
stale = [(r[0], r[1], r[3], hits) for r, hits in zip(WRONG_RULES, rule_hits) if hits != r[3]]
|
||||
if stale:
|
||||
# The rules judge the sites of the FROZEN denominator, so they are held to
|
||||
# their counts there. In a later tree a judged site is meant to be fixed --
|
||||
# it then carries the year and never reaches a rule -- and a site a rule no
|
||||
# longer reaches still counts as `old` or `same`, which the permanent test
|
||||
# refuses as well. A stale rule there changes a label, never the total.
|
||||
if stale and rev is not None:
|
||||
raise SystemExit(f"stale wrong-rules (path, substring, expected, matched): {stale}")
|
||||
return rows
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue