feat(watch): source list for the 16 catalogs and the 30-day release rule
docs/threat-2026/watch-sources.json is the catalog register in the edition watch's input format: 12 entries probed (A, B, F, L x2, O x7), 4 manual as class S with a reason. Run live on 2026-11-01: 16 current, 12 probes ok. A frozen copy of the catalogs as v1.4.1 used them is reported behind (OWASP LLM 2025) and on watch (RAG cheat sheet); the watch calls its Unicode 17.0 current, which only the 30-day clause catches. tests/test_release_rule.py stops a release whose manual check or watch run is more than 30 days older than the newest CHANGELOG date, or whose watch output has a behind, notVerified or watch row. Each clause mutated away turns a test red (5 of 5). No release: the package is unchanged.
This commit is contained in:
parent
56fcc844d6
commit
c4ecf0dba6
7 changed files with 973 additions and 4 deletions
64
docs/PLAN.md
64
docs/PLAN.md
|
|
@ -24,7 +24,7 @@ Lest slik (antakelser, ikke operatørens ord):
|
|||
- Ikke tatt med (operatøren valgte bort «bare rydding»): blokk-skalaren `a #b`,
|
||||
støyen i ReDoS-sveipet og repo-standard-sjekken. De står under § V.5.
|
||||
|
||||
**Dekket i:** katalogene over tid → § V.2 rad 1, økt 2 · hullene → rad 2, økt 1
|
||||
**Dekket i:** katalogene over tid → § V.2 rad 1, økt 2 (§ V.7) · hullene → rad 2, økt 1
|
||||
(måling) og økt 3 og utover (lukking) · ferdig → siste økt.
|
||||
|
||||
---
|
||||
|
|
@ -190,6 +190,68 @@ så de kommer med i samme release som den første lukkingen.
|
|||
|
||||
---
|
||||
|
||||
## V.7 Økt 2, vakten (01.11.2026, ingen release)
|
||||
|
||||
**Kildelista** er `docs/threat-2026/watch-sources.json`: de 16 i registeret, i vaktens
|
||||
egen form (en liste med ett `probe`-objekt per oppføring). Den er generert fra
|
||||
`catalogs.json`, og `tests/test_release_rule.py` holder dem like: samme id-er, samme
|
||||
`cited`, `latest`, `source` og `checked`. 12 oppføringer har en probe som gjør en
|
||||
forespørsel (A, B, F, L×2, O×7, som målt i § V.6). De 4 manuelle (unicode, rfc-3986,
|
||||
commons, Willison) står med klasse S. Det er vaktens egen form for «ingen maskin
|
||||
kan svare» (llm-security bruker den på 6 av sine 58). Hver har en begrunnelse som
|
||||
starter med `manual:`, og sjekkdatoen er registerets `checked`.
|
||||
|
||||
**Kjørt live 01.11** med vakten ved `45d3459`, med `--used` og en egen last-run-fil
|
||||
i scratch. llm-securitys stempel hadde samme sha256 før og etter.
|
||||
|
||||
| Liste | behind | watch | notVerified | current | Prober ok | exit |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `watch-sources.json` | 0 | 0 | 0 | 16 | 12 | 0 |
|
||||
| `watch-sources-v1.4.1.json` (frosset) | 1: owasp-llm-top10 (2025 → 2026) | 1: owasp-rag-cheat-sheet (41f67299 → 510ea484) | 0 | 12 | 10 | 1 |
|
||||
|
||||
Utdataene er committet som `watch-last.json` og `watch-v1.4.1-run.json`.
|
||||
|
||||
**Den frosne v1.4.1-kopien** inneholder bare det v1.4.1 (tagget 2026-09-23) siterte,
|
||||
og hver oppføring har utgaven utgiveren serverte den dagen. OWASP Agentic og ATLAS er
|
||||
utelatt fordi v1.4.1 ikke siterte dem. RAG-arket sto på `41f67299` (2026-05-06), lest
|
||||
fra GitHubs commit-API med `until=2026-09-23`. Uversjonerte arXiv-lenker gir de samme
|
||||
versjonene som nå, fordi alle ble publisert før tag-dagen. Fila er låst med sha256 i
|
||||
testen.
|
||||
|
||||
**Funn:** vakten melder Unicode 17.0 som `current`, fordi klasse S ikke spør. Det er
|
||||
bare 30-dagersregelen som fanger den. Positiv kontroll med alle seks problemene: 4
|
||||
manuelle sjekker fra 23.09, 1 behind og 1 watch.
|
||||
|
||||
**Release-regelen** (`tests/test_release_rule.py`) leser releasedatoen fra den
|
||||
nyeste CHANGELOG-overskriften. Den binder derfor på release-commiten og ikke på en
|
||||
vilkårlig dag mellom to releaser. En test holder overskriften lik versjonen i
|
||||
`pyproject.toml`. Regelen stopper når:
|
||||
- en manuell sjekk er mer enn 30 dager eldre enn releasedatoen;
|
||||
- vaktens siste kjøring er mer enn 30 dager eldre (den er sjekkdatoen for de 12);
|
||||
- utdataene ikke er en kjøring av denne lista;
|
||||
- det finnes en rad i `behind`, `notVerified` eller `watch`. En `watch`-rad teller
|
||||
aldri med i vaktens exit-kode, så regelen leser den for seg. Den ryddes av en
|
||||
sjekk for hånd som oppdaterer `known_value`, og deretter en ny kjøring.
|
||||
|
||||
Hver klausul er mutert bort én om gangen, og hver mutasjon gjorde en test rød (5 av 5).
|
||||
|
||||
**Før en release:** kjør
|
||||
`LLM_SECURITY_EDITION_WATCH_LAST_RUN=<scratch>/lr.json node <llm-security>/scripts/edition-watch.mjs --used docs/threat-2026/watch-sources.json > docs/threat-2026/watch-last.json`
|
||||
og commit utdataene. Uten den miljøvariabelen stempler kjøringen llm-securitys egen
|
||||
last-run-fil.
|
||||
|
||||
**Ukentlig kjøring:** foreslått til llm-security via coord 01.11. Det er deres jobb
|
||||
(`run.sh` i deres tilstandsmappe, launchd mandager 09:15), så den endres ikke herfra.
|
||||
Rad 1 er ikke framme før jobben har kjørt lista vår de siste 7 dagene.
|
||||
|
||||
**Sett, ikke endret:** commons-proben (N) svarer `ok` under llm-securitys jobb, som
|
||||
setter en ssh-remote. Deres `last.json` fra 26.10 har 0 i `notVerified`. Commons blir
|
||||
likevel stående manuell her. llm-securitys egen liste følger de samme commons-taggene
|
||||
hver uke. Under en annen kjøring uten ssh-remote ville proben her svare
|
||||
`auth-required`.
|
||||
|
||||
Ingen release, fordi ingenting i den utgitte pakken er endret.
|
||||
|
||||
## Forrige retning (godkjent av operatøren 28.10.2026, nådd 31.10)
|
||||
|
||||
> «llm-ingestion-guard bygger på nyeste utgave av hver trusselkatalog den viser til,
|
||||
|
|
|
|||
25
docs/threat-2026/watch-last.json
Normal file
25
docs/threat-2026/watch-last.json
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
{
|
||||
"behind": [],
|
||||
"notVerified": [],
|
||||
"watch": [],
|
||||
"current": [
|
||||
"owasp-llm-top10",
|
||||
"owasp-agentic-top10",
|
||||
"mitre-atlas",
|
||||
"owasp-rag-cheat-sheet",
|
||||
"unicode",
|
||||
"cve-2025-32711",
|
||||
"arxiv-2402.07867",
|
||||
"arxiv-2504.11168",
|
||||
"arxiv-2509.14285",
|
||||
"arxiv-2505.03574",
|
||||
"arxiv-2503.18813",
|
||||
"arxiv-2604.00387",
|
||||
"arxiv-2605.00460",
|
||||
"willison-dual-llm",
|
||||
"rfc-3986",
|
||||
"llm-security-commons"
|
||||
],
|
||||
"lastRun": "2026-11-01T11:45:11.499Z",
|
||||
"weeklyJobLastRun": "never"
|
||||
}
|
||||
290
docs/threat-2026/watch-sources-v1.4.1.json
Normal file
290
docs/threat-2026/watch-sources-v1.4.1.json
Normal file
|
|
@ -0,0 +1,290 @@
|
|||
[
|
||||
{
|
||||
"id": "owasp-llm-top10",
|
||||
"name": "OWASP Top 10 for LLM Applications",
|
||||
"kind": "catalog",
|
||||
"location": "Finding.owasp on every finding; README, LIMITATIONS, coverage matrix",
|
||||
"used_edition": "2025",
|
||||
"newest_edition": "2026",
|
||||
"newest_source_url": "https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "A",
|
||||
"signal": "edition-year",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": "2025",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "owasp-rag-cheat-sheet",
|
||||
"name": "Retrieval-Augmented Generation (RAG) Security Cheat Sheet",
|
||||
"kind": "guidance",
|
||||
"location": "README \u00a7 Design & threat model (linked, with the check date)",
|
||||
"used_edition": "living document, commit 41f67299 (2026-05-06), the newest on 2026-09-23",
|
||||
"newest_edition": "living document, last modified 2026-10-04 (commit 510ea484)",
|
||||
"newest_source_url": "https://cheatsheetseries.owasp.org/cheatsheets/RAG_Security_Cheat_Sheet.html",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "L",
|
||||
"signal": "watch-only",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"url": "https://api.github.com/repos/OWASP/CheatSheetSeries/commits?path=cheatsheets/RAG_Security_Cheat_Sheet.md&per_page=1",
|
||||
"parse": "0.sha",
|
||||
"known_value": "41f67299945eeb85cc4387e073bea3d8343ad92d",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "unicode",
|
||||
"name": "The Unicode Standard (UCD)",
|
||||
"kind": "standard",
|
||||
"location": "invisible-character tables (sanitize, output); the character denominator in docs/threat-2026/codepoints.json",
|
||||
"used_edition": "17.0",
|
||||
"newest_edition": "18.0.0",
|
||||
"newest_source_url": "https://www.unicode.org/versions/Unicode18.0.0/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: unicode.org is not on the watch's host list and the UCD ReadMe is text, not JSON; unicodetools on GitHub has final-17.0 as its newest release while 18.0 is current, so no probe class can read the edition",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cve-2025-32711",
|
||||
"name": "CVE-2025-32711 (EchoLeak), M365 Copilot Information Disclosure",
|
||||
"kind": "advisory",
|
||||
"location": "active-content detector docs, coverage matrix, README",
|
||||
"used_edition": "CVE-2025-32711",
|
||||
"newest_edition": "MSRC revision 2026-02-20; NVD last modified 2026-06-17",
|
||||
"newest_source_url": "https://nvd.nist.gov/vuln/detail/CVE-2025-32711",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "L",
|
||||
"signal": "watch-only",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"url": "https://raw.githubusercontent.com/CVEProject/cvelistV5/main/cves/2025/32xxx/CVE-2025-32711.json",
|
||||
"parse": "cveMetadata.dateUpdated",
|
||||
"known_value": "2026-02-26T17:50:41.404Z",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2402.07867",
|
||||
"name": "PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models",
|
||||
"kind": "paper",
|
||||
"location": "README \u00a7 Design & threat model (linked); semantic-poisoning gap",
|
||||
"used_edition": "2402.07867 (no version)",
|
||||
"newest_edition": "v3",
|
||||
"newest_source_url": "https://arxiv.org/abs/2402.07867",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2402.07867v3"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2504.11168",
|
||||
"name": "Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems",
|
||||
"kind": "paper",
|
||||
"location": "README and BRIEF, detection-is-evadable claim",
|
||||
"used_edition": "2504.11168 (no version)",
|
||||
"newest_edition": "v3",
|
||||
"newest_source_url": "https://arxiv.org/abs/2504.11168",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2504.11168v3"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2509.14285",
|
||||
"name": "A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks",
|
||||
"kind": "paper",
|
||||
"location": "docs/BRIEF.md",
|
||||
"used_edition": "2509.14285 (no version)",
|
||||
"newest_edition": "v4",
|
||||
"newest_source_url": "https://arxiv.org/abs/2509.14285",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2509.14285v4"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2505.03574",
|
||||
"name": "LlamaFirewall: An open source guardrail system for building secure AI agents",
|
||||
"kind": "paper",
|
||||
"location": "docs/BRIEF.md",
|
||||
"used_edition": "2505.03574 (no version)",
|
||||
"newest_edition": "v1",
|
||||
"newest_source_url": "https://arxiv.org/abs/2505.03574",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2505.03574v1"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2503.18813",
|
||||
"name": "Defeating Prompt Injections by Design (CaMeL)",
|
||||
"kind": "paper",
|
||||
"location": "architecture lineage",
|
||||
"used_edition": "2503.18813 (no version)",
|
||||
"newest_edition": "v2",
|
||||
"newest_source_url": "https://arxiv.org/abs/2503.18813",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2503.18813v2"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2604.00387",
|
||||
"name": "RAGShield: Detecting Numerical Claim Manipulation in Government RAG Systems",
|
||||
"kind": "paper",
|
||||
"location": "docs/PLAN.md v1 anchors (history)",
|
||||
"used_edition": "2604.00387 (no version), labelled 'RAGShield'",
|
||||
"newest_edition": "v2",
|
||||
"newest_source_url": "https://arxiv.org/abs/2604.00387",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2604.00387v2"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2605.00460",
|
||||
"name": "CleanBase: Detecting Malicious Documents in RAG Knowledge Databases",
|
||||
"kind": "paper",
|
||||
"location": "docs/BRIEF.md \u00a7 11, write-time RAG-DB poisoning is under-tooled",
|
||||
"used_edition": "2605.00460 (no version, pdf link)",
|
||||
"newest_edition": "v1",
|
||||
"newest_source_url": "https://arxiv.org/abs/2605.00460",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2605.00460v1"
|
||||
],
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "willison-dual-llm",
|
||||
"name": "The Dual LLM pattern for building AI assistants that can resist prompt injection",
|
||||
"kind": "article",
|
||||
"location": "architecture lineage",
|
||||
"used_edition": "by name, 2023",
|
||||
"newest_edition": "2023-04-25 (single post)",
|
||||
"newest_source_url": "https://simonwillison.net/2023/Apr/25/dual-llm-pattern/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: a single dated blog post with no editions; there is nothing for a probe to follow",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "rfc-3986",
|
||||
"name": "RFC 3986, Uniform Resource Identifier (URI): Generic Syntax (STD 66)",
|
||||
"kind": "standard",
|
||||
"location": "active_content.py scheme grammar",
|
||||
"used_edition": "RFC 3986",
|
||||
"newest_edition": "RFC 3986, INTERNET STANDARD, not obsoleted (updated by RFC 7320, RFC 8820)",
|
||||
"newest_source_url": "https://www.rfc-editor.org/rfc/rfc3986.json",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: rfc-editor.org is not on the watch's host list; its rfc3986.json (obsoleted_by empty) would serve a class L probe once the host is added",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "llm-security-commons",
|
||||
"name": "llm-security-commons (injection lexicon, carrier code points)",
|
||||
"kind": "data-source",
|
||||
"location": "src/llm_ingestion_guard/injection_lexicon.json",
|
||||
"used_edition": "lexicon 0.8.0 (83 patterns, byte-equal copy)",
|
||||
"newest_edition": "v0.9.0 (lexicon/injection-lexicon.json 0.8.0, codepoints/carriers.json 0.2.0)",
|
||||
"newest_source_url": "ssh://git@git.fromaitochitta.com/ktg/llm-security-commons.git",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: the repository is private, so the class N probe answers auth-required without credentials; llm-security's own source list watches the same commons tags under its weekly job",
|
||||
"known_at": "2026-09-23"
|
||||
}
|
||||
}
|
||||
]
|
||||
329
docs/threat-2026/watch-sources.json
Normal file
329
docs/threat-2026/watch-sources.json
Normal file
|
|
@ -0,0 +1,329 @@
|
|||
[
|
||||
{
|
||||
"id": "owasp-llm-top10",
|
||||
"name": "OWASP Top 10 for LLM Applications",
|
||||
"kind": "catalog",
|
||||
"location": "Finding.owasp on every finding; README, LIMITATIONS, coverage matrix",
|
||||
"used_edition": "2026",
|
||||
"newest_edition": "2026",
|
||||
"newest_source_url": "https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "A",
|
||||
"signal": "edition-year",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": "2026",
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "owasp-agentic-top10",
|
||||
"name": "OWASP Top 10 for Agentic Applications",
|
||||
"kind": "catalog",
|
||||
"location": "README \u00a7 What it protects against (ASI06)",
|
||||
"used_edition": "for 2026",
|
||||
"newest_edition": "for 2026",
|
||||
"newest_source_url": "https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "B",
|
||||
"signal": "edition-year",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": "2026",
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "mitre-atlas",
|
||||
"name": "MITRE ATLAS",
|
||||
"kind": "catalog",
|
||||
"location": "README \u00a7 What it protects against",
|
||||
"used_edition": "v2026.10",
|
||||
"newest_edition": "v2026.10",
|
||||
"newest_source_url": "https://github.com/mitre-atlas/atlas-data/releases/tag/v2026.10",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "F",
|
||||
"signal": "stable-release",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"url": "https://api.github.com/repos/mitre-atlas/atlas-data/releases/latest",
|
||||
"known_value": "v2026.10",
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "owasp-rag-cheat-sheet",
|
||||
"name": "Retrieval-Augmented Generation (RAG) Security Cheat Sheet",
|
||||
"kind": "guidance",
|
||||
"location": "README \u00a7 Design & threat model (linked, with the check date)",
|
||||
"used_edition": "living document, last modified 2026-10-04 (commit 510ea484)",
|
||||
"newest_edition": "living document, last modified 2026-10-04 (commit 510ea484)",
|
||||
"newest_source_url": "https://cheatsheetseries.owasp.org/cheatsheets/RAG_Security_Cheat_Sheet.html",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "L",
|
||||
"signal": "watch-only",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"url": "https://api.github.com/repos/OWASP/CheatSheetSeries/commits?path=cheatsheets/RAG_Security_Cheat_Sheet.md&per_page=1",
|
||||
"parse": "0.sha",
|
||||
"known_value": "510ea484cfc304ec2722289237e936bc372eee34",
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "unicode",
|
||||
"name": "The Unicode Standard (UCD)",
|
||||
"kind": "standard",
|
||||
"location": "invisible-character tables (sanitize, output); the character denominator in docs/threat-2026/codepoints.json",
|
||||
"used_edition": "18.0.0",
|
||||
"newest_edition": "18.0.0",
|
||||
"newest_source_url": "https://www.unicode.org/versions/Unicode18.0.0/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: unicode.org is not on the watch's host list and the UCD ReadMe is text, not JSON; unicodetools on GitHub has final-17.0 as its newest release while 18.0 is current, so no probe class can read the edition",
|
||||
"known_at": "2026-10-31"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cve-2025-32711",
|
||||
"name": "CVE-2025-32711 (EchoLeak), M365 Copilot Information Disclosure",
|
||||
"kind": "advisory",
|
||||
"location": "active-content detector docs, coverage matrix, README",
|
||||
"used_edition": "MSRC revision 2026-02-20; NVD last modified 2026-06-17",
|
||||
"newest_edition": "MSRC revision 2026-02-20; NVD last modified 2026-06-17",
|
||||
"newest_source_url": "https://nvd.nist.gov/vuln/detail/CVE-2025-32711",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "L",
|
||||
"signal": "watch-only",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"url": "https://raw.githubusercontent.com/CVEProject/cvelistV5/main/cves/2025/32xxx/CVE-2025-32711.json",
|
||||
"parse": "cveMetadata.dateUpdated",
|
||||
"known_value": "2026-02-26T17:50:41.404Z",
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2402.07867",
|
||||
"name": "PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models",
|
||||
"kind": "paper",
|
||||
"location": "README \u00a7 Design & threat model (linked); semantic-poisoning gap",
|
||||
"used_edition": "v3",
|
||||
"newest_edition": "v3",
|
||||
"newest_source_url": "https://arxiv.org/abs/2402.07867",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2402.07867v3"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2504.11168",
|
||||
"name": "Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems",
|
||||
"kind": "paper",
|
||||
"location": "README and BRIEF, detection-is-evadable claim",
|
||||
"used_edition": "v3",
|
||||
"newest_edition": "v3",
|
||||
"newest_source_url": "https://arxiv.org/abs/2504.11168",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2504.11168v3"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2509.14285",
|
||||
"name": "A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks",
|
||||
"kind": "paper",
|
||||
"location": "docs/BRIEF.md",
|
||||
"used_edition": "v4",
|
||||
"newest_edition": "v4",
|
||||
"newest_source_url": "https://arxiv.org/abs/2509.14285",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2509.14285v4"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2505.03574",
|
||||
"name": "LlamaFirewall: An open source guardrail system for building secure AI agents",
|
||||
"kind": "paper",
|
||||
"location": "docs/BRIEF.md",
|
||||
"used_edition": "v1",
|
||||
"newest_edition": "v1",
|
||||
"newest_source_url": "https://arxiv.org/abs/2505.03574",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2505.03574v1"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2503.18813",
|
||||
"name": "Defeating Prompt Injections by Design (CaMeL)",
|
||||
"kind": "paper",
|
||||
"location": "architecture lineage",
|
||||
"used_edition": "v2",
|
||||
"newest_edition": "v2",
|
||||
"newest_source_url": "https://arxiv.org/abs/2503.18813",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2503.18813v2"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2604.00387",
|
||||
"name": "RAGShield: Detecting Numerical Claim Manipulation in Government RAG Systems",
|
||||
"kind": "paper",
|
||||
"location": "docs/PLAN.md v1 anchors (history)",
|
||||
"used_edition": "v2",
|
||||
"newest_edition": "v2",
|
||||
"newest_source_url": "https://arxiv.org/abs/2604.00387",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2604.00387v2"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "arxiv-2605.00460",
|
||||
"name": "CleanBase: Detecting Malicious Documents in RAG Knowledge Databases",
|
||||
"kind": "paper",
|
||||
"location": "docs/BRIEF.md \u00a7 11, write-time RAG-DB poisoning is under-tooled",
|
||||
"used_edition": "v1",
|
||||
"newest_edition": "v1",
|
||||
"newest_source_url": "https://arxiv.org/abs/2605.00460",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "O",
|
||||
"signal": "version-field",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": [
|
||||
"2605.00460v1"
|
||||
],
|
||||
"known_at": "2026-11-01"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "willison-dual-llm",
|
||||
"name": "The Dual LLM pattern for building AI assistants that can resist prompt injection",
|
||||
"kind": "article",
|
||||
"location": "architecture lineage",
|
||||
"used_edition": "2023-04-25 (single post)",
|
||||
"newest_edition": "2023-04-25 (single post)",
|
||||
"newest_source_url": "https://simonwillison.net/2023/Apr/25/dual-llm-pattern/",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: a single dated blog post with no editions; there is nothing for a probe to follow",
|
||||
"known_at": "2026-10-31"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "rfc-3986",
|
||||
"name": "RFC 3986, Uniform Resource Identifier (URI): Generic Syntax (STD 66)",
|
||||
"kind": "standard",
|
||||
"location": "active_content.py scheme grammar",
|
||||
"used_edition": "RFC 3986, INTERNET STANDARD, not obsoleted (updated by RFC 7320, RFC 8820)",
|
||||
"newest_edition": "RFC 3986, INTERNET STANDARD, not obsoleted (updated by RFC 7320, RFC 8820)",
|
||||
"newest_source_url": "https://www.rfc-editor.org/rfc/rfc3986.json",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: rfc-editor.org is not on the watch's host list; its rfc3986.json (obsoleted_by empty) would serve a class L probe once the host is added",
|
||||
"known_at": "2026-10-31"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "llm-security-commons",
|
||||
"name": "llm-security-commons (injection lexicon, carrier code points)",
|
||||
"kind": "data-source",
|
||||
"location": "src/llm_ingestion_guard/injection_lexicon.json",
|
||||
"used_edition": "v0.9.0 (lexicon/injection-lexicon.json 0.8.0, codepoints/carriers.json 0.2.0)",
|
||||
"newest_edition": "v0.9.0 (lexicon/injection-lexicon.json 0.8.0, codepoints/carriers.json 0.2.0)",
|
||||
"newest_source_url": "ssh://git@git.fromaitochitta.com/ktg/llm-security-commons.git",
|
||||
"checked": "2026-10-31",
|
||||
"probe": {
|
||||
"class": "S",
|
||||
"signal": "none",
|
||||
"timeout_ms": 20000,
|
||||
"max_bytes": 1048576,
|
||||
"cadence_days": 7,
|
||||
"known_value": null,
|
||||
"reason": "manual: the repository is private, so the class N probe answers auth-required without credentials; llm-security's own source list watches the same commons tags under its weekly job",
|
||||
"known_at": "2026-10-31"
|
||||
}
|
||||
}
|
||||
]
|
||||
36
docs/threat-2026/watch-v1.4.1-run.json
Normal file
36
docs/threat-2026/watch-v1.4.1-run.json
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
{
|
||||
"behind": [
|
||||
{
|
||||
"id": "owasp-llm-top10",
|
||||
"source": "OWASP Top 10 for LLM Applications",
|
||||
"used": "2025",
|
||||
"newest": "2026",
|
||||
"url": "https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/",
|
||||
"probe_status": "ok"
|
||||
}
|
||||
],
|
||||
"notVerified": [],
|
||||
"watch": [
|
||||
{
|
||||
"id": "owasp-rag-cheat-sheet",
|
||||
"signal": "watch-only",
|
||||
"observed": "510ea484cfc304ec2722289237e936bc372eee34"
|
||||
}
|
||||
],
|
||||
"current": [
|
||||
"unicode",
|
||||
"cve-2025-32711",
|
||||
"arxiv-2402.07867",
|
||||
"arxiv-2504.11168",
|
||||
"arxiv-2509.14285",
|
||||
"arxiv-2505.03574",
|
||||
"arxiv-2503.18813",
|
||||
"arxiv-2604.00387",
|
||||
"arxiv-2605.00460",
|
||||
"willison-dual-llm",
|
||||
"rfc-3986",
|
||||
"llm-security-commons"
|
||||
],
|
||||
"lastRun": "2026-11-01T11:47:10.999Z",
|
||||
"weeklyJobLastRun": "never"
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue