feat(ms-ai-architect): R7.5 extract-fase komplett (51/51, 986 claims) + dømmebølge 1 delvis — 8/10 ingestet (1 pass, 7 flagged, 34 flagg; ledger 192→200) [skip-docs]
This commit is contained in:
parent
f52a486e3a
commit
845d8fbdc9
1 changed files with 496 additions and 1 deletions
|
|
@ -4,7 +4,7 @@
|
|||
"derived_from": "scripts/kb-update/data/full-pass-worklist.json (243 due, all never-verified)",
|
||||
"cadence": "R7–R10 (5 økter × ~49, 8–10 samtidige)",
|
||||
"batch": "R7.1",
|
||||
"count": 192,
|
||||
"count": 200,
|
||||
"generated": "2026-07-18"
|
||||
},
|
||||
"files": [
|
||||
|
|
@ -10623,6 +10623,87 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-infrastructure/references/hybrid-edge/disconnected-ai-scenarios.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "pass",
|
||||
"claim_count": 19,
|
||||
"verified": "2026-07-31",
|
||||
"verified_by": "judge-v3.1",
|
||||
"flags": []
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/adversarial-input-robustness-testing.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 20,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/adversarial-input-robustness-testing.md#2",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "1 | Perturbation attack ... 2 | Poisoning attack ... 3 | Model Inversion ... 4 | Membership Inference ... 5 | Model Stealing ... 6 | Reprogramming ML system ... 7 | Adversarial Example in Physical Domain ... 8 | Malicious ML provider recovering training data ... 9 | Attacking the ML supply chain ... 10 | Backdoor ML ... 11 | Exploit Software Dependencies",
|
||||
"reason": "Jeg hentet den kanoniske taksonomi-siden (og søsteren failure-modes-in-machine-learning som enumererer alle 11 intensjonelle feilmodi): ingen kategori «innholdsbaserte angrep» og verken prompt injection, jailbreaking eller XPIA finnes i taksonomien — fraværet i den enumererende siden er bevis mot eksistenspåstanden (R2), og fritekstsøk fant heller ingen Learn-side som plasserer disse tre i denne taksonomien.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/adversarial-input-robustness-testing.md",
|
||||
"line": 38,
|
||||
"claim": "Innholdsbaserte angrep i taksonomien består av: Prompt injection | Jailbreaking | Indirect prompt injection (XPIA).",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/adversarial-input-robustness-testing.md#14",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "",
|
||||
"evidence_quote": "",
|
||||
"reason": "Verken den siterte siden eller noe annet learn.microsoft.com-treff dokumenterer PyRITs interne arkitekturkomponenter (Orchestrator/Target/Scorers/Attack Strategy/Memory) — disse er kun beskrevet i PyRIT-repoet på GitHub, som ligger utenfor Learn-flaten, så påstanden kan verken bekreftes eller avkreftes her.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/adversarial-input-robustness-testing.md",
|
||||
"line": 134,
|
||||
"claim": "PyRITs arkitektur består av komponentene: Orchestrator | Target | Scorers | Attack Strategy | Memory.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/adversarial-input-robustness-testing.md#15",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/python/api/azure-ai-projects/azure.ai.projects.models.attackstrategy?view=azure-python",
|
||||
"evidence_quote": "AttackStrategy Enum. Strategies for attacks. Fields: ANSI_ATTACK | ASCII_ART | ASCII_SMUGGLER | ATBASH | BASE64 | BASELINE | BINARY | CAESAR | CHARACTER_SPACE | CHARACTER_SWAP | CRESCENDO | DIACRITIC | DIFFICULT | EASY | FLIP | INDIRECT_JAILBREAK | JAILBREAK | LEETSPEAK | MODERATE | MORSE | MULTI_TURN | ROT13 | STRING_JOIN | SUFFIX_APPEND | TENSE | UNICODE_CONFUSABLE | UNICODE_SUBSTITUTION | URL",
|
||||
"reason": "I en PyRIT-basert skanning (AI Red Teaming Agent) velges AttackStrategy — der finnes JAILBREAK (UPIA) og INDIRECT_JAILBREAK (XPIA), men ADVERSARIAL_QA finnes ikke; ADVERSARIAL_QA hører til AdversarialScenario-enumet for AdversarialSimulator, som ikke er PyRIT-basert, så den ene lastbærende delen er feil (R8).",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/adversarial-input-robustness-testing.md",
|
||||
"line": 148,
|
||||
"claim": "Attack-scenarioene som kan velges i en PyRIT-basert skanning inkluderer ADVERSARIAL_QA | UPIA | XPIA.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/adversarial-input-robustness-testing.md#17",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent",
|
||||
"evidence_quote": "Prohibited Actions | Universally banned (for example, facial recognition, emotion inference, social scoring). | ❌ Never allowed ... High-Risk Actions | Sensitive actions need explicit human authorization (for example, financial transactions, medical decisions). | ⚠️ Allowed with human-in-the-loop confirmation",
|
||||
"reason": "Delene om sensitive data leakage (format-nivå lekkasje via pattern matching), jailbreak og task adherence stemmer, men den lastbærende delen «Prohibited Actions = forbudt handling uten human-in-the-loop» motsies: siden sier ASR for prohibited actions er «policy violations exhibited by the agent based on the user-provided policies», og forbudte handlinger er «Never allowed» — human-in-the-loop-forbeholdet gjelder High-Risk Actions, ikke Prohibited Actions (R8).",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/adversarial-input-robustness-testing.md",
|
||||
"line": 261,
|
||||
"claim": "Attack Success Rate defineres per risikokategori slik: Hateful/Sexual/Violent Content = modell genererer harmful content over severity-terskel | Jailbreak = safety guardrails omgås | Prohibited Actions = agent utfører forbudt handling uten human-in-the-loop | Sensitive Data Leakage = format-nivå lekkasje detektert via pattern matching | Task Adherence = agent feiler i goal/rule/procedure compliance.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/adversarial-input-robustness-testing.md#18",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "Remote Elevation of Privilege since attacker is now in control of your model",
|
||||
"reason": "Den siterte failure-modes-siden inneholder ingen alvorlighetsskala i det hele tatt; på Microsofts faktiske severity-kilde (threat-modeling-aiml) er Targeted misclassification rangert Critical (ikke Important), og både Random misclassification og Confidence Reduction er Important (ikke Moderate) — flere lastbærende deler av inndelingen er feil (R8).",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/adversarial-input-robustness-testing.md",
|
||||
"line": 268,
|
||||
"claim": "Alvorlighetsnivåene er: Critical (remote EOP, modellkontroll, dataeksfiltrering) | Important (targeted misclassification, model stealing, personvernlekkasjer) | Moderate (random misclassification, confidence reduction).",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-prompt-shield-network.md",
|
||||
"batch": "R7.1",
|
||||
|
|
@ -10706,6 +10787,373 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-red-team-operations-practical.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 21,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-red-team-operations-practical.md#12",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/foundry/concepts/ai-red-teaming-agent",
|
||||
"evidence_quote": "Cloud red teaming is currently available in the following regions: East US 2, France Central, Sweden Central, Switzerland West, and US North Central.",
|
||||
"reason": "Kravet stenger listen med «kun» på fire regioner, mens live-siden lister fem — US North Central mangler i kravet, så den uttømmende regionpåstanden er superseded (how-to-siden lister samme fem).",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-red-team-operations-practical.md",
|
||||
"line": 195,
|
||||
"claim": "AI Red Teaming Agent er kun tilgjengelig i regionene East US2 | Sweden Central | France Central | Switzerland West.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-red-team-operations-practical.md#14",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen learn.microsoft.com-side beskriver PyRITs interne nøkkelkonsepter Prompt Targets / Attack Strategies / Scorers — Learn omtaler kun PyRIT utenfra og lenker til GitHub- og microsoft.github.io-dokumentasjonen, som faller utenfor verifiseringsflaten.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-red-team-operations-practical.md",
|
||||
"line": 216,
|
||||
"claim": "PyRITs nøkkelkonsepter er: Prompt Targets (systemet du tester) | Attack Strategies (conversion methods) | Scorers (evaluering av om angrepet lyktes).",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-red-team-operations-practical.md#16",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/foundry/concepts/ai-red-teaming-agent",
|
||||
"evidence_quote": "For red teaming agentic risk categories, we ensure that when an AI red teaming run targets a Foundry hosted agent, it's a transient run so that harmful data isn't logged by the Foundry Agent Service and chat completions aren't stored.",
|
||||
"reason": "Kilden sier at kjøringen gjøres transient FOR AT chat completions ikke skal lagres; kravet snur årsaksretningen og oppgir manglende lagring i Foundry Agent Service som grunnen til transiente agenter — den begrunnende delen er bærende og motsies av kilden.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-red-team-operations-practical.md",
|
||||
"line": 353,
|
||||
"claim": "Transient agents brukes ved red teaming fordi chat completions ikke lagres i Foundry Agent Service.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-red-team-operations-practical.md#18",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen gjeldende Learn-side beskriver en «AI red teaming»-fane i Foundrys Evaluation-side med de to Report view-ene og Data-siden; «View evaluation results in the Microsoft Foundry portal» omtaler ikke red teaming i det hele tatt, så påstanden kan verken bekreftes eller motbevises (kun «human feedback»-kolonnen er dokumentert, i transparency note-en).",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-red-team-operations-practical.md",
|
||||
"line": 388,
|
||||
"claim": "Resultater vises i Foundry på Evaluation-siden under fanen AI red teaming, med Report view per risikokategori | Report view per attack complexity | Data-side med attack-response-par (full samtalehistorikk, brukt attack strategy, success/failure-status, human feedback).",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-red-team-operations-practical.md#20",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen Learn-side fastslår ansvarsdelingen kravet beskriver for M365 Copilot (at Microsoft red teamer plattformen mens kundene tester custom plugins og declarative agents); nærmeste dekning er generelt delt ansvar for egenutviklede plugins/connectors, som ikke omtaler red teaming.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-red-team-operations-practical.md",
|
||||
"line": 636,
|
||||
"claim": "For M365 Copilot red teamer Microsoft selv plattformen, mens kundene tester custom plugins og declarative agents.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-security-scoring-framework.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 19,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-security-scoring-framework.md#9",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/ai-services/policy-reference",
|
||||
"evidence_quote": "Azure AI Services resources should use Azure Private Link ... Diagnostic logs in Azure AI services resources should be enabled",
|
||||
"reason": "Den kanoniske katalogen over innebygde policyer har ingen policy kalt Azure AI services should use private endpoints (den heter Azure AI Services resources should use Azure Private Link), og loggpolicyen heter Diagnostic logs in Azure AI services **resources** should be enabled; bare ML-policynavnet matcher ordrett, så to av tre bærende deler er feil.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-security-scoring-framework.md",
|
||||
"line": 260,
|
||||
"claim": "Innebygde Azure Policy-policyer for AI-styring er: «Azure AI services should use private endpoints» | «Azure Machine Learning workspaces should disable public network access» | «Diagnostic logs in Azure AI services should be enabled».",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-security-scoring-framework.md#12",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/purview/purview-billing-models",
|
||||
"evidence_quote": "Microsoft Purview supports two complementary billing models to support this diverse environment: 1. Per-user license for Microsoft 365 and Windows/macOS endpoint sources 2. Pay-as-you-go model for non-Microsoft 365 data sources and certain other capabilities",
|
||||
"reason": "Den kanoniske faktureringssiden lister to modeller — per bruker og pay-as-you-go med forbruksenheter (aktiva/dag, forespørsler, GB, DGPU) — ingen lisensiering per datakilde finnes i oppregningen.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-security-scoring-framework.md",
|
||||
"line": 366,
|
||||
"claim": "Microsoft Purview Compliance lisensieres per datakilde.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-security-scoring-framework.md#13",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R3",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/concepts/observability",
|
||||
"evidence_quote": "Observability features such as risk and safety evaluations and evaluations in the agent playground are billed based on consumption as listed in our Azure pricing page (foundryobservability).",
|
||||
"reason": "Risk- og sikkerhetsevalueringer faktureres som forbruk under Foundry Observability, og kjører mot Microsofts hostede sikkerhetsmodeller (Risk and safety evaluators use the hosted Foundry Evaluation service and don't require a deployment_name), ikke inkludert i Azure OpenAI med token-basert fakturering.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-security-scoring-framework.md",
|
||||
"line": 367,
|
||||
"claim": "Safety evaluations for Azure OpenAI er inkludert i Azure OpenAI og faktureres token-basert.",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 19,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#2",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "#8 Malicious ML providers who can recover training data ... Traditional Parallels: Targeted information disclosure ... Severity: Important if data is PII, Moderate otherwise",
|
||||
"reason": "Kilden plasserer «Malicious ML Providers» under information disclosure — ikke Spoofing — og gir den alvorlighetsgrad Important/Moderate, mens Neural Net Reprogramming beskrives som «an abuse scenario»; to lastbærende deler (STRIDE-kategorien og alvorlighetsbåndet) motsies.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 37,
|
||||
"claim": "STRIDE-kategorien Spoofing dekker AI-truslene Neural Net Reprogramming | Malicious ML Providers, med alvorlighetsgrad Important-Critical.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#3",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "#2b Indiscriminate Data Poisoning ... Traditional Parallels: Authenticated Denial of service against a high-value asset ... Severity: Important",
|
||||
"reason": "Claimen sier begge poisoning-variantene er Critical og hører under Tampering, men kilden gir vilkårlig (indiscriminate) datapoisoning alvorlighetsgrad Important og parallellen Denial of service — én lastbærende del er direkte motsagt selv om targeted poisoning og Backdoored Models faktisk er Critical.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 38,
|
||||
"claim": "STRIDE-kategorien Tampering dekker AI-truslene Data Poisoning (målrettet/vilkårlig) | Backdoored Models, med alvorlighetsgrad Critical.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#4",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "#1 Adversarial Perturbation ... #2a Targeted Data Poisoning ... #2b Indiscriminate Data Poisoning ... #3 Model Inversion Attacks ... #4 Membership Inference Attack ... #5 Model Stealing ... #6 Neural Net Reprogramming ... #7 Adversarial Example in the Physical domain ... #8 Malicious ML providers who can recover training data ... #9 Attacking the ML Supply Chain ... #10 Backdoor Machine Learning ... #11 Exploit software dependencies of the ML system",
|
||||
"reason": "Jeg hentet den kanoniske trusseloppregningen (11 trusler) og verken «manipulasjon av modelloutput» eller «tap av lineage for treningsdata» finnes som trusler der, ingen Repudiation-kategori er definert, og ingen slik alvorlighetsgrad Moderate er oppgitt — fraværet i den enumererende siden er bevis mot eksistenspåstanden.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 39,
|
||||
"claim": "STRIDE-kategorien Repudiation dekker AI-truslene manipulasjon av modelloutput | tap av lineage for treningsdata, med alvorlighetsgrad Moderate.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#5",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "#4 Membership Inference Attack ... Traditional Parallels: Data Privacy. Inferences are being made about a data point's inclusion in the training set but the training data itself is not being disclosed ... Severity: This is a privacy issue, not a security issue.",
|
||||
"reason": "Model Inversion og Model Stealing knyttes riktignok til information disclosure i kilden, men Membership Inference plasseres eksplisitt under Data Privacy med presiseringen at treningsdata IKKE avsløres og uten sikkerhetsalvorlighet, og Model Stealing er «Important in security-sensitive models, Moderate otherwise» — båndet Important-Critical holder ikke for alle tre.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 40,
|
||||
"claim": "STRIDE-kategorien Information Disclosure dekker AI-truslene Model Inversion | Membership Inference | Model Stealing, med alvorlighetsgrad Important-Critical.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#7",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml",
|
||||
"evidence_quote": "#1 Adversarial Perturbation ... #6 Neural Net Reprogramming ... #7 Adversarial Example in the Physical domain (bits->atoms) ... #11 Exploit software dependencies of the ML system",
|
||||
"reason": "Adversarial Perturbation («Remote Elevation of Privilege», Critical) og Physical Domain Attacks («Elevation of Privilege, remote code execution», Critical) stemmer, men «Excessive Agency» finnes ikke i dokumentets kanoniske oppregning av de 11 truslene — det er et OWASP LLM-begrep, og fraværet i den enumererende siden gjør eksistenspåstanden feil.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 42,
|
||||
"claim": "STRIDE-kategorien Elevation of Privilege dekker AI-truslene Adversarial Perturbation | Excessive Agency | Physical Domain Attacks, med alvorlighetsgrad Critical.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#11",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/defender-for-cloud/ai-security-posture",
|
||||
"evidence_quote": "Defender for Cloud automatically and continuously discovers deployed AI workloads across the following services: Azure OpenAI Service, Azure AI Foundry, Azure Machine Learning, Amazon Bedrock, Google Vertex AI",
|
||||
"reason": "Den midterste delen («automate detection and remediation of generative AI risks») er korrekt, men den kanoniske AISPM-siden oppgir ikke Azure Resource Graph som oppdagelsesmekanisme og nevner ikke Purview-integrasjon i det hele tatt — CAF-siden lister Azure Resource Graph og Purview som SEPARATE verktøy («Use Azure Resource Graph to discover AI resources across subscriptions. Use Microsoft Defender for Cloud to identify generative AI workloads.»), så to lastbærende deler er feilattribuert til AISPM.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 211,
|
||||
"claim": "Microsoft Defender for Cloud AI Security Posture Management omfatter automatisk oppdagelse av AI-arbeidsbelastninger på tvers av Azure-abonnementer via Azure Resource Graph | automatisert deteksjon og utbedring av risiko i generativ AI | sikkerhetsanbefalinger for AI-modeller, datalagre og nettverksisolasjon | integrasjon med Purview for dataklassifisering, DLP og Insider Risk Management for prompt-basert dataeksfiltrering.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#12",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/security/develop/threat-modeling-tool-getting-started",
|
||||
"evidence_quote": "You must select which template to use before creating a model. Our main template is the Azure Threat Model Template, which contains Azure-specific stencils, threats and mitigations. For generic models, select the SDL TM Knowledge Base from the drop-down menu.",
|
||||
"reason": "Den kanoniske siden som enumererer malene i Threat Modeling Tool lister Azure Threat Model Template, SDL TM Knowledge Base og et medisinsk utstyr-stensilsett fra fellesskapet — ingen «ML Training Pipeline», «Model API» eller «LLM Agent»-mal finnes, så fraværet i den enumererende siden motbeviser eksistenspåstanden.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 227,
|
||||
"claim": "Microsoft Threat Modeling Tool har AI-spesifikke maler: ML Training Pipeline | Model API | LLM Agent.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#13",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/security/develop/threat-modeling-tool-getting-started",
|
||||
"evidence_quote": "Our main template is the Azure Threat Model Template, which contains Azure-specific stencils, threats and mitigations. For generic models, select the SDL TM Knowledge Base from the drop-down menu.",
|
||||
"reason": "STRIDE-automatisering («STRIDE per Element: Guided analysis of threats and mitigations») og trusselrapporter («Reports: Create HTML reports to share with others») er forankret, men den lastbærende delen «AI-spesifikke maler» finnes ikke i malopplistingen — én stated, lastbærende del er feil, så hele claimen faller.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 280,
|
||||
"claim": "Microsoft Threat Modeling Tool er gratis nedlasting (ingen lisenskostnad) og gir STRIDE-automatisering, AI-spesifikke maler og trusselrapporter.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#14",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/defender-for-cloud/ai-security-posture",
|
||||
"evidence_quote": "The Defender Cloud Security Posture Management (CSPM) plan in Microsoft Defender for Cloud secures enterprise-built, multicloud, or hybrid cloud environments.",
|
||||
"reason": "AI-kapabilitetene krever Defender CSPM-planen (posture/oppdagelse) og Defender for AI Services-planen (trusseldeteksjon) — ikke en «standard tier»; og faktureringen er ikke per server, men «Defender CSPM billing is based on specific resources» og for AI Services per skannede tokens («capped at 75 billion tokens scanned»).",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 281,
|
||||
"claim": "AI-kapabilitetene i Microsoft Defender for Cloud (oppdagelse av AI-arbeidsbelastninger, posture management, trusseldeteksjon) krever standard tier, lisensiert per server.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/ai-threat-modeling-stride.md#18",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/ai/playbook/technology-guidance/generative-ai/mlops-in-openai/security/security-plan-llm-application",
|
||||
"evidence_quote": "Security principles: Confidentiality refers to the objective of keeping data private or secret. ... Integrity is about ensuring that data has not been tampered with ... Availability means that networks, systems, and applications are up and running. ... Privacy relates to activities that focus on individual users' rights.",
|
||||
"reason": "Antallet stemmer (Threat #1 til #11) og mitigeringene refererer Azure OpenAI/Azure AI Content Safety, men dokumentet kartlegger hver trussel mot CIA-prinsippene pluss Privacy («Principle: Confidentiality, integrity, availability») — ikke mot STRIDE; den lastbærende rammepåstanden «kartlagt mot STRIDE» er erstattet av et annet rammeverk.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
|
||||
"line": 359,
|
||||
"claim": "Microsoft Learn-dokumentet «Security Planning for LLM-based Applications» beskriver 11 LLM-spesifikke trusler kartlagt mot STRIDE, med mitigeringsmønstre for Azure OpenAI.",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 18,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/content-safety-filter-calibration.md#5",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/custom-categories",
|
||||
"evidence_quote": "Another way to deal with emerging content incidents is to use Blocklists, but that option only allows exact text matching and no image matching.",
|
||||
"reason": "Severity threshold, Annotate only og No filters stemmer, men den lastbærende delen Blocklists (custom termlister for text og image) motsies direkte — blocklists gjør kun eksakt tekstmatching og har ingen bildematching.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"line": 48,
|
||||
"claim": "Konfigurerbare parametere for innholdsfilter er: Severity threshold (per kategori hate/sexual/violence/self-harm, separat for prompts og completions) | Annotate-only mode | Blocklists (custom termlister for text og image) | Custom categories (basert på RAI-policy, text og image) | No filters.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/content-safety-filter-calibration.md#10",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/foundry/openai/reference#chat-completions",
|
||||
"evidence_quote": "POST https://{endpoint}/openai/deployments/{deployment-id}/chat/completions?api-version=2024-10-21",
|
||||
"reason": "Stien stemmer, men den lastbærende api-version-strengen gjør det ikke: GA-versjonen er 2024-10-21, og API-livssyklussiden lister 2024-10-01-preview — ingen ren 2024-10-01 finnes, og en leser ville kopiert strengen rett inn i kode.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"line": 277,
|
||||
"claim": "Azure OpenAI chat completions-endepunktet kalles med api-version=2024-10-01 på stien /openai/deployments/<model>/chat/completions.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/content-safety-filter-calibration.md#14",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/connectors/cognitiveservicescontentmoderator/",
|
||||
"evidence_quote": "Content Moderator is a Microsoft Cognitive Services product which provides machine-assisted moderation of text and images for potentially offensive or unwanted content, augmented with built-in human review tools",
|
||||
"reason": "Jeg sjekket den kanoniske Power Platform-konnektorkatalogen: moderasjonskonnektoren som faktisk er tilgjengelig i Power Automate er Content Moderator (den utdaterte tjenesten), og ingen Content Safety-konnektor er oppført — kandidat-sidene /connectors/aicontentsafety, /connectors/contentsafety og /connectors/azureaicontentsafety gir alle 404.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"line": 342,
|
||||
"claim": "Det finnes en Content Safety-connector tilgjengelig i Power Automate.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/content-safety-filter-calibration.md#15",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview",
|
||||
"evidence_quote": "To use the Content Safety APIs, you must create your Azure AI Content Safety resource in a supported region. Currently, the Content Safety features are available in the following Azure regions with different API versions:",
|
||||
"reason": "Regionprinsippet er grunnet (FAQ: data forblir i regionen du valgte) og West Europe står i regionstabellen, men Norway East er ikke oppført blant de støttede Content Safety-regionene i det hele tatt — den lastbærende norske region-eksempeldelen er feil.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"line": 379,
|
||||
"claim": "Azure AI Content Safety prosesserer data i regionen ressursen opprettes i, for eksempel Norway East eller West Europe.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/content-safety-filter-calibration.md#16",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview",
|
||||
"evidence_quote": "",
|
||||
"reason": "Oversiktssiden oppgir kun prisnivåene F0 og S0 med RPS-grenser, og FAQ-en nevner en transaksjonsgrense på Free tier uten tall; ingen Learn-side oppgir 5000 transaksjoner per måned for Text eller Image API — tallet ligger på den JS-rendrede pris-siden.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"line": 419,
|
||||
"claim": "Både Text API og Image API i Azure AI Content Safety har en Free tier med 5000 transaksjoner per måned.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/content-safety-filter-calibration.md#18",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-application-card",
|
||||
"evidence_quote": "Classifiers such as those available in Azure AI Content Safety and metaprompting are two examples of mitigations that have been implemented in Microsoft 365 Copilot to help reduce the risk of these types of content.",
|
||||
"reason": "Første del (Azure OpenAI-deployments og Foundry-prosjekter) stemmer, men den lastbærende delen om M365 Copilot motsies direkte — M365 Copilot bruker nettopp Azure AI Content Safety-klassifiserere, og Copilot Studio håndhever innebygd innholdsmoderering på alle generative forespørsler uten at en separat Content Safety-ressurs kreves.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||
"line": 451,
|
||||
"claim": "Content Safety er inkludert i Azure OpenAI-deployments (default aktivert) og Microsoft Foundry-prosjekter, men IKKE i Microsoft 365 Copilot (bruker annen filtreringsstack) eller Copilot Studio (krever separat Content Safety-ressurs for custom filtering).",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/defender-threat-protection-ai-services.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 16,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/defender-threat-protection-ai-services.md#8",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/defender-for-cloud/alerts-ai-workloads",
|
||||
"evidence_quote": "## Alerts for AI applications ... ## Alerts for AI models ... ### (Preview) Malicious content detected in uploaded AI model (AI.AIModelScan_MalwareDetected) Description: ... Severity: High",
|
||||
"reason": "Den kanoniske varsellisten har kun to kategorier — AI applications og AI models — ingen AI agents-seksjon; dessuten mangler flere varsler (AI-modell-varselet og Exposed Kubernetes service detected) MITRE ATT&CK-taktikker, så begge de bærende delene svikter.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/defender-threat-protection-ai-services.md",
|
||||
"line": 35,
|
||||
"claim": "Varslene i AI threat protection er gruppert i tre kategorier: AI applications | AI agents | AI models, og hvert varsel er kartlagt til MITRE ATT&CK-taktikker.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/defender-threat-protection-ai-services.md#11",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/defender-for-cloud/alerts-ai-workloads",
|
||||
"evidence_quote": "## Alerts for AI applications ... ### (Preview) LLM Reconnaissance Attempt Detected (AI.Azure_LLMReconnaissance) ... ## Alerts for AI models ... ### (Preview) Malicious content detected in uploaded AI model",
|
||||
"reason": "Den kanoniske varsellisten enumererer ingen agent-varsler i det hele tatt, og verken instruction prompt leak eller agent reconnaissance attempt finnes der eller i søk på Microsoft Learn; fraværet i den autoritative listen er bevis mot eksistenspåstanden.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/defender-threat-protection-ai-services.md",
|
||||
"line": 51,
|
||||
"claim": "For AI agents (Foundry Agent Service) finnes tilsvarende varsler, alle i Preview, samt «instruction prompt leak» og «agent reconnaissance attempt».",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/entra-agent-id-zero-trust.md",
|
||||
"batch": "R7.1",
|
||||
|
|
@ -10777,6 +11225,53 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/jailbreak-prevention-production.md",
|
||||
"batch": "R7.5",
|
||||
"judged_at": "2026-07-31",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 16,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/jailbreak-prevention-production.md#4",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/ai-services/content-safety/concepts/jailbreak-detection",
|
||||
"evidence_quote": "Prompt Shields for Documents attacks recognizes the following classes of attacks: Manipulated Content | Allowing a compromised LLM to access system infrastructures | Information Gathering | Availability | Fraud | Malware | Attempt to change system rules | Embedding a conversation mockup to confuse the model | Role-Play | Encoding Attacks",
|
||||
"reason": "Antallet er en bærende del og er feil: siden enumererer ti kategorier for Document attacks (og claimet lister selv ti elementer), ikke ni.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/jailbreak-prevention-production.md",
|
||||
"line": 46,
|
||||
"claim": "Document Attacks (indirekte angrep, også kalt Indirect Prompt Attacks / Cross-Domain Prompt Injection Attacks) har ni hovedkategorier, angitt i filen som: Manipulated Content | Infrastructure Access | Information Gathering | Availability | Fraud | Malware | Attempt to change system rules | Embedding a conversation mockup | Role-Play | Encoding Attacks.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/jailbreak-prevention-production.md#9",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/default-safety-policies",
|
||||
"evidence_quote": "| Risk category | Prompt or completion | Severity threshold | ... | User prompt injection attack (jailbreak) | Prompts | N/A | | Protected material – text | Completions | N/A |",
|
||||
"reason": "Kategori/prompt-completion/terskel stemmer eksakt, men de bærende handlings-påstandene holder ikke: dagens tabell har ingen handlings-kolonne, og konfigurasjonssiden beskriver motsatt paring — Prompt Shields for direct attacks (jailbreak) «Filters / annotates user prompts», mens Protected material - text «Identifies and blocks known text content».",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/jailbreak-prevention-production.md",
|
||||
"line": 227,
|
||||
"claim": "Default safety policies for Azure OpenAI tekstmodeller: Hate and Fairness | Violence | Sexual | Self-Harm filtreres på prompts og completions med severity threshold Medium; User prompt injection attack (Jailbreak) på prompts har terskel N/A og handling Detect and block; Protected Material – Text og Protected Material – Code på completions har terskel N/A og handling Annotate/Filter.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-security/ai-security-engineering/jailbreak-prevention-production.md#12",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/ai-services/content-safety/quickstart-custom-categories",
|
||||
"evidence_quote": "curl -X POST \"<your_endpoint>/contentsafety/text:analyzeCustomCategory?api-version=2024-09-15-preview\" ... -d \"{ \\\"text\\\": \\\"<Example text to analyze>\\\", \\\"categoryName\\\": \\\"survival-advice\\\", \\\"version\\\": 1 }\"",
|
||||
"reason": "Sti og felter (text, categoryName, version) stemmer, men den bærende api-version-strengen er feil: live-siden bruker 2024-09-15-preview, ikke 2024-09-01.",
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/jailbreak-prevention-production.md",
|
||||
"line": 305,
|
||||
"claim": "Azure AI Content Safety custom categories kalles via POST <endpoint>/contentsafety/text:analyzeCustomCategory med api-version=2024-09-01 og feltene text, categoryName og version.",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-security/references/ai-security-engineering/zero-trust-ai-services.md",
|
||||
"batch": "R7.1",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue