Kjell Tore Guttormsen
03d596e4ec
Verifisert mot offisiell MS-doc (juni 2026): «Microsoft Foundry» er det gjeldende produkt-/portalnavnet; «Foundry (classic)» = gamle «Azure AI Foundry» (/azure/foundry/ vs /azure/foundry-classic/). Premiss bekreftet før sveip. Multi-regel, IKKE naiv s/Azure AI Foundry/Microsoft Foundry/ — MS dropper «Azure AI» (legger IKKE til «Microsoft») for to produktvarianter: - «Azure AI Foundry Agent[ Service|s]» → «Foundry Agent Service/Agents» (MS-form) - «Azure AI Foundry Models» → «Foundry Models» (i «Azure OpenAI in Foundry Models») - «Azure AI Foundry SDK» → «Microsoft Foundry SDK» (operatør-valg) - «Azure AI Foundry portal/project» + generisk → «Microsoft Foundry» - Pre-eksisterende «Microsoft Foundry Models» (4) normalisert → «Foundry Models» Bevart: «Azure OpenAI», «Azure AI Inference SDK», «Azure AI Search», «Azure AI Services», kode-IDer. Historisk ref «(tidligere Azure AI Foundry)» i model-catalog-2026.md beskyttet via lookbehind. URL /azure/ai-foundry/→ /azure/foundry/ kun i owasp-llm-top10 (KB-ref); docs/-filer deferred. Scope: skills (inkl. 3 SKILL.md) + commands + agents + README + CLAUDE. Ekskludert: docs/ (interne), playground/+tests/ fixtures (testdata), CHANGELOG.md (historisk logg), STATE.md (gitignored). 3 SKILL.md endret (advisor/engineering/security) → judge-cache teknisk invalidert for disse, men scorer uendret: advisor 91, eng/gov/infra/sec 96 (alle ≥90). validate 239/0. 0 «Azure AI Foundry» igjen (utenom bevart ref). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
196 lines
13 KiB
Markdown
196 lines
13 KiB
Markdown
---
|
|
name: ms-ai-security
|
|
description: >-
|
|
Security assessment, cost estimation, OWASP LLM Top 10 mitigations, performance optimization for AI on Microsoft stack. Deterministic 6x5 security scoring, P10/P50/P90 cost confidence intervals, FinOps practices. Triggers on: "security assessment for AI", "AI threat modeling", "cost estimation for Azure AI", "FinOps for AI workloads", "OWASP LLM", "kostnadsestimat for AI-løsning".
|
|
---
|
|
|
|
> **INSTRUKSJON:** Denne skillen dekker kvantitative vurderingsaktiviteter med deterministiske
|
|
> scoringsmodeller. Bruk rammeverket systematisk — ikke hopp over dimensjoner eller anta scorer.
|
|
> Alle vurderinger skal produsere konkrete, etterprøvbare resultater med tallverdier.
|
|
|
|
# Sikkerhets- og kostnadsvurdering for Microsoft AI
|
|
|
|
Strukturerte metoder for tre vurderingsaktiviteter:
|
|
|
|
1. **Sikkerhetsvurdering** — Deterministisk 6x5 sikkerhetsscoring med OWASP LLM Top 10-mapping
|
|
2. **Kostnadsestimering** — TCO-beregning med P10/P50/P90 konfidensintervaller og FinOps-praksis
|
|
3. **Ytelsesgjennomgang** — Latency-optimalisering, skalering og benchmarking
|
|
|
|
**Primære agenter:** security-assessment-agent, cost-estimation-agent
|
|
|
|
---
|
|
|
|
## 1. Sikkerhetsrammeverk
|
|
|
|
### 6-dimensjons sikkerhetsmodell
|
|
|
|
To assess security, score each of the six dimensions independently on a 1-5 scale:
|
|
|
|
| Dimensjon | Dekker |
|
|
|-----------|--------|
|
|
| Identity & Access Control | Entra ID, Managed Identities, RBAC, API-nøkkelrotasjon, JIT-tilgang |
|
|
| Network Security | Private Endpoints, VNet, NSG, Azure Firewall, DNS, utgående trafikk |
|
|
| Data Protection | Kryptering (rest/transit), Key Vault, data residency, PII-maskering, backup |
|
|
| Content Safety & AI Security | Content Safety-filtre, prompt injection-forsvar, jailbreak, output-validering, STRIDE-AI |
|
|
| Compliance & Governance | AI Act-klassifisering, GDPR/Schrems II, Purview, Digdir/NSM, DPIA |
|
|
| Monitoring & Incident Response | Azure Monitor, token-bruk, anomalideteksjon, audit logging, alerting |
|
|
|
|
### Scoringmodell (1-5)
|
|
|
|
| Score | Nivå | Kriterium |
|
|
|-------|------|-----------|
|
|
| **1** | Kritisk | Ingen kontroller. Umiddelbar risiko. |
|
|
| **2** | Utilstrekkelig | Grunnleggende kontroller med vesentlige hull. Kun PoC/sandbox. |
|
|
| **3** | Akseptabel | Sentrale kontroller på plass. Minimum for lav-risiko produksjon. |
|
|
| **4** | God | Robuste, automatiserte kontroller med overvåking. Sensitiv data OK. |
|
|
| **5** | Utmerket | State-of-the-art. Zero Trust. Defense in depth. Høy-risiko AI Act OK. |
|
|
|
|
### Vektet scoring
|
|
|
|
Apply weights based on workload type, then calculate: **Samlet score = Sum(dimensjon_score x vekt)**. De kanoniske vektene (Standard-profil) og de to arbeidsbelastnings-variantene (eksternt eksponert, persondata-intensiv) ligger i rubrikkfila — `references/ai-security-engineering/security-scoring-rubrics-6x5.md` — slik at scoringen holdes konsistent med `security-assessment-agent`. Ikke dupliser vekttallene her.
|
|
|
|
### Risikoklassifisering
|
|
|
|
Etter at samlet score er beregnet, klassifiser løsningen i risikokategori med anbefalt handling. De kanoniske terskelverdiene (samlet score → risikokategori, inkludert «Uakseptabel»-kategorien) ligger i samme rubrikkfil som vektene — `references/ai-security-engineering/security-scoring-rubrics-6x5.md` — slik at klassifiseringen holdes konsistent med `security-assessment-agent`. Ikke dupliser terskeltallene her.
|
|
|
|
For fullstendige rubrikker med eksempler per dimensjon og score, see `references/ai-security-engineering/security-scoring-rubrics-6x5.md` and `references/ai-security-engineering/ai-security-scoring-framework.md`.
|
|
|
|
### OWASP LLM Top 10 (2025)
|
|
|
|
Map each threat to the solution under assessment. Use the reference files for detailed mitigation patterns.
|
|
|
|
| ID | Threat | Key Microsoft Mitigation | Reference |
|
|
|----|--------|--------------------------|-----------|
|
|
| LLM01 | Prompt Injection | Content Safety Prompt Shields, system message hardening, Groundedness Detection | `prompt-injection-defense-patterns.md` |
|
|
| LLM02 | Sensitive Information Disclosure | PII-filter, Purview DLP, output-filtrering | `data-leakage-prevention-ai.md`, `pii-detection-norwegian-context.md` |
|
|
| LLM03 | Supply Chain Vulnerabilities | AI Foundry curated models, signed models, DLP for connectors | `supply-chain-security-ai-models.md` |
|
|
| LLM04 | Data and Model Poisoning | Azure ML data lineage, isolated fine-tuning, Purview validation | `owasp-llm-top10-azure-mitigations.md` |
|
|
| LLM05 | Improper Output Handling | Grounding Detection API, Content Safety output-filtre, Structured Outputs | `output-validation-grounding-verification.md` |
|
|
| LLM06 | Excessive Agency | Copilot Studio scoped tools, RBAC per project, human-in-the-loop, budget caps | `owasp-llm-top10-azure-mitigations.md` |
|
|
| LLM07 | System Prompt Leakage | Metaprompt patterns, Prompt Shields, output monitoring | `jailbreak-prevention-production.md` |
|
|
| LLM08 | Vector and Embedding Weaknesses | AI Search managed identities, index-level security filters, Private Endpoints | `owasp-llm-top10-azure-mitigations.md` |
|
|
| LLM09 | Misinformation | RAG grounding, Groundedness Detection, citation patterns, confidence scoring | `owasp-llm-top10-azure-mitigations.md` |
|
|
| LLM10 | Unbounded Consumption | Rate limits, token budgets, PTU for capacity, Cost Management alerts | — |
|
|
|
|
All reference files are in `references/ai-security-engineering/`. LLM04/06/08/09 deler den konsoliderte filen `references/ai-security-engineering/owasp-llm-top10-azure-mitigations.md`; LLM10 dekkes av rate-limit-/kostnadsfiler.
|
|
|
|
Kjøretids-trusseldeteksjon for AI-endepunkter dekkes av `references/ai-security-engineering/defender-threat-protection-ai-services.md` (Defender for Cloud AI threat protection). Release-status, agent-dekning og regionstilgjengelighet — inkludert begrensningen for Azure Government — er dokumentert i ref-fila; verifiser mot den før du oppgir status.
|
|
|
|
### Azure AI-spesifikke sikkerhetskontroller
|
|
|
|
For detailed per-service security controls tables, see `references/ai-security-engineering/secure-model-deployment-hardening.md` and `references/ai-security-engineering/zero-trust-ai-services.md`. Key services covered:
|
|
|
|
- **Azure OpenAI Service** — Content Filtering, Abuse Monitoring, VNet/Private Endpoints, Managed Identity, CMK
|
|
- **Azure AI Search** — Managed Identities, index-level security filters, encryption, Private Endpoints
|
|
- **Copilot Studio** — Entra ID auth, Power Platform DLP, generative AI guardrails, environment isolation
|
|
- **Microsoft Foundry** — Project isolation, granular RBAC, Private Endpoints, curated model catalog, tracing
|
|
|
|
---
|
|
|
|
## 2. Kostnadsestimering
|
|
|
|
### P10/P50/P90 konfidensintervaller
|
|
|
|
Provide all estimates with three scenarios — P10 (lavt volum / minimumskostnad), P50 (forventet/median), P90 (høyt volum / worst-case budsjettering). De kanoniske usikkerhetsfaktorene beregnes **per komponent** (ikke som én flat multiplikator) og eies av `references/cost-optimization/deterministic-cost-calculation-model.md` §3 — bruk faktorene derfra, slik at estimatet er konsistent med `cost-estimation-agent` (som har modellen som OBLIGATORISK kilde). Verify current prices via `microsoft_docs_search` before calculating.
|
|
|
|
Always present both USD and NOK (add 3-5% currency buffer for NOK).
|
|
|
|
### TCO-komponenter
|
|
|
|
Calculate for 1, 12, and 36 months. Present Budget/Recommended/Premium alternatives.
|
|
|
|
| Komponent | Inkluderer | Eksempler |
|
|
|-----------|-----------|----------|
|
|
| **Lisenser** | Software per bruker/org | M365 Copilot, Copilot Studio, Power Platform |
|
|
| **Compute** | AI-inferens, hosting | Azure OpenAI tokens, App Service, Functions |
|
|
| **Storage** | Datalagring | AI Search indekser, Blob Storage, Cosmos DB |
|
|
| **Networking** | Dataoverføring | Egress, Private Link, Application Gateway |
|
|
| **Support** | Microsoft Support | Unified/Premier Support |
|
|
| **Drift** | Internt personell | Utviklere, MLOps, sikkerhetsteam |
|
|
|
|
See `references/cost-optimization/deterministic-cost-calculation-model.md` and `references/cost-optimization/budget-forecasting-ai-projects.md` for full calculation methodology.
|
|
|
|
### FinOps for AI
|
|
|
|
Apply these optimization strategies and refer to detailed guidance in references:
|
|
|
|
- **Token-optimalisering:** Shorter prompts, context window management, model tiering (bruk en rimeligere modell for enkle oppgaver), prompt caching. See `references/cost-optimization/token-counting-optimization.md` and `references/cost-optimization/model-selection-price-performance.md` for gjeldende modellvalg.
|
|
- **PTU vs Pay-As-You-Go:** PTU for stable, høyt-volum-arbeidsbelastninger (over et break-even-punkt for utnyttelse), PAYG for variable. Konkret break-even-terskel er pris-avhengig og ligger i ref-fila. See `references/cost-optimization/ptu-vs-paygo-economics.md`.
|
|
- **Caching:** Semantic caching, prompt caching, RAG result caching. See `references/cost-optimization/semantic-caching-patterns.md`.
|
|
- **Right-sizing:** Start with lowest SKU, monitor 2-4 weeks, consider SLMs for specialized tasks. See `references/cost-optimization/model-selection-price-performance.md`.
|
|
|
|
---
|
|
|
|
## 3. Ytelse og skalerbarhet
|
|
|
|
Optimize latency, throughput, and scalability for AI workloads. Konkrete tall — latens-reduksjon, caching-rabatt og batch-prising — er volatile og ligger i ref-filene; oppgi dem aldri fra hukommelsen. Key strategies:
|
|
|
|
- **Regional deployment** in Norway East / West Europe to reduce latency. See `references/performance-scalability/latency-optimization-azure-openai.md`.
|
|
- **Streaming responses** to reduce perceived latency for interactive use. See `references/performance-scalability/streaming-response-patterns.md`.
|
|
- **Prompt caching** for repeated prompt prefixes to cut both cost and latency. See `references/performance-scalability/prompt-caching-performance.md`.
|
|
- **Batch API** for non-interactive workloads at reduced price. See `references/cost-optimization/request-batching-aggregation.md`.
|
|
- **Auto-scaling patterns:** Horizontal scaling (App Service/AKS), load balancing (APIM/Traffic Manager), queue-based buffering (Service Bus+Functions), PTU+PAYG hybrid
|
|
- **Rate limit management:** TPM/RPM quotas, exponential backoff with jitter, multi-deployment, APIM for centralized throttling
|
|
- **Load testing:** Establish baseline, simulate peak traffic, identify breaking points, long-running soak tests
|
|
|
|
For detailed implementation guidance, see specific files in `references/performance-scalability/`:
|
|
- `references/performance-scalability/latency-optimization-azure-openai.md` — Latency tuning
|
|
- `references/performance-scalability/auto-scaling-ai-infrastructure.md` — Scaling patterns
|
|
- `references/performance-scalability/rate-limit-management.md` — TPM/RPM quota management
|
|
- `references/performance-scalability/load-testing-ai-services.md` — Load testing methodology
|
|
- `references/performance-scalability/gpu-compute-sizing.md` — GPU VM-sizing for selvhostet inferens (brukt av `/architect:cost --capacity`)
|
|
|
|
---
|
|
|
|
## 4. Referansekatalog
|
|
|
|
### Eide referanser
|
|
|
|
| Katalog | Filer | Innhold |
|
|
|---------|-------|---------|
|
|
| `references/ai-security-engineering/` | 22 | Forsvar, testing, scoring, hendelseshåndtering, Zero Trust, STRIDE-AI, prompt injection, content safety, OWASP LLM-tiltak, Defender AI threat protection |
|
|
| `references/cost-optimization/` | 22 | Kostnadsmodellering, FinOps, token-optimalisering, PTU/PAYG, caching, right-sizing, SLM-økonomi |
|
|
| `references/performance-scalability/` | 18 | Latency, skalering, streaming, batch API, rate limits, benchmarking, GPU-dimensjonering |
|
|
|
|
### Kryss-referanser
|
|
|
|
- **Compliance/governance:** `skills/ms-ai-governance/references/responsible-ai/` (AI Act, bias, etikk) and `references/norwegian-public-sector-governance/` (Digdir, NSM, Schrems II, DPIA)
|
|
- **Arkitektur:** `skills/ms-ai-advisor/references/architecture/` (sikkerhetssoner, arkitekturmønstre, offentlig sektor-sjekkliste)
|
|
|
|
---
|
|
|
|
## 5. MCP-verktøy
|
|
|
|
| Behov | Verktøy | Bruk |
|
|
|-------|---------|------|
|
|
| Sikkerhetsdokumentasjon | `microsoft_docs_search` | Verifiser kontroller, sjekk oppdateringer |
|
|
| Fullstendig veiledning | `microsoft_docs_fetch` | Security baselines, konfigurasjonsguider |
|
|
| Kodeeksempler | `microsoft_code_sample_search` | SDK for Content Safety, RBAC, Key Vault |
|
|
|
|
Never trust the knowledge base blindly for prices and feature availability — verify via MCP tools.
|
|
|
|
---
|
|
|
|
## 6. Arbeidsprosess
|
|
|
|
### Sikkerhetsvurdering
|
|
|
|
1. Map the solution's AI components and data flows
|
|
2. Score each of the 6 dimensions using rubrics from references
|
|
3. Calculate weighted risk score with appropriate weight profile
|
|
4. Map OWASP LLM Top 10 threats to the solution
|
|
5. Document findings with concrete remediation recommendations, prioritized by risk and cost
|
|
|
|
### Kostnadsestimering
|
|
|
|
1. Identify all Azure services in the solution
|
|
2. Estimate consumption per service (tokens, storage, traffic)
|
|
3. Fetch current prices via MCP tools
|
|
4. Calculate P10/P50/P90 per component, sum to TCO for 1/12/36 months
|
|
5. Present Budget/Recommended/Premium alternatives with FinOps opportunities
|
|
|
|
### Ytelsesgjennomgang
|
|
|
|
1. Define performance requirements (latency, throughput, availability)
|
|
2. Identify bottlenecks and recommend optimizations from reference catalog
|
|
3. Estimate performance impact and propose monitoring/benchmarking setup
|