docs(architect): weekly KB update — 66 files refreshed (2026-04)
Updated 66 stale knowledge base reference files (10 critical, 56 high) across all 5 skills using Microsoft Learn MCP research. Key factual updates: - Groundedness Detection API: `correction` → `mitigating` param, `correctedText` → `correctionText` (breaking change) - Copilot Studio: GPT-4.1 mini now default (was GPT-4o mini); Claude Sonnet 4.5 + Opus 4.5 added (experimental, 200K ctx) - Agentic Retrieval: still public preview; 50M free tokens/month - Azure security baselines: "Cognitive Services" → "Foundry Tools" - Databricks: Delta Live Tables → Lakeflow Spark Declarative Pipelines - MLflow 3 GenAI: new Feedback/Expectation data model - Token tracking doc: "Azure OpenAI in Foundry Models through a gateway" - Agent Registry: Risks column (M365 E7), Graph API (preview) - Copilot DLP: new Entra AI Admin + Purview Data Security AI Admin roles - ISO/IEC 42001: scope expanded to M365 Copilot, Foundry, Security Copilot - Zero Trust: CAE now via Conditional Access, Strict Location Enforcement - Purview: new Fabric Copilots/agents governance section - AG-UI HITL: ApprovalRequiredAIFunction (C#), @tool approval_mode (Python) All files: Last updated → 2026-04, *(Verified MCP 2026-04)* markers added. Build registry: 1341 URLs from 387 files (+2 new URLs). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen
parent
f72592ebf3
commit
565043dbde
73 changed files with 727 additions and 301 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Alerting Strategies and Escalation Policies for AI Incidents
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Last updated:** 2026-04
|
||||
**Status:** GA
|
||||
**Category:** Monitoring & Observability
|
||||
|
||||
|
|
@ -24,7 +24,7 @@ Eskaleringsrutiner må reflektere organisasjonens modenhetsnivå. En Minimum Via
|
|||
|-----------|-------------|-------------|
|
||||
| **Alert Rules** | Definerer betingelser som trigger alerts (metrics, logs, activity log) | Token rate limits, model latency, failed requests |
|
||||
| **Action Groups** | Samling av notifications og actions som kjøres når alert fires | Email, SMS, webhook, runbook, Logic App |
|
||||
| **Alert Processing Rules** | Overstyr eller beriket alert-oppførsel (f.eks. suppression under maintenance) | Prevent alert fatigue under model redeployments |
|
||||
| **Alert Processing Rules** | Overstyr eller berik alert-oppførsel (suppression, tillegg av action groups, scheduling). Filtre støtter opptil 5 verdier per filter (logisk OR). Suppression har høyere prioritet enn Apply action groups. | Prevent alert fatigue under model redeployments; recurring maintenance windows *(Verified MCP 2026-04)* |
|
||||
| **Common Alert Schema** | Uniform JSON payload på tvers av alle alert-typer | Forenkler webhook-integrasjoner og ITSM-connectors |
|
||||
| **Severity Levels** | Sev 0 (Critical) til Sev 4 (Informational) | Map til business impact (Sev 0 = PII leak, Sev 4 = latency spike) |
|
||||
|
||||
|
|
@ -39,6 +39,8 @@ Eskaleringsrutiner må reflektere organisasjonens modenhetsnivå. En Minimum Via
|
|||
| **Azure App Push** | Mobile notifications til Azure-appen | Begrenset til Azure mobile app | ✅ Yes |
|
||||
| **Event Hub** | Stream alerts til analytics/SIEM | Supports Private Link og NSP | ✅ Yes (regional) |
|
||||
|
||||
**Reusability:** Du kan legge til opptil 5 action groups per alert rule. Action groups kjøres concurrent (ingen bestemt rekkefølge). Multiple alert rules kan bruke samme action group. *(Verified MCP 2026-04)*
|
||||
|
||||
**Rate Limiting:** Azure Monitor rate-limiter notifications for å hindre spam. Hvis samme email/SMS/telefonnummer mottar for mange alerts, suspenderes notifications midlertidig. For AI-systemer som kan generere høy alert-volum (f.eks. per-request failures), bruk programmatic actions (Logic Apps, Automation Runbooks) i stedet.
|
||||
|
||||
### Action Types for AI Incidents
|
||||
|
|
@ -53,16 +55,18 @@ Eskaleringsrutiner må reflektere organisasjonens modenhetsnivå. En Minimum Via
|
|||
}
|
||||
```
|
||||
|
||||
| Action Type | AI Use Case | Authentication | Cross-tenant Support |
|
||||
|-------------|-------------|----------------|---------------------|
|
||||
| **Automation Runbook** | Auto-scale Azure OpenAI TPM, restart failing deployments | Managed Identity (Automation Contributor role) | ❌ No |
|
||||
| **Logic App** | Enrich alert med model metadata, post til Teams/Slack | Managed Identity (Logic App Contributor) | ❌ No |
|
||||
| **Azure Function** | Custom logic (e.g., invoke model rollback API) | HTTP trigger med access key | ❌ No |
|
||||
| **Webhook** | Invoke external incident mgmt (PagerDuty, ServiceNow) | Basic auth via URI eller secure webhook (Entra ID) | ✅ Yes (limited) |
|
||||
| **Event Hub** | Stream til SIEM (Microsoft Sentinel) for correlation | Managed Identity (Event Hubs Data Sender) | ✅ Yes (up to API 2023-09) |
|
||||
| **ITSM Connector** | Create incidents i ServiceNow, Cherwell | ITSM connection credentials | ❌ No |
|
||||
| Action Type | AI Use Case | Authentication | Managed Identity | Cross-tenant Support |
|
||||
|-------------|-------------|----------------|-----------------|---------------------|
|
||||
| **Automation Runbook** | Auto-scale Azure OpenAI TPM, restart failing deployments | Managed Identity (Automation Contributor, Role ID: f353d9bd) | ✅ Yes | ❌ No |
|
||||
| **Logic App** | Enrich alert med model metadata, post til Teams/Slack | Managed Identity (Logic App Contributor, Role ID: 87a39d53) | ✅ Yes | ❌ No |
|
||||
| **Azure Function** | Custom logic (e.g., invoke model rollback API) | HTTP trigger med access key | ❌ No | ❌ No |
|
||||
| **Webhook** | Invoke external incident mgmt (PagerDuty, ServiceNow) | Basic auth via URI eller secure webhook (Entra ID) | ❌ No | ✅ Yes (limited) |
|
||||
| **Event Hub** | Stream til SIEM (Microsoft Sentinel) for correlation | Managed Identity (Event Hubs Data Sender, Role ID: 2b629674) | ✅ Yes | ✅ Yes (up to API 2023-09-01-preview) |
|
||||
| **ITSM Connector** | Create incidents i ServiceNow, Cherwell | ITSM connection credentials | ❌ No | ❌ No |
|
||||
|
||||
**Managed Identity Best Practice:** For Automation Runbooks og Logic Apps, bruk managed identity i stedet for service principals. Azure Portal legger automatisk til role assignments. For PowerShell/CLI/SDK må du manuelt tildele roller (se tabell over).
|
||||
*(Verified MCP 2026-04)*
|
||||
|
||||
**Managed Identity Best Practice:** For Automation Runbooks, Logic Apps og Event Hubs, bruk managed identity i stedet for service principals. Azure Function og Webhook støtter ikke managed identity — bruk HTTP trigger access key respektive secure webhook med Entra ID. Azure Portal legger automatisk til role assignments ved konfigurasjon. For PowerShell/CLI/SDK må du manuelt tildele roller (se tabell over). *(Verified MCP 2026-04)*
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -225,6 +229,18 @@ START: AI Alert Fired
|
|||
YES─> Email only
|
||||
```
|
||||
|
||||
### Anbefalinger fra Well-Architected Framework
|
||||
|
||||
*(Verified MCP 2026-04)*
|
||||
|
||||
| Anbefaling | Fordel |
|
||||
|------------|--------|
|
||||
| **Bruk dynamic thresholds i metric alerts** | ML-baserte terskler tilpasser seg automatisk — unngår manuell threshold-tuning for AI-workloads med variabelt load |
|
||||
| **Bruk managed identity for log search alert rules** | Credential-free auth; påkrevd hvis query aksesserer Azure Data Explorer (ADX) eller Azure Resource Graph (ARG) |
|
||||
| **Bruk custom properties i alert rules** | Legg til metadata (owner, cost center, AI Act reportable) i alert payload — enklere triaging og compliance-logging |
|
||||
| **Bruk Logic Apps for enrichment** | Customiser notification-format, hent tags fra ressurser, integrer med Teams/Slack/PagerDuty uten å endre alert rule |
|
||||
| **Activity log alerts er gratis** | Bruk activity log alerts for administrative hendelser der mulig — ingen kostnad per evaluering |
|
||||
|
||||
### Vanlige Feil (Red Flags)
|
||||
|
||||
| Anti-pattern | Problem | Anbefaling |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue