ms-ai-architect/skills/ms-ai-governance/references/monitoring-observability/anomaly-detection-ai-systems.md

# Anomaly Detection for AI Systems

**Last updated:** 5. februar 2026
**Category:** Monitoring & Observability
**Målgruppe:** AI-arkitekter, DevOps-team, MLOps-ingeniører
**Type:** reference
**Source:** https://learn.microsoft.com/azure/ai-services/anomaly-detector/overview
**Status:** Established Practice

## Innhold

- [Oversikt](#oversikt)
- [Smart Detection Capabilities](#smart-detection-capabilities)
- [Custom Anomaly Rules for AI](#custom-anomaly-rules-for-ai)
- [Behavioral Baseline Detection](#behavioral-baseline-detection)
- [Drift Detection Patterns](#drift-detection-patterns)
- [Alert Correlation](#alert-correlation)
- [Azure AI-Specific Detection](#azure-ai-specific-detection)
- [Implementeringsmønster](#implementeringsmønster)
- [For Cosmo](#for-cosmo)

## Oversikt

Anomaly detection er kritisk for proaktiv overvåking av AI-systemer. Microsoft Azure tilbyr flere mekanismer for å oppdage avvikende oppførsel i AI-applikasjoner, fra innebygde ML-baserte funksjoner til dedikerte tjenester. Effektiv anomaly detection reduserer tiden fra et problem oppstår til det blir oppdaget (dwell time) og muliggjør raskere respons på trusler og systemfeil.

## Smart Detection Capabilities

### Application Insights Smart Detection

Application Insights inkluderer automatisk smart detection som bruker maskinlæring til å oppdage avvik uten konfigurasjon. Systemet analyserer telemetri kontinuerlig og varsler automatisk ved potensielle problemer.

**Hovedfunksjoner:**

1. **Failure Anomalies Detection**
   - Oppdager unormal økning i feilrate
   - Korrelerer feilrater med last og andre faktorer
   - Bruker maskinlæring til å etablere forventet baseline
   - Trenger 24 timer med data før aktivering

2. **Performance Anomalies Detection**
   - Detekterer degradering i responstid
   - Analyserer både requests og dependencies
   - Identifiserer mønstre i page load time
   - Sammenligner med historisk baseline

3. **General Degradations**
   - Trace severity degradation
   - Memory leaks
   - Abnormal exception volume
   - Security anti-patterns

**Konfigurasjon:**

Smart detection krever ingen oppsett hvis Application Insights sender nok telemetri. Default e-postvarsler sendes til Monitoring Reader og Monitoring Contributor-roller.

```json
// Azure Resource Manager template for konfigurasjon
{
  "type": "Microsoft.Insights/components/ProactiveDetectionConfigs",
  "properties": {
    "enabled": true,
    "sendEmailsToSubscriptionOwners": true,
    "customEmails": ["ops-team@example.com"]
  }
}
```

### Migrering til Alert-Based Smart Detection

Microsoft anbefaler å migrere smart detection til alerts-basert system for bedre kontroll:

- Oppretter alert rules for hver deteksjonsmodul
- Muliggjør action groups for notifikasjoner
- Gir bedre integrasjon med Azure Monitor alerts
- Støtter multiple notification methods

**Migreringsmåter:**
1. Via Azure Portal (manuell migrering)
2. Via Azure CLI med REST API
3. Via ARM templates for batch-migrering

## Custom Anomaly Rules for AI

### KQL Machine Learning Functions

Azure Monitor Logs støtter KQL-baserte ML-funksjoner for anomaly detection uten behov for datascience-ekspertise.

**series_decompose_anomalies() - Hovedfunksjon:**

```kusto
// Detect anomalies i AI-telemetri
let starttime = 21d;
let endtime = 0d;
let timeframe = 1h; // Sample frequency
AIRequests
| where TimeGenerated between (startofday(ago(starttime))..startofday(ago(endtime)))
| make-series RequestRate=count() default=0
    on TimeGenerated
    from startofday(ago(starttime))
    to startofday(ago(endtime))
    step timeframe
    by ModelName
| extend (Anomalies, AnomalyScore, ExpectedRate) =
    series_decompose_anomalies(RequestRate, 1.5, -1, 'avg', 1)
| mv-expand RequestRate to typeof(double),
    TimeGenerated to typeof(datetime),
    Anomalies to typeof(double),
    AnomalyScore to typeof(double),
    ExpectedRate to typeof(long)
| where Anomalies != 0
| project TimeGenerated, ModelName, RequestRate, ExpectedRate, AnomalyScore, Anomalies
| sort by abs(AnomalyScore) desc
```

**Parametere for tuning:**

- **Threshold** (default 1.5): Justerer sensitivitet – lavere verdi gir flere anomalier
- **Seasonality** (default -1): Auto-detect sesongvariasjoner
- **Trend** (default 'avg'): 'avg', 'linefit', eller 'none'
- **Test_points**: Antall punkter å ekskludere fra learning (for outliers)
- **AD_method**: Anomaly detection-metode

### Root Cause Analysis med diffpatterns()

Når anomalier oppdages, bruk `diffpatterns()` plugin for å identifisere årsaker:

```kusto
let anomalyDate = datetime(2026-02-05T12:00:00Z);
AIRequests
| extend AnomalyDate = iff(TimeGenerated == anomalyDate, "AnomalyDate", "OtherDates")
| where TimeGenerated between (ago(7d)..now())
| project AnomalyDate, Operation, ResultCode, ModelVersion, Region
| evaluate diffpatterns(AnomalyDate, "OtherDates", "AnomalyDate", "~", 0.20)
```

**Output:** Tabell som viser hvilke dimensjoner (operation, resultcode, etc.) som varierer mest mellom normal og anomal periode.

## Behavioral Baseline Detection

### Etablering av Baseline

Smart detection etablerer automatisk behavioral baselines over tid:

1. **Learning Period**: Minimum 24 timer (ofte 7-14 dager for robust baseline)
2. **Continuous Learning**: Modellen oppdateres kontinuerlig med nye data
3. **Context-Aware**: Korrelerer med faktorer som load, tid på døgnet, ukedag
4. **Adaptive Thresholds**: Dynamiske terskler basert på historikk

### Dynamic Thresholds for Metric Alerts

Azure Monitor tilbyr dynamiske terskler basert på maskinlæring for metric alerts:

```json
{
  "criteria": {
    "allOf": [{
      "name": "AI Model Response Time",
      "metricName": "ResponseTime",
      "operator": "GreaterThan",
      "threshold": "dynamic",
      "sensitivity": "Medium",
      "failingPeriods": {
        "numberOfEvaluationPeriods": 4,
        "minFailingPeriodsToAlert": 3
      }
    }]
  }
}
```

**Sensitivity levels:**
- **High**: Lavere toleranse, fanger flere anomalier (mer false positives)
- **Medium**: Balansert (anbefalt for de fleste scenarioer)
- **Low**: Høyere toleranse, færre varsler

### Behavioral Patterns for AI Systems

Spesifikke mønstre å overvåke for AI-systemer:

1. **Input Anomalies**
   - Uventede prompt-lengder
   - Høy forekomst av special characters
   - Repetitive patterns (potensielt angrep)

2. **Output Anomalies**
   - Plutselig endring i response-lengder
   - Avvik i token consumption patterns
   - Uventede confidence scores

3. **Performance Anomalies**
   - Latency spikes
   - Throughput degradation
   - Rate limit hits

4. **Resource Anomalies**
   - Abnormal compute usage
   - Memory consumption spikes
   - Storage I/O patterns

## Drift Detection Patterns

Model drift er en spesiell form for anomaly detection kritisk for AI-systemer.

### Data Drift Detection

Overvåk endringer i input-distribusjon:

```kusto
// Detect distribution shifts i prompt-karakteristikker
let baseline_period = 7d;
let current_period = 1d;
let baseline = AIRequests
    | where TimeGenerated between (ago(baseline_period + current_period)..ago(current_period))
    | summarize
        AvgTokens=avg(PromptTokens),
        StdDevTokens=stdev(PromptTokens),
        P50=percentile(PromptTokens, 50),
        P95=percentile(PromptTokens, 95)
    | extend Period = "Baseline";
let current = AIRequests
    | where TimeGenerated > ago(current_period)
    | summarize
        AvgTokens=avg(PromptTokens),
        StdDevTokens=stdev(PromptTokens),
        P50=percentile(PromptTokens, 50),
        P95=percentile(PromptTokens, 95)
    | extend Period = "Current";
union baseline, current
| evaluate pivot(Period, sum(AvgTokens), sum(P50), sum(P95))
| extend
    AvgDrift = (Current_AvgTokens - Baseline_AvgTokens) / Baseline_AvgTokens * 100,
    P95Drift = (Current_P95 - Baseline_P95) / Baseline_P95 * 100
| where abs(AvgDrift) > 15 or abs(P95Drift) > 20 // Threshold: 15% avg eller 20% P95
```

### Concept Drift Detection

Overvåk endringer i modell-utdata:

```python
from azure.ai.anomalydetector import AnomalyDetectorClient
from azure.core.credentials import AzureKeyCredential

# Azure AI Anomaly Detector for univariate series
client = AnomalyDetectorClient(endpoint, AzureKeyCredential(api_key))

# Time series av confidence scores
series = [
    TimeSeriesPoint(timestamp=row[0], value=row[1])  # confidence score
    for row in data
]

request = UnivariateDetectionOptions(
    series=series,
    granularity=TimeGranularity.HOURLY,
    sensitivity=90
)

# Detect både anomalies og change points
anomaly_response = client.detect_univariate_entire_series(request)
changepoint_response = client.detect_univariate_change_point(request)

for i, (is_anomaly, is_changepoint) in enumerate(
    zip(anomaly_response.is_anomaly, changepoint_response.is_change_point)
):
    if is_changepoint:
        # Persistent shift - potential concept drift
        alert_drift(timestamp=series[i].timestamp)
    elif is_anomaly:
        # Temporary spike - potential transient issue
        alert_anomaly(timestamp=series[i].timestamp)
```

## Alert Correlation

### Korrelere Anomalier med Hendelser

Best practice er å korrelere anomalier med andre events:

```kusto
// Korrelere performance anomalies med deployment events
let anomalies = AIRequests
    | where TimeGenerated > ago(7d)
    | make-series RequestRate=count() default=0 on TimeGenerated step 5m
    | extend (Anomalies, Score, Expected) = series_decompose_anomalies(RequestRate)
    | mv-expand TimeGenerated to typeof(datetime), Anomalies to typeof(double), Score to typeof(double)
    | where Anomalies != 0
    | project AnomalyTime=TimeGenerated, Score;
let deployments = AzureActivity
    | where OperationNameValue == "MICROSOFT.RESOURCES/DEPLOYMENTS/WRITE"
    | where ActivityStatusValue == "Success"
    | project DeploymentTime=TimeGenerated, ResourceGroup, Deployment=Properties.deployment;
anomalies
| join kind=inner (deployments) on $left.AnomalyTime == $right.DeploymentTime
| where abs(datetime_diff('minute', AnomalyTime, DeploymentTime)) < 30
| project AnomalyTime, DeploymentTime, Score, ResourceGroup, Deployment
| order by Score desc
```

### Multi-Signal Correlation

Korrelere anomalier på tvers av signaler:

1. **Application-level metrics** (latency, throughput, errors)
2. **Infrastructure metrics** (CPU, memory, network)
3. **Model metrics** (confidence scores, token usage)
4. **Security signals** (authentication failures, suspicious patterns)

### Action Groups for Automated Response

Konfigurer action groups for koordinerte responser:

```json
{
  "actionGroups": [
    {
      "actionGroupId": "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Insights/actionGroups/AIAnomalyResponse",
      "webhookProperties": {
        "anomaly_type": "performance",
        "severity": "high",
        "auto_scale": "true"
      }
    }
  ]
}
```

**Mulige actions:**
- Email/SMS/Push notifications
- Webhook til incident management system
- Azure Function for automated remediation
- Logic App for workflow orchestration
- ITSM connector (ServiceNow, etc.)

## Azure AI-Specific Detection

### Defender for AI Services

Microsoft Defender for AI tilbyr spesialisert anomaly detection for AI-spesifikke trusler:

1. **Jailbreak Attempt Detection**
   - Mønstergjenkjenning av jailbreak-teknikker
   - Analyser prompt injection patterns
   - Korrelere med MITRE ATLAS framework

2. **Model Inference Anomalies**
   - Uvanlige API call patterns
   - Excessive inference requests
   - Suspicious input/output correlations

3. **Data Exfiltration Patterns**
   - Abnormal data access via model queries
   - High-volume low-latency requests
   - Sensitive data in prompts/responses

**Aktivering:**

Defender for AI aktiveres via Security Center:

```bash
# Azure CLI
az security pricing create \
  --name AIServices \
  --tier Standard \
  --subscription <subscription-id>
```

### Azure AI Anomaly Detector Service

Dedikert service for anomaly detection (NB: Retired 1. oktober 2026 – bruk alternativene nedenfor):

**Alternativer etter retirement:**

1. **Azure ML model monitoring** – for model-spesifikk anomaly detection
2. **Azure Monitor KQL-baserte funksjoner** – for log-basert detection
3. **Azure Stream Analytics** – for real-time streaming anomaly detection
4. **Custom models** i Azure ML – for spesialiserte use cases

### Real-Time Intelligence Anomaly Detection (Fabric)

For organisasjoner med Microsoft Fabric:

```python
# Python plugin i Eventhouse
from synapse.ml.services import SimpleDetectAnomalies

anomaly_detector = (SimpleDetectAnomalies()
  .setTimestampCol("timestamp")
  .setValueCol("model_confidence")
  .setOutputCol("anomalies")
  .setGroupbyCol("model_name")
  .setGranularity("hourly"))

result = anomaly_detector.transform(df)
display(result.select("timestamp", "model_confidence", "anomalies.isAnomaly"))
```

## Implementeringsmønster

### 1. Etabler Baseline (Uke 1-2)

```kusto
// Etabler baseline for key metrics
AIRequests
| where TimeGenerated between (ago(14d)..now())
| summarize
    P50_Latency=percentile(Duration, 50),
    P95_Latency=percentile(Duration, 95),
    P99_Latency=percentile(Duration, 99),
    AvgTokens=avg(TotalTokens),
    ErrorRate=countif(Success == false) * 100.0 / count()
  by bin(TimeGenerated, 1h), ModelName
| render timechart
```

### 2. Konfigurer Anomaly Detection

```bash
# Opprett alert rule med dynamic threshold
az monitor metrics alert create \
  --name "AI-Latency-Anomaly" \
  --resource-group <rg> \
  --scopes <app-insights-id> \
  --condition "avg requests/duration > dynamic High 4 of 4" \
  --window-size 5m \
  --evaluation-frequency 1m \
  --action <action-group-id>
```

### 3. Implementer Root Cause Analysis Automation

```python
# Azure Function triggered av alert
import azure.functions as func
from azure.monitor.query import LogsQueryClient

def main(req: func.HttpRequest) -> func.HttpResponse:
    alert_data = req.get_json()
    anomaly_time = alert_data['data']['context']['timestamp']

    # Query for root cause
    query = f"""
    AIRequests
    | where TimeGenerated between (datetime({anomaly_time}) - 30m .. datetime({anomaly_time}) + 30m)
    | summarize ErrorCount=countif(Success==false) by Operation, ResultCode
    | top 10 by ErrorCount desc
    """

    result = logs_client.query_workspace(workspace_id, query)

    # Send enriched alert
    send_enriched_alert(result)
```

### 4. Continuous Tuning

Juster sensitivitet basert på false positive rate:

- Hvis > 30% false positives: øk threshold eller sensitivity
- Hvis < 5% false positives: reduser threshold for tidligere detection
- Revurder baseline hver måned ved sesongrelaterte endringer

## For Cosmo

### Når anbefale anomaly detection

**ALLTID anbefal** for:
- Produksjons-AI-applikasjoner med høy trafikk
- AI-systemer med sensitive data eller compliance-krav
- Multimodal AI-løsninger med komplekse dependencies
- AI-agenter med autonom beslutningskraft

**Ikke kritisk** for:
- Proof-of-concepts under utvikling
- Lavtrafikks prototype-løsninger uten produksjonsdata

### Platform-spesifikke anbefalinger

| Plattform | Primær Metode | Sekundær Metode |
|-----------|---------------|-----------------|
| Microsoft Foundry | Application Insights Smart Detection | KQL-baserte custom queries |
| Copilot Studio | M365 audit logs + KQL | Application Insights (via plugin) |
| Power Platform AI | Application Insights + Power Platform analytics | Custom Dataverse queries |
| Azure OpenAI Service | Application Insights + Defender for AI | Azure Monitor metric alerts |

### Arkitekturdialog

**Spørsmål å stille:**

1. "Hvilke typer avvik er viktigst å oppdage for deres AI-applikasjon – performance, sikkerhet, eller datakvalitet?"
2. "Har dere eksisterende alert-systemer dette må integreres med?"
3. "Hva er akseptabel responstid fra anomaly til varsling?"
4. "Trenger dere automated remediation eller kun notifikasjoner?"

**Typiske trade-offs:**

- **Sensitivity vs. Alert Fatigue**: Høyere sensitivitet gir flere false positives
- **Real-time vs. Batch**: Real-time detection krever mer ressurser
- **Custom vs. Built-in**: Custom ML-modeller gir bedre presisjon men høyere vedlikeholdskostnad

### Kostnadsestimat

Anomaly detection koster primært via:
1. **Log Analytics ingestion**: ~NOK 30/GB
2. **Application Insights**: Inkludert i Basic-tier (gratis til 5 GB/mnd)
3. **Alert rules**: Gratis for første 10 metric alerts, NOK 1/mnd per ekstra
4. **Action groups**: Gratis for de fleste notification types

**Tommelfingerregel:** Budsjetter NOK 500-2000/mnd for typisk produksjons-AI-app med comprehensive anomaly detection.

---

**Sources:**
- [Tutorial: Detect and analyze anomalies using KQL](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/kql-machine-learning-azure-monitor)
- [Smart detection in Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/proactive-diagnostics)
- [Detect and mitigate potential issues using AIOps and machine learning](https://learn.microsoft.com/en-us/azure/azure-monitor/aiops/aiops-machine-learning)
- [Azure Monitor dynamic thresholds](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-dynamic-thresholds)
- [Microsoft Defender for AI Services](https://learn.microsoft.com/en-us/azure/defender-for-cloud/ai-threat-protection)
- [Anomaly detection in Real-Time Intelligence (Fabric)](https://learn.microsoft.com/en-us/fabric/real-time-intelligence/anomaly-detection)
- [Azure AI Anomaly Detector](https://learn.microsoft.com/en-us/azure/ai-services/anomaly-detector/overview) (retired Oct 2026)
- [Azure Stream Analytics anomaly detection](https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-machine-learning-anomaly-detection)