ms-ai-architect/skills/ms-ai-infrastructure/references/bcdr/network-resilience-patterns-ai.md

Network Resilience Patterns for AI Workloads

Last updated: 2026-06-24 Status: GA Category: Business Continuity & Disaster Recovery Type: reference Source: https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview

---

Innhold

• Introduksjon
• Redundante nettverksstier og tilkoblinger
• Circuit Breaker-mønster for API-kall
• Graceful degradation av AI-tjenester
• Private endepunkter og nettverksisolering
• DDoS-beskyttelse og trafikkfiltrering
• Referanser
• For Cosmo

Introduksjon

Nettverksresiliens er en kritisk komponent i BCDR for AI-arbeidsbelastninger. AI-systemer er avhengige av pålitelig nettverkskommunikasjon mellom flere tjenester: applikasjonslaget, Azure OpenAI-endepunkter, AI Search-tjenester, embeddings-APIer og datastores. En nettverksforstyrrelse i ett punkt kan kaskadere og ta ned hele AI-løsningen.

Azure Well-Architected Framework definerer flere resiliensmønstre som er særlig relevante for AI-workloads: Circuit Breaker for å forhindre kaskadefeil, Retry med exponential backoff for transiente feil, Bulkhead for isolering av feildomener, og Throttling for å beskytte mot overbelastning. Disse mønstrene bør implementeres systematisk i alle AI-applikasjoner.

For norsk offentlig sektor er nettverkssikkerhet regulert gjennom NSMs grunnprinsipper, og mange organisasjoner bruker private endepunkter (Private Link) for sine Azure AI-tjenester. BCDR-designet må ta hensyn til disse nettverksrestriksjonene og sikre at failover fungerer også med private nettverkskonfigurasjoner.

Redundante nettverksstier og tilkoblinger

Multi-path nettverksarkitektur

┌─────────────┐
│  Brukere    │
└──────┬──────┘
       │
┌──────▼──────────────────────┐
│  Azure Front Door (Global)  │  ← innebygd DDoS-beskyttelse (edge)
└──────┬──────────────────────┘
       │
  ┌────┴────┐
  │         │
┌─▼──┐   ┌─▼──┐
│ R1 │   │ R2 │  ← To Azure-regioner
└─┬──┘   └─┬──┘
  │         │
┌─▼──────┐ ┌▼───────┐
│ VNet A │ │ VNet B │  ← Isolerte VNets per region
│ ├─APIM │ │ ├─APIM │
│ ├─App  │ │ ├─App  │
│ ├─PE   │ │ ├─PE   │  ← Private Endpoints til AI-tjenester
│ └─NSG  │ │ └─NSG  │
└────────┘ └────────┘

Azure ExpressRoute redundans

# Primær ExpressRoute-tilkobling
az network express-route create \
  --name "er-primary-norwayeast" \
  --resource-group "rg-networking" \
  --location "norwayeast" \
  --bandwidth-in-mbps 1000 \
  --peering-location "Oslo" \
  --provider "Telenor"

# Sekundær ExpressRoute (annen provider/lokasjon)
az network express-route create \
  --name "er-secondary-norwayeast" \
  --resource-group "rg-networking" \
  --location "norwayeast" \
  --bandwidth-in-mbps 1000 \
  --peering-location "Stavanger" \
  --provider "GlobalConnect"

# VPN som backup for ExpressRoute
az network vnet-gateway create \
  --name "vpn-gw-norwayeast" \
  --resource-group "rg-networking" \
  --location "norwayeast" \
  --vnet "vnet-ai-norwayeast" \
  --gateway-type Vpn \
  --sku VpnGw2AZ \
  --vpn-type RouteBased

DNS-resiliens

# Azure Private DNS Zones for AI-tjenester med failover
az network private-dns zone create \
  --resource-group "rg-networking" \
  --name "privatelink.openai.azure.com"

# Link til VNets i begge regioner
az network private-dns link vnet create \
  --resource-group "rg-networking" \
  --zone-name "privatelink.openai.azure.com" \
  --name "link-vnet-norwayeast" \
  --virtual-network "vnet-ai-norwayeast" \
  --registration-enabled false

az network private-dns link vnet create \
  --resource-group "rg-networking" \
  --zone-name "privatelink.openai.azure.com" \
  --name "link-vnet-swedencentral" \
  --virtual-network "vnet-ai-swedencentral" \
  --registration-enabled false

Circuit Breaker-mønster for API-kall

Circuit Breaker implementering

Circuit Breaker-mønsteret forhindrer at en applikasjon gjentatte ganger forsøker å kalle en tjeneste som feiler, noe som kan forårsake kaskadefeil og ressursutmattelse.

# Python Circuit Breaker for Azure OpenAI
import time
from enum import Enum
from threading import Lock

class CircuitState(Enum):
    CLOSED = "closed"      # Normal drift
    OPEN = "open"          # Stopp alle kall
    HALF_OPEN = "half_open"  # Prøv ett kall

class CircuitBreaker:
    """Circuit breaker for Azure AI service calls."""

    def __init__(
        self,
        failure_threshold=5,
        recovery_timeout=30,
        success_threshold=3
    ):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.success_threshold = success_threshold
        self.state = CircuitState.CLOSED
        self.failure_count = 0
        self.success_count = 0
        self.last_failure_time = None
        self.lock = Lock()

    def can_execute(self):
        """Check if a request can be made."""
        with self.lock:
            if self.state == CircuitState.CLOSED:
                return True
            elif self.state == CircuitState.OPEN:
                if time.time() - self.last_failure_time > self.recovery_timeout:
                    self.state = CircuitState.HALF_OPEN
                    return True
                return False
            elif self.state == CircuitState.HALF_OPEN:
                return True

    def record_success(self):
        """Record a successful call."""
        with self.lock:
            if self.state == CircuitState.HALF_OPEN:
                self.success_count += 1
                if self.success_count >= self.success_threshold:
                    self.state = CircuitState.CLOSED
                    self.failure_count = 0
                    self.success_count = 0
            else:
                self.failure_count = 0

    def record_failure(self):
        """Record a failed call."""
        with self.lock:
            self.failure_count += 1
            self.last_failure_time = time.time()
            if self.state == CircuitState.HALF_OPEN:
                self.state = CircuitState.OPEN
                self.success_count = 0
            elif self.failure_count >= self.failure_threshold:
                self.state = CircuitState.OPEN

# Bruk med Azure OpenAI
cb_openai = CircuitBreaker(failure_threshold=3, recovery_timeout=60)
cb_search = CircuitBreaker(failure_threshold=5, recovery_timeout=30)

async def call_openai_with_circuit_breaker(messages):
    if not cb_openai.can_execute():
        # Fallback: returner cached eller statisk respons
        return get_fallback_response(messages)

    try:
        response = await openai_client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            timeout=30
        )
        cb_openai.record_success()
        return response
    except Exception as e:
        cb_openai.record_failure()
        raise

Circuit Breaker i .NET med Polly

// C# med Polly for resilient Azure AI-kall
using Polly;
using Polly.CircuitBreaker;

var circuitBreakerPolicy = Policy
    .Handle<HttpRequestException>()
    .Or<TaskCanceledException>()
    .CircuitBreakerAsync(
        exceptionsAllowedBeforeBreaking: 5,
        durationOfBreak: TimeSpan.FromSeconds(30),
        onBreak: (ex, breakDuration) =>
            logger.LogWarning($"Circuit opened for {breakDuration.TotalSeconds}s: {ex.Message}"),
        onReset: () =>
            logger.LogInformation("Circuit closed, resuming normal operation"),
        onHalfOpen: () =>
            logger.LogInformation("Circuit half-open, testing with next request")
    );

var retryPolicy = Policy
    .Handle<HttpRequestException>()
    .WaitAndRetryAsync(
        retryCount: 3,
        sleepDurationProvider: retryAttempt =>
            TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)),
        onRetry: (ex, delay, retryCount, context) =>
            logger.LogWarning($"Retry {retryCount} after {delay.TotalSeconds}s: {ex.Message}")
    );

// Kombiner retry + circuit breaker
var resilientPolicy = Policy.WrapAsync(retryPolicy, circuitBreakerPolicy);

var result = await resilientPolicy.ExecuteAsync(async () =>
    await searchClient.SearchAsync<SearchDocument>(query)
);

Graceful degradation av AI-tjenester

Degraderingsstrategier

Feiltilstand	Degraderingsstrategi	Brukeropplevelse
Azure OpenAI nede	Returnér cached svar eller statiske meldinger	"Vi opplever tekniske problemer..."
AI Search nede	Fall tilbake til enklere tekstsøk	Redusert relevans, men funksjonelt
Embedding API nede	Bruk keyword-basert search	Ingen semantisk søk, men resultater
Alle AI-tjenester nede	Full graceful degradation	Manuell betjening eller køsystem

Implementering

# Graceful degradation for RAG-applikasjon
class ResilientRAGService:
    """RAG service with multiple fallback levels."""

    async def get_response(self, user_query: str) -> dict:
        """Try full RAG, then degrade gracefully."""

        # Level 1: Full RAG (AI Search + Azure OpenAI)
        try:
            context = await self._search_with_ai(user_query)
            response = await self._generate_with_openai(user_query, context)
            return {"level": "full", "response": response}
        except ServiceUnavailableError:
            pass

        # Level 2: Keyword search + Azure OpenAI
        try:
            context = await self._keyword_search(user_query)
            response = await self._generate_with_openai(user_query, context)
            return {"level": "degraded_search", "response": response}
        except ServiceUnavailableError:
            pass

        # Level 3: Cached/FAQ responses
        try:
            response = await self._get_cached_response(user_query)
            if response:
                return {"level": "cached", "response": response}
        except Exception:
            pass

        # Level 4: Static fallback
        return {
            "level": "fallback",
            "response": "Vi opplever tekniske problemer med vår AI-tjeneste. "
                       "Vennligst prøv igjen senere eller kontakt oss direkte."
        }

Private endepunkter og nettverksisolering

Private Link for AI-tjenester

# Opprett Private Endpoints for AI-tjenester i begge regioner

# Azure OpenAI Private Endpoint — Primær region
az network private-endpoint create \
  --name "pe-aoai-norwayeast" \
  --resource-group "rg-ai-prod" \
  --vnet-name "vnet-ai-norwayeast" \
  --subnet "snet-private-endpoints" \
  --private-connection-resource-id "/subscriptions/{sub}/resourceGroups/rg-ai-prod/providers/Microsoft.CognitiveServices/accounts/aoai-prod" \
  --group-ids "account" \
  --connection-name "aoai-primary"

# Azure OpenAI Private Endpoint — DR region
az network private-endpoint create \
  --name "pe-aoai-swedencentral" \
  --resource-group "rg-ai-dr" \
  --vnet-name "vnet-ai-swedencentral" \
  --subnet "snet-private-endpoints" \
  --private-connection-resource-id "/subscriptions/{sub}/resourceGroups/rg-ai-dr/providers/Microsoft.CognitiveServices/accounts/aoai-dr" \
  --group-ids "account" \
  --connection-name "aoai-secondary"

# AI Search Private Endpoint — Primær region
az network private-endpoint create \
  --name "pe-search-norwayeast" \
  --resource-group "rg-ai-prod" \
  --vnet-name "vnet-ai-norwayeast" \
  --subnet "snet-private-endpoints" \
  --private-connection-resource-id "/subscriptions/{sub}/resourceGroups/rg-ai-prod/providers/Microsoft.Search/searchServices/search-prod" \
  --group-ids "searchService" \
  --connection-name "search-primary"

VNet Peering mellom regioner

# VNet peering for cross-region kommunikasjon
az network vnet peering create \
  --name "peer-norwayeast-to-swedencentral" \
  --resource-group "rg-networking" \
  --vnet-name "vnet-ai-norwayeast" \
  --remote-vnet "/subscriptions/{sub}/resourceGroups/rg-networking/providers/Microsoft.Network/virtualNetworks/vnet-ai-swedencentral" \
  --allow-vnet-access true \
  --allow-forwarded-traffic true

az network vnet peering create \
  --name "peer-swedencentral-to-norwayeast" \
  --resource-group "rg-networking" \
  --vnet-name "vnet-ai-swedencentral" \
  --remote-vnet "/subscriptions/{sub}/resourceGroups/rg-networking/providers/Microsoft.Network/virtualNetworks/vnet-ai-norwayeast" \
  --allow-vnet-access true \
  --allow-forwarded-traffic true

DDoS-beskyttelse og trafikkfiltrering

Azure DDoS Protection

# Aktiver DDoS Network Protection
az network ddos-protection create \
  --name "ddos-ai-protection" \
  --resource-group "rg-networking" \
  --location "norwayeast"

# Koble til VNet
az network vnet update \
  --name "vnet-ai-norwayeast" \
  --resource-group "rg-networking" \
  --ddos-protection-plan "ddos-ai-protection"

NSG-regler for AI-tjenester

# Network Security Group for AI-subnet
az network nsg rule create \
  --resource-group "rg-networking" \
  --nsg-name "nsg-ai-app" \
  --name "AllowAzureOpenAI" \
  --priority 100 \
  --direction Outbound \
  --access Allow \
  --protocol Tcp \
  --destination-port-ranges 443 \
  --destination-address-prefixes "CognitiveServicesManagement" \
  --description "Allow outbound to Azure OpenAI"

az network nsg rule create \
  --resource-group "rg-networking" \
  --nsg-name "nsg-ai-app" \
  --name "AllowAzureSearch" \
  --priority 110 \
  --direction Outbound \
  --access Allow \
  --protocol Tcp \
  --destination-port-ranges 443 \
  --destination-address-prefixes "AzureCognitiveSearch" \
  --description "Allow outbound to Azure AI Search"

az network nsg rule create \
  --resource-group "rg-networking" \
  --nsg-name "nsg-ai-app" \
  --name "DenyAllOtherOutbound" \
  --priority 4000 \
  --direction Outbound \
  --access Deny \
  --protocol "*" \
  --destination-port-ranges "*" \
  --destination-address-prefixes "*" \
  --description "Deny all other outbound traffic"

Referanser

• Circuit Breaker pattern — Detaljert mønsterbeskrivelse
• Retry pattern — Retry-strategier
• Architecture design patterns that support reliability — WAF resiliensmønstre
• Transient fault handling — Best practices for transiente feil
• Azure DDoS Protection overview — DDoS-beskyttelse
• Azure Private Link overview — Private Endpoints

For Cosmo

• Bruk denne referansen når kunden designer nettverksarkitektur for resiliente AI-løsninger, eller når de implementerer failover med private endepunkter.
• Circuit Breaker + Retry med exponential backoff er sterkt anbefalt for Azure AI API-kall — Microsoft omtaler dette som beste praksis for resiliens (Azure Well-Architected), ikke et absolutt krav.
• For private endpoints: Husk at failover mellom regioner krever at Private DNS-soner er linket til begge VNets.
• Graceful degradation bør alltid designes i lag — full AI → enklere søk → cached svar → statisk fallback.
• Anbefal Azure Front Door (Premium) for AI-workloads som trenger global load balancing med DDoS-beskyttelse og WAF i ett produkt.