docs(architect): weekly KB update — 106 files refreshed (2026-04)

Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering,
ms-ai-governance, ms-ai-security, ms-ai-infrastructure.

Key changes:
- Language Services (Custom Text Classification, Text Analytics, QnA):
  retirement warning 2029-03-31, migration guides to Foundry/GPT-4o
- Agentic Retrieval: 50M free reasoning tokens/month (Public Preview)
- Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models
- Agent Registry: Risks column (M365 E7), user-shared/org-published types
- Declarative agents: schema v1.5 → v1.6, Store validation requirements
- MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code
- AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python)
- Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access
- Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned
- Fast Transcription API: phrase lists, 14-language multi-lingual transcription
- Azure Monitor Workbooks: Bicep support, RBAC specifics
- Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA)
- RAG security-rbac: 4-approach table (GA + 3 preview access control methods)
- IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns
- Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP)

All 106 files: Last updated 2026-04 | Verified: MCP 2026-04

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

skills/ms-ai-engineering/references/mlops-genaiops/ab-testing-llm-applications.md

@@ -1,11 +1,14 @@
 # A/B Testing and Experimentation for AI Models
 
 **Last updated:** 2026-02
+**Verified:** MCP 2026-04
 **Status:** GA
 **Category:** MLOps & GenAIOps
 
 ---
 
+**Verified:** MCP 2026-04
+
 ## Introduksjon
 
 A/B-testing og eksperimentering er kritiske teknikker for å validere og optimalisere AI-modeller i produksjon. I motsetning til tradisjonell programvareutvikling, hvor funksjonalitet er binær (fungerer/fungerer ikke), er AI-modeller probabilistiske — ytelsen deres varierer med data, kontekst og bruksmønster. A/B-testing gjør det mulig å sammenligne modelversjoner, fine-tuning-strategier, prompt-varianter eller RAG-konfigurasjoner under reelle forhold, med ekte brukere og reell trafikk.
@@ -438,3 +441,35 @@ Krever:
 - Shadow deployment patterns
 
 **Antall unike kilder:** 7 (Microsoft Learn) + 3 (baseline concepts) = **10 kilder**
+
+
+### A/B Testing with Azure ML Managed Online Endpoints + MLflow 3 (2026)
+
+**Traffic splitting via managed online endpoints**:
+```bash
+# Deploy challenger model with 10% traffic
+az ml online-deployment create --name challenger --endpoint my-endpoint
+az ml online-endpoint update --name my-endpoint --traffic control=90 challenger=10
+
+# Monitor with MLflow 3 scorers — same metrics for both variants
+# Use RelevanceToQuery, Correctness, custom business scorers
+```
+
+**MLflow 3 A/B evaluation pattern**:
+- Use `mlflow.genai.evaluate()` on traces from each variant
+- Compare scorers: `Correctness`, `RelevanceToQuery`, `ToolCallEfficiency`
+- Statistical significance: MLflow tracks Cohen's Kappa against human baseline
+- Aliases in Prompt Registry: `@control` and `@challenger` for prompt A/B testing
+
+**Azure ML safe rollout progression**:
+1. **Shadow testing**: Mirror X% of traffic to new model (no user impact)
+2. **Canary**: Route 10% live traffic, monitor bake time (hours/days)
+3. **Progressive**: 10% → 50% → 100% with health gate at each step
+4. **Rollback trigger**: Automatic halt on health signal degradation
+
+**Evaluation metrics for LLM A/B tests**:
+- Quality: Groundedness, Relevance, Correctness (MLflow judges)
+- Latency: P50, P90, P99 response times
+- Cost: Token usage per request
+- Business: Task completion rate, user satisfaction
+