docs(architect): weekly KB update — 52 files refreshed (2026-04)

Key content changes:
- MLOps: MLflow 3 scorers expanded (RetrievalRelevance, Fluency, multi-turn judges)
- MLflow 3 A/B eval: mirror_traffic GA confirmed, new scorer catalog
- CI/CD: OIDC auth replaces deprecated --sdk-auth (Azure ML GitHub Actions)
- Agent framework A2A: updated SDK patterns (A2ACardResolver, BearerAuth)
- AG-UI backend tool rendering: accurate TOOL_CALL_* event shapes
- Computer Use agents: US region requirement, credentials patterns
- Purview governance: bulk term edit, expire/delete workflows
- CAF AI Secure: 3-phase structure confirmed current
- Copilot Studio: Claude Sonnet 4.5/4.6 GA, new orchestration controls
- M365 manifest: v1.26 GA (April 2026), copilotAgents node
- Power Platform: agent flow capacity enforcement corrected
- Azure Monitor: Simple Log Alerts GA, AMBA for policy-based alerting
- Security Copilot: SCU capacity model (400 SCU/1000 users)
- EU Data Boundary: all EU + EFTA countries confirmed
- gateway-multi-backend: added 4th topology, subscription-level quota note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

skills/ms-ai-engineering/references/mlops-genaiops/feedback-loops-continuous-improvement.md

@@ -2,6 +2,7 @@
 
 **Kategori:** MLOps & GenAIOps
 **Dato:** 2026-02-04
+**Last updated:** 2026-04
 **Confidence:** HIGH (basert på offisiell Microsoft-dokumentasjon)
 
 **Verified:** MCP 2026-04
@@ -711,7 +712,7 @@ mlflow.log_param("user_id_hash", user_id_hash)  # Logged
 
 **Primærkilder (Microsoft Learn):**
 
-1. [MLflow for GenAI Apps and Agents - Continuous Improvement Cycle](https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/overview/)
+1. [MLflow for GenAI Apps and Agents - Continuous Improvement Cycle](https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/overview/) (Verified MCP 2026-04 — updated 10-step cycle; new: Trace UI for pattern identification, evaluation harness, version/prompt management tracking)
 2. [Machine Learning Operations v2 - Monitoring & Feedback](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/machine-learning-operations-v2)
 3. [Generative AI App Developer Workflow - Production Monitoring](https://learn.microsoft.com/en-us/azure/databricks/generative-ai/tutorials/ai-cookbook/genai-developer-workflow)
 4. [Azure AI Foundry - Observability in Generative AI](https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/observability)
@@ -720,10 +721,10 @@ mlflow.log_param("user_id_hash", user_id_hash)  # Logged
 
 **Code samples:**
 - MLflow feedback logging: [Azure Databricks - Agent Framework](https://learn.microsoft.com/en-us/azure/databricks/generative-ai/agent-framework/non-conversational-agents#log-user-feedback)
-- Model monitoring setup: [Azure ML - Monitor Model Performance](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2)
-- GenAI evaluation: [MLflow 3.x - Evaluate App](https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/evaluate-app)
+- Model monitoring setup: [Azure ML - Monitor Model Performance](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2) (Verified MCP 2026-04 — supports data quality, data drift, prediction drift, feature attribution drift, and custom signals; integrates with Azure Event Grid for alerting)
+- GenAI evaluation: [MLflow 3.x - Evaluate App](https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/evaluate-app) (Verified MCP 2026-04 — tutorial covers RAG email app evaluation; new scorers: RetrievalGroundedness, Guidelines, RelevanceToQuery, Safety; version comparison with mlflow.genai.evaluate())
 
-**Dato for siste verifikasjon:** 2026-02-04
+**Dato for siste verifikasjon:** 2026-04-10
 
 **MCP calls:** 6 (microsoft_docs_search: 3, microsoft_docs_fetch: 3, microsoft_code_sample_search: 2)
 
@@ -742,7 +743,7 @@ Dette dokumentet dekker hele feedback loop-syklusen for både classical ML og Ge
 Bruk arkitekturmønstrene til å visualisere løsningen for kunden. Påpek at MLflow Tracing + Agent Evaluation gir "free" observability (built-in i Databricks).
 
 
-### MLflow 3 Evaluation & Feedback Loop (2026)
+### MLflow 3 Evaluation & Feedback Loop (Verified MCP 2026-04)
 
 MLflow 3 introduces a unified evaluation-monitoring lifecycle for GenAI feedback loops:
 
@@ -753,19 +754,27 @@ MLflow 3 introduces a unified evaluation-monitoring lifecycle for GenAI feedback
 4. **Gather human feedback** via Review App (expert annotations)
 5. **Improve** prompts/models based on evaluation datasets
 
+**Built-in LLM judges (scorers)**:
+- `RetrievalGroundedness` — checks if response is grounded in retrieved data
+- `RelevanceToQuery` — checks if response addresses the user request
+- `Safety` — checks for harmful/inappropriate content
+- `Guidelines(name, guidelines)` — custom policy/tone/style checks
+- `Correctness` — factual correctness with expected_facts
+
 **Azure ML Model Monitoring signals**:
 - Data quality: null values, out-of-range, type mismatch
 - Data drift: statistical distribution changes between training and production data
 - Prediction drift: distribution shift in model outputs
 - Feature attribution drift: changes in feature importance
 - Custom signals: user-defined metrics via custom scripts
+- Integrates with **Azure Event Grid** for alerting on threshold breaches
 
-**Monitoring setup**:
-```python
-# Set up out-of-box monitoring for Azure ML online endpoints
-# Monitors data drift, prediction drift automatically
-# Integrates with Azure Event Grid for alerting
-```
+**Evaluation dataset workflow (new 2026-04)**:
+1. Search production traces → select problematic + high-quality examples
+2. Save to versioned eval dataset in Unity Catalog (`mlflow.genai.datasets.create_dataset()`)
+3. Run evaluation harness with `mlflow.genai.evaluate(data=eval_dataset, predict_fn=..., scorers=...)`
+4. Compare runs in UI (`Evaluation runs` view) or SDK (`mlflow.search_runs`)
+5. Identify regressions per-metric before promoting new versions
 
 **Continuous improvement cycle**: Production traces → MLflow evaluation datasets → Scorer alignment → Prompt/model update → A/B test → Production rollout