docs(architect): weekly KB update — 106 files refreshed (2026-04)
Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering, ms-ai-governance, ms-ai-security, ms-ai-infrastructure. Key changes: - Language Services (Custom Text Classification, Text Analytics, QnA): retirement warning 2029-03-31, migration guides to Foundry/GPT-4o - Agentic Retrieval: 50M free reasoning tokens/month (Public Preview) - Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models - Agent Registry: Risks column (M365 E7), user-shared/org-published types - Declarative agents: schema v1.5 → v1.6, Store validation requirements - MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code - AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python) - Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access - Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned - Fast Transcription API: phrase lists, 14-language multi-lingual transcription - Azure Monitor Workbooks: Bicep support, RBAC specifics - Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA) - RAG security-rbac: 4-approach table (GA + 3 preview access control methods) - IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns - Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP) All 106 files: Last updated 2026-04 | Verified: MCP 2026-04 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
0eb30fa853
commit
6645e93205
104 changed files with 1986 additions and 520 deletions
|
|
@ -1,11 +1,14 @@
|
|||
# Model Drift and Performance Degradation Detection
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Verified:** MCP 2026-04
|
||||
**Status:** GA
|
||||
**Category:** MLOps & GenAIOps
|
||||
|
||||
---
|
||||
|
||||
**Verified:** MCP 2026-04
|
||||
|
||||
## Introduksjon
|
||||
|
||||
Model drift og performance degradation er kritiske fenomener som oppstår når en maskinlæringsmodells ytelse forverres over tid i produksjon. Dette skjer fordi virkeligheten endrer seg – input-data får andre distribusjoner, forretningslogikk endres, sensorer kalibreres feil, eller brukernes atferd endrer seg. Uten kontinuerlig overvåking kan modeller raskt bli utdaterte og levere feil prediksjoner som undergraver forretningsmål eller skaper compliance-problemer i regulerte sektorer.
|
||||
|
|
@ -633,3 +636,39 @@ Email Alerts + Azure Monitor Dashboard
|
|||
### Sist oppdatert
|
||||
|
||||
**2026-02** – Basert på Microsoft Learn-dokumentasjon (azure-ai-ml SDK v2, API version 2).
|
||||
|
||||
|
||||
### Azure ML Model Drift & Performance Degradation Monitoring (2026)
|
||||
|
||||
**Model monitoring** provides continuous tracking of production model performance:
|
||||
|
||||
**Degradation signals**:
|
||||
- **Prediction drift**: Output distribution shifts away from training baseline
|
||||
- **Feature attribution drift**: Feature importance changes indicate concept drift
|
||||
- **Data quality degradation**: Input data quality issues upstream
|
||||
- **Performance metric degradation**: Track against ground truth when labels available
|
||||
|
||||
**Monitoring configuration**:
|
||||
```python
|
||||
# Set up monitoring for deployed models on online endpoints
|
||||
# Azure ML handles data collection and signal computation
|
||||
# Monitoring jobs run on schedule (default: daily)
|
||||
```
|
||||
|
||||
**Alert thresholds** (recommended):
|
||||
- Data drift coefficient > 0.1: Investigate
|
||||
- Data drift coefficient > 0.3: Retrain trigger
|
||||
- Prediction drift > 15%: Production alert
|
||||
- Unusable nodes > 0: Infrastructure alert (Azure Monitor)
|
||||
|
||||
**Continuous learning loop**:
|
||||
1. Monitor signals → detect drift early
|
||||
2. Critically evaluate inherent model risks
|
||||
3. Identify hidden problems before business impact
|
||||
4. Trigger retraining or model update workflow
|
||||
5. Validate new model before rollout (blue-green/canary)
|
||||
|
||||
**Integration**: Azure Event Grid for alerting → Logic Apps / Functions → automated retraining trigger
|
||||
|
||||
**For GenAI/LLM**: MLflow 3 production monitoring reuses development scorers (Groundedness, Relevance) on production traces — consistent quality measurement throughout lifecycle.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue