docs(architect): weekly KB update — 106 files refreshed (2026-04)
Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering, ms-ai-governance, ms-ai-security, ms-ai-infrastructure. Key changes: - Language Services (Custom Text Classification, Text Analytics, QnA): retirement warning 2029-03-31, migration guides to Foundry/GPT-4o - Agentic Retrieval: 50M free reasoning tokens/month (Public Preview) - Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models - Agent Registry: Risks column (M365 E7), user-shared/org-published types - Declarative agents: schema v1.5 → v1.6, Store validation requirements - MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code - AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python) - Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access - Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned - Fast Transcription API: phrase lists, 14-language multi-lingual transcription - Azure Monitor Workbooks: Bicep support, RBAC specifics - Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA) - RAG security-rbac: 4-approach table (GA + 3 preview access control methods) - IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns - Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP) All 106 files: Last updated 2026-04 | Verified: MCP 2026-04 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
0eb30fa853
commit
6645e93205
104 changed files with 1986 additions and 520 deletions
|
|
@ -4,6 +4,8 @@
|
|||
**Dato:** 2026-02-04
|
||||
**Confidence:** HIGH (basert på offisiell Microsoft-dokumentasjon)
|
||||
|
||||
**Verified:** MCP 2026-04
|
||||
|
||||
## Introduksjon
|
||||
|
||||
Feedback loops og kontinuerlig forbedring er kritiske komponenter i moderne AI-operasjoner. I motsetning til tradisjonell programvare, hvor funksjonalitet er deterministisk, kan AI-modeller vise kvalitetsdrift eller uventet oppførsel når de møter reelle data. Et velfungerende feedback-system sikrer at modeller forblir nøyaktige, relevante og trygge gjennom hele sin livssyklus.
|
||||
|
|
@ -738,3 +740,32 @@ Dette dokumentet dekker hele feedback loop-syklusen for både classical ML og Ge
|
|||
5. **Kostnad:** Threshold-based retraining kan spare 50-70% compute vs daily retraining
|
||||
|
||||
Bruk arkitekturmønstrene til å visualisere løsningen for kunden. Påpek at MLflow Tracing + Agent Evaluation gir "free" observability (built-in i Databricks).
|
||||
|
||||
|
||||
### MLflow 3 Evaluation & Feedback Loop (2026)
|
||||
|
||||
MLflow 3 introduces a unified evaluation-monitoring lifecycle for GenAI feedback loops:
|
||||
|
||||
**Iterative workflow**:
|
||||
1. **Trace** production requests (MLflow Tracing — end-to-end observability)
|
||||
2. **Evaluate** against scorers during development (`mlflow.genai.evaluate()`)
|
||||
3. **Monitor** production with same scorers (consistent quality measurement)
|
||||
4. **Gather human feedback** via Review App (expert annotations)
|
||||
5. **Improve** prompts/models based on evaluation datasets
|
||||
|
||||
**Azure ML Model Monitoring signals**:
|
||||
- Data quality: null values, out-of-range, type mismatch
|
||||
- Data drift: statistical distribution changes between training and production data
|
||||
- Prediction drift: distribution shift in model outputs
|
||||
- Feature attribution drift: changes in feature importance
|
||||
- Custom signals: user-defined metrics via custom scripts
|
||||
|
||||
**Monitoring setup**:
|
||||
```python
|
||||
# Set up out-of-box monitoring for Azure ML online endpoints
|
||||
# Monitors data drift, prediction drift automatically
|
||||
# Integrates with Azure Event Grid for alerting
|
||||
```
|
||||
|
||||
**Continuous improvement cycle**: Production traces → MLflow evaluation datasets → Scorer alignment → Prompt/model update → A/B test → Production rollout
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue