docs(architect): weekly KB update — 106 files refreshed (2026-04)

Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering,
ms-ai-governance, ms-ai-security, ms-ai-infrastructure.

Key changes:
- Language Services (Custom Text Classification, Text Analytics, QnA):
  retirement warning 2029-03-31, migration guides to Foundry/GPT-4o
- Agentic Retrieval: 50M free reasoning tokens/month (Public Preview)
- Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models
- Agent Registry: Risks column (M365 E7), user-shared/org-published types
- Declarative agents: schema v1.5 → v1.6, Store validation requirements
- MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code
- AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python)
- Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access
- Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned
- Fast Transcription API: phrase lists, 14-language multi-lingual transcription
- Azure Monitor Workbooks: Bicep support, RBAC specifics
- Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA)
- RAG security-rbac: 4-approach table (GA + 3 preview access control methods)
- IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns
- Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP)

All 106 files: Last updated 2026-04 | Verified: MCP 2026-04

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-10 09:13:24 +02:00
commit 6645e93205
104 changed files with 1986 additions and 520 deletions

View file

@ -1,6 +1,6 @@
# Agent Evaluation and Testing Frameworks
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Azure AI Evaluation SDK), Preview (Agent-specific evaluators)
**Category:** Agent Orchestration & Automation
@ -276,6 +276,33 @@ evaluator = TaskAdherenceEvaluator(
- **KQL queries:** Flexible querying av evaluation metrics over tid
- **Alerts:** Sett opp alerts hvis pass rate dropper under threshold
### MLflow 3 (Databricks / Cross-platform)
MLflow 3 tilbyr komprehensiv GenAI-evaluering for agenter paa tvers av plattformer:
| Feature | Beskrivelse |
|---------|-------------|
| **Built-in LLM judges** | Innebygde dommere for kvalitetsmetrikker (relevance, groundedness, safety, etc.) |
| **Custom scorers** | Definer egne kvalitetsmetrikker med Python-funksjoner |
| **Eval harness** | Test GenAI-app mot eval-datasett under utvikling; sammenlign appversjoner |
| **Conversation evaluation** | Vurder multi-turn samtalekvaltiet (completeness, user frustration, dialogue coherence) |
| **Conversation simulation** | Generer syntetiske multi-turn samtaler for testing |
| **Production monitoring** | Kjoer scorers og judges paa produksjons-traces automatisk (Beta) |
| **Review App** | Samle ekspertfeedback og bygg eval-datasett |
MLflow Tracing gir real-time trace logging gjennom hele livssyklusen. Samme judges og scorers kan brukes i baade development og produksjon — konsistent evaluering.
```python
# MLflow 3 evaluation eksempel
import mlflow
results = mlflow.genai.evaluate(
data=eval_dataset,
predict_fn=my_agent,
scorers=[mlflow.genai.scorers.groundedness(), mlflow.genai.scorers.safety()]
)
```
### Prompt Flow
- **Evaluation flows:** Custom evaluation logic som Prompt Flow (deprecated approach — bruk Azure AI Evaluation SDK i stedet)
@ -488,7 +515,7 @@ evaluator = TaskAdherenceEvaluator(
9. **Evaluate and monitor AI agents (MLflow 3 on Databricks)**
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/
*Confidence: Verified* — MLflow-based evaluation for cross-platform agents
*Confidence: Verified* — MLflow 3 GenAI evaluation: built-in LLM judges og scorers, eval-harness for development, production monitoring (Beta), conversation evaluation (multi-turn), conversation simulation, Review App for human feedback, Genie Code for observability; integrert med MLflow Tracing paa tvers av development/test/produksjon; oppdatert 2026-04
10. **Run automated tests for agent quality and reliability (Copilot Studio)**
https://learn.microsoft.com/en-us/power-platform/release-plan/2025wave1/microsoft-copilot-studio/run-automated-tests-agent-quality-reliability