diff --git a/skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md b/skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md index 091b65a..86a491c 100644 --- a/skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md +++ b/skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md @@ -1,6 +1,6 @@ # Agent Evaluation and Testing Frameworks -**Last updated:** 2026-04 | Verified: MCP 2026-04 +**Last updated:** 2026-06-24 | Verified: MCP 2026-06 **Status:** GA (Azure AI Evaluation SDK), Preview (Agent-specific evaluators) **Category:** Agent Orchestration & Automation @@ -31,6 +31,15 @@ Microsoft Foundry støtter både Foundry Agent Service (built-in agents), Semant | **ContentSafetyEvaluator** | Detekterer harmful content (violence, hate, sexual, self-harm) | query, response | 0-7 severity | Ja (Azure AI Content Safety) | | **IndirectAttackEvaluator** | Sjekker jailbreak attempts via indirect injection | query, response | Pass/Fail | Ja | | **CodeVulnerabilityEvaluator** | Identifiserer usikker kode i agentsvar | response | Pass/Fail | Ja | +| **TaskCompletionEvaluator** (preview) | Måler om agenten fullførte oppgaven end-to-end med brukbar leveranse | query, response, (tool_definitions) | Pass/Fail | Ja | +| **CustomerSatisfactionEvaluator** (preview) | Predikerer brukertilfredshet over en samtale (6 dimensjoner) | messages | 1-5 Likert | Ja | +| **ToolSelectionEvaluator** | Måler om agenten valgte riktige (og kun nødvendige) tools | query, response, tool_definitions | Pass/Fail | Ja | +| **ToolInputAccuracyEvaluator** | Validerer at alle tool-parametere er korrekte (6 strenge kriterier) | query, response, tool_definitions | Pass/Fail | Ja | +| **ToolOutputUtilizationEvaluator** | Måler om agenten brukte tool-outputs riktig i svaret | query, response, tool_definitions | Pass/Fail | Ja | +| **ToolCallSuccessEvaluator** | Sjekker om tool-kall ble utført uten tekniske feil | response | Pass/Fail | Ja | +| **TaskNavigationEfficiencyEvaluator** | Sammenligner agentens steg mot optimal/forventet sti (krever ground truth) | actions, expected_actions | Pass/Fail | Nei | + +> **Microsoft Agent Framework — `FoundryEvals`:** I tillegg til `AIAgentConverter` kan en `AIAgent` kobles direkte til Foundrys evalueringstjeneste via `FoundryEvals` (kjører som standard relevance/coherence/task_adherence og legger automatisk til tool_call_accuracy når tool-definisjoner finnes; resultater vises i Foundry-portalen). For kompleks evaluering anbefaler Microsoft nå en sterk reasoning-modell som `gpt-4.1-mini` som judge for balanse mellom ytelse og kostnad. ### Evaluator output format