feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command
Add /ultraresearch-local for structured research combining local codebase analysis with external knowledge via parallel agent swarms. Produces research briefs with triangulation, confidence ratings, and source quality assessment. New command: /ultraresearch-local with modes --quick, --local, --external, --fg. New agents: research-orchestrator (opus), docs-researcher, community-researcher, security-researcher, contrarian-researcher, gemini-bridge (all sonnet). New template: research-brief-template.md. Integration: --research flag in /ultraplan-local accepts pre-built research briefs (up to 3), enriches the interview and exploration phases. Planning orchestrator cross-references brief findings during synthesis. Design principle: Context Engineering — right information to right agent at right time. Research briefs are structured artifacts in the pipeline: ultraresearch → brief → ultraplan --research → plan → ultraexecute. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen
commit
baa2d0220b
488 changed files with 213221 additions and 0 deletions
|
|
@ -0,0 +1,516 @@
|
|||
# Agent Evaluation and Testing Frameworks
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Status:** GA (Azure AI Evaluation SDK), Preview (Agent-specific evaluators)
|
||||
**Category:** Agent Orchestration & Automation
|
||||
|
||||
---
|
||||
|
||||
## Introduksjon
|
||||
|
||||
Agent-baserte AI-systemer representerer en ny kompleksitet i testing og validering sammenlignet med tradisjonelle deterministic workflows. Der en enkel LLM-applikasjon kun har én inngangspunkt og ett svar, har agenter multippel tool-calling, dynamisk reasoning, multi-turn samtaler, og ikke-deterministisk oppførsel. Microsoft tilbyr et komprehensivt evalueringsrammeverk gjennom Azure AI Evaluation SDK og Azure AI Foundry som håndterer både pre-deployment testing (batch evaluation) og post-deployment monitoring (continuous evaluation).
|
||||
|
||||
Evalueringsrammeverket støtter tre hovedtyper testing: **System Evaluation** (helhetsoppførsel til agenten), **Process Evaluation** (kvalitet på tool calls og reasoning steps), og **Safety Evaluation** (content safety, jailbreak-resistance, bias). Alle evaluators opererer som LLM judges (typisk GPT-4.1 eller o-series reasoning models) som gir både scores, pass/fail labels, og reasoning explanations.
|
||||
|
||||
Azure AI Foundry støtter både Foundry Agent Service (built-in agents), Semantic Kernel agents, og custom agents via OpenAI-style message schema. Evaluering kan kjøres lokalt på utviklermaskinen, i cloud for CI/CD-integrasjon, eller kontinuerlig i produksjon med sampling rates og Azure Monitor Application Insights-integrasjon.
|
||||
|
||||
## Kjernekomponenter
|
||||
|
||||
### Evaluator-typer
|
||||
|
||||
| Evaluator | Formål | Input | Score range | LLM Judge? |
|
||||
|-----------|--------|-------|-------------|-----------|
|
||||
| **IntentResolutionEvaluator** | Måler om agenten identifiserer brukerens intent korrekt | query, response, (tool_definitions optional) | 1-5 Likert | Ja (GPT-4.1 / o-series) |
|
||||
| **TaskAdherenceEvaluator** | Sjekker om agentens svar følger system message og prior steps | query, response, (tool_calls optional) | 1-5 Likert | Ja |
|
||||
| **ToolCallAccuracyEvaluator** | Validerer at agenten kaller riktige tools med riktige parameters | query, tool_definitions, (response/tool_calls) | 1-5 Likert | Ja |
|
||||
| **ResponseCompletenessEvaluator** | Evaluerer om svar er komplett og dekker alle deler av query | query, response | 1-5 Likert | Ja |
|
||||
| **GroundednessEvaluator** | Måler om agentsvar er forankret i tool outputs (ikke hallusinert) | query, response, tool_definitions | 1-5 Likert | Ja |
|
||||
| **RelevanceEvaluator** | Sjekker om svar er relevant for query | query, response | 1-5 Likert | Ja |
|
||||
| **CoherenceEvaluator** | Evaluerer logisk sammenheng i svar | query, response | 1-5 Likert | Ja |
|
||||
| **FluencyEvaluator** | Måler språklig kvalitet og grammatikk | query, response | 1-5 Likert | Ja |
|
||||
| **ContentSafetyEvaluator** | Detekterer harmful content (violence, hate, sexual, self-harm) | query, response | 0-7 severity | Ja (Azure AI Content Safety) |
|
||||
| **IndirectAttackEvaluator** | Sjekker jailbreak attempts via indirect injection | query, response | Pass/Fail | Ja |
|
||||
| **CodeVulnerabilityEvaluator** | Identifiserer usikker kode i agentsvar | response | Pass/Fail | Ja |
|
||||
|
||||
### Evaluator output format
|
||||
|
||||
Alle evaluators returnerer standardisert JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"{metric_name}": 4.0, // Score (1-5, 0-7, 0-1 avhengig av type)
|
||||
"{metric_name}_result": "pass", // Pass/fail basert på threshold
|
||||
"{metric_name}_threshold": 3, // Binarization threshold (default eller user-defined)
|
||||
"{metric_name}_reason": "The agent correctly...", // LLM judge reasoning
|
||||
"details": { ... } // Optional debug info (f.eks. tool call breakdown)
|
||||
}
|
||||
```
|
||||
|
||||
### Supported agent frameworks
|
||||
|
||||
| Framework | Converter support? | Evaluators |
|
||||
|-----------|-------------------|-----------|
|
||||
| **Foundry Agent Service** | Ja (`AIAgentConverter`) | Alle |
|
||||
| **Semantic Kernel** | Ja (`AIAgentConverter`) | Alle |
|
||||
| **Custom agents** | Nei (bruk OpenAI-style message schema) | Alle (krever manuell parsing) |
|
||||
|
||||
### Tool call evaluation support
|
||||
|
||||
`ToolCallAccuracyEvaluator` støtter disse tool-typene i Foundry Agent Service:
|
||||
|
||||
1. File Search
|
||||
2. Azure AI Search
|
||||
3. Bing Grounding
|
||||
4. Bing Custom Search
|
||||
5. SharePoint Grounding
|
||||
6. Code Interpreter
|
||||
7. Fabric Data Agent
|
||||
8. OpenAPI
|
||||
9. Function Tool (user-defined)
|
||||
|
||||
**Viktig:** Custom tools utenfor denne listen må wrappes som Function Tools for å evalueres.
|
||||
|
||||
## Arkitekturmønstre
|
||||
|
||||
### 1. Pre-deployment batch evaluation (Cloud Evaluation)
|
||||
|
||||
**Bruk:** Test agenten mot et større dataset før deploy (100-1000+ test cases).
|
||||
|
||||
**Fordeler:**
|
||||
- Ingen local compute-krav (kjører i Azure)
|
||||
- CI/CD-integrasjon via Azure AI Projects SDK
|
||||
- Resultat logges i Foundry portal med trace-debugger
|
||||
- Supports både custom evaluators og built-in
|
||||
|
||||
**Ulemper:**
|
||||
- Koster Azure OpenAI tokens (evaluator LLM calls)
|
||||
- Krever Azure AI Foundry project setup
|
||||
|
||||
**Eksempel (Python):**
|
||||
|
||||
```python
|
||||
from azure.ai.evaluation import evaluate
|
||||
from azure.ai.evaluation import IntentResolutionEvaluator, TaskAdherenceEvaluator
|
||||
|
||||
# Initialize evaluators with reasoning model for complex tasks
|
||||
quality_evaluators = {
|
||||
"IntentResolutionEvaluator": IntentResolutionEvaluator(
|
||||
model_config=reasoning_model_config,
|
||||
is_reasoning_model=True
|
||||
),
|
||||
"TaskAdherenceEvaluator": TaskAdherenceEvaluator(
|
||||
model_config=reasoning_model_config,
|
||||
is_reasoning_model=True
|
||||
),
|
||||
}
|
||||
|
||||
# Batch evaluate with converter support
|
||||
converter = AIAgentConverter(project_client)
|
||||
filename = "evaluation_input_data.jsonl"
|
||||
converter.prepare_evaluation_data(thread_ids=[thread1_id, thread2_id], filename=filename)
|
||||
|
||||
response = evaluate(
|
||||
data=filename,
|
||||
evaluation_name="agent-qa-regression",
|
||||
evaluators=quality_evaluators,
|
||||
azure_ai_project=os.environ["AZURE_AI_PROJECT"]
|
||||
)
|
||||
|
||||
print(response["metrics"]) # Averaged scores
|
||||
print(response["studio_url"]) # Foundry portal link
|
||||
```
|
||||
|
||||
### 2. Continuous evaluation (Production Monitoring)
|
||||
|
||||
**Bruk:** Automatisk evaluering av agent-interaksjoner i produksjon med sampling.
|
||||
|
||||
**Fordeler:**
|
||||
- Near real-time observability i Azure Monitor
|
||||
- Sampling configuration (0-100%, max 1000/hour)
|
||||
- Kobles til traces for debugging
|
||||
- Integration med Foundry Observability dashboard
|
||||
|
||||
**Ulemper:**
|
||||
- Krever Application Insights oppsett
|
||||
- Cost overhead (evaluator LLM calls + Application Insights storage)
|
||||
- Reasoning explanations kan inneholde sensitiv data (må redact via `redact_score_properties=True`)
|
||||
|
||||
**Eksempel (Python):**
|
||||
|
||||
```python
|
||||
from azure.ai.projects.models import AgentEvaluationRequest, EvaluatorIds
|
||||
|
||||
# Define evaluators for continuous monitoring
|
||||
evaluators = {
|
||||
"Relevance": {"Id": EvaluatorIds.Relevance.value},
|
||||
"Fluency": {"Id": EvaluatorIds.Fluency.value},
|
||||
"ContentSafety": {"Id": EvaluatorIds.ContentSafety.value}
|
||||
}
|
||||
|
||||
# Submit continuous evaluation after each agent run
|
||||
project_client.evaluation.create_agent_evaluation(
|
||||
AgentEvaluationRequest(
|
||||
thread=thread.id,
|
||||
run=run.id,
|
||||
evaluators=evaluators,
|
||||
samplingConfiguration=AgentEvaluationSamplingConfiguration(
|
||||
name=agent.id,
|
||||
samplingPercent=100, # 100% of runs
|
||||
maxRequestRate=250 # Max 250 evals/hour
|
||||
),
|
||||
appInsightsConnectionString=project_client.telemetry.get_application_insights_connection_string()
|
||||
)
|
||||
)
|
||||
|
||||
# Query results from Application Insights (KQL)
|
||||
query = f"""
|
||||
traces
|
||||
| where message == "gen_ai.evaluation.result"
|
||||
| where customDimensions["gen_ai.thread.run.id"] == "{run.id}"
|
||||
"""
|
||||
```
|
||||
|
||||
### 3. Local evaluation (Development Testing)
|
||||
|
||||
**Bruk:** Rask testing under utvikling (1-10 test cases).
|
||||
|
||||
**Fordeler:**
|
||||
- Umiddelbar feedback loop
|
||||
- Lavere cost (færre test cases)
|
||||
- Ingen cloud dependency
|
||||
|
||||
**Ulemper:**
|
||||
- Ikke skalerbar til store datasets
|
||||
- Local compute-krav
|
||||
- Manuelt resultat-håndtering
|
||||
|
||||
**Eksempel (Python):**
|
||||
|
||||
```python
|
||||
from azure.ai.evaluation import IntentResolutionEvaluator
|
||||
|
||||
evaluator = IntentResolutionEvaluator(model_config)
|
||||
|
||||
# Evaluate single agent run
|
||||
result = evaluator(
|
||||
query="What is the weather in Seattle?",
|
||||
response="The current weather in Seattle is Sunny, 25°C."
|
||||
)
|
||||
|
||||
print(result["intent_resolution"]) # 5.0
|
||||
print(result["intent_resolution_result"]) # "pass"
|
||||
print(result["intent_resolution_reason"]) # LLM explanation
|
||||
```
|
||||
|
||||
## Beslutningsveiledning
|
||||
|
||||
### Når bruke hvilken evalueringstype?
|
||||
|
||||
| Scenario | Anbefalt type | Evaluators | Frequency |
|
||||
|----------|---------------|-----------|-----------|
|
||||
| **Prototype-fase (1-10 test cases)** | Local evaluation | IntentResolution, TaskAdherence | Ad-hoc testing |
|
||||
| **Pre-deployment (100+ test cases)** | Cloud batch evaluation | Alle quality + safety evaluators | Før hver release |
|
||||
| **CI/CD pipeline** | Cloud batch evaluation | Subset (fast evaluators: Relevance, Coherence) | Hver PR |
|
||||
| **Production monitoring** | Continuous evaluation | ContentSafety, IntentResolution, TaskAdherence | 10-50% sampling |
|
||||
| **Red teaming validation** | Local + Cloud | IndirectAttack, CodeVulnerability, ContentSafety | Før initial deploy + quarterly |
|
||||
|
||||
### Model selection for LLM judges
|
||||
|
||||
| Judge model | Use case | Cost | Reasoning quality |
|
||||
|-------------|----------|------|-------------------|
|
||||
| **gpt-4o** | Standard evaluation (Coherence, Fluency, Relevance) | Moderat | God |
|
||||
| **gpt-4.1** | Standard evaluation med bedre reasoning | Høyere | Bedre |
|
||||
| **o3-mini / o-series** | Kompleks evaluation (TaskAdherence, ToolCallAccuracy) | Høyest | Best (chain-of-thought) |
|
||||
|
||||
**Konfigurasjon:**
|
||||
|
||||
```python
|
||||
reasoning_model_config = {
|
||||
"azure_deployment": "o3-mini",
|
||||
"api_key": os.getenv("AZURE_API_KEY"),
|
||||
"azure_endpoint": os.getenv("AZURE_ENDPOINT"),
|
||||
"api_version": "2024-08-01-preview",
|
||||
}
|
||||
|
||||
evaluator = TaskAdherenceEvaluator(
|
||||
model_config=reasoning_model_config,
|
||||
is_reasoning_model=True # Aktiverer extended thinking budget
|
||||
)
|
||||
```
|
||||
|
||||
### Vanlige feil
|
||||
|
||||
| Feil | Symptom | Løsning |
|
||||
|------|---------|---------|
|
||||
| **Missing system message** | Evaluator warning: "Cannot parse query" | Alltid inkluder system message som første melding i `query` |
|
||||
| **Tool call schema mismatch** | ToolCallAccuracyEvaluator scorer lavt uten grunn | Sjekk at tool_definitions matcher faktisk tool signature |
|
||||
| **Evaluator cost explosion** | Uventet høy Azure OpenAI-faktura | Reduser sampling rate i continuous eval, bruk billigere judge model (gpt-4o > o3-mini) |
|
||||
| **Thread ID collision** | Feil evalueringsresultater | Bruk unique thread IDs, ikke gjenbruk threads |
|
||||
| **Non-supported tool types** | ToolCallAccuracyEvaluator returnerer "pass" med "unsupported tool" reason | Wrap custom tools som Function Tools |
|
||||
|
||||
### Røde flagg
|
||||
|
||||
- **Pass rate < 60% for IntentResolution:** Agent forstår ikke user intents — revurder system message eller few-shot examples
|
||||
- **ToolCallAccuracy score < 3:** Agent caller feil tools — vurder tydeligere tool descriptions eller færre tools
|
||||
- **TaskAdherence score < 3:** Agent ignorerer instruksjoner — sjekk system message, eller agenten har for mange tools (tool confusion)
|
||||
- **ContentSafety violations > 1%:** Agenten genererer harmful content — implementer content filters, revurder system instructions
|
||||
- **GroundednessEvaluator score < 4:** Agent hallusinerer — sjekk at tool outputs brukes korrekt, vurder RAG-forbedringer
|
||||
|
||||
## Integrasjon med Microsoft-stakken
|
||||
|
||||
### Azure AI Foundry
|
||||
|
||||
- **Evaluation wizard (UI):** No-code batch evaluation med built-in evaluators
|
||||
- **Trace debugger:** Step-by-step agent execution trace koblet til evaluation scores
|
||||
- **Evaluation library:** Lagre custom evaluators som reusable assets
|
||||
- **Comparison view:** Sammenlign flere evaluation runs (A/B testing)
|
||||
|
||||
### Foundry Agent Service
|
||||
|
||||
- **Auto-converter:** `AIAgentConverter` transformerer Foundry agent threads til evaluation data automatisk
|
||||
- **Tool call tracking:** Built-in logging av alle tool invocations for ToolCallAccuracyEvaluator
|
||||
|
||||
### Azure Monitor + Application Insights
|
||||
|
||||
- **Continuous evaluation storage:** Alle eval results logges som traces
|
||||
- **KQL queries:** Flexible querying av evaluation metrics over tid
|
||||
- **Alerts:** Sett opp alerts hvis pass rate dropper under threshold
|
||||
|
||||
### Prompt Flow
|
||||
|
||||
- **Evaluation flows:** Custom evaluation logic som Prompt Flow (deprecated approach — bruk Azure AI Evaluation SDK i stedet)
|
||||
- **Batch run evaluation:** Kjør evaluation som Prompt Flow batch run
|
||||
|
||||
### Semantic Kernel
|
||||
|
||||
- **Converter support:** `AIAgentConverter` støtter Semantic Kernel agents direkte
|
||||
- **Plugin evaluation:** Evaluer Semantic Kernel plugins som tools
|
||||
|
||||
## Offentlig sektor (Norge)
|
||||
|
||||
### GDPR og databehandling
|
||||
|
||||
**Risiko:** Evaluators sender conversation data til Azure OpenAI judge models (kan inneholde persondata).
|
||||
|
||||
**Mitigering:**
|
||||
- **Anonymisering:** Fjern PII fra test datasets før evaluation
|
||||
- **Redaction configuration:** Bruk `redact_score_properties=True` i continuous evaluation for å hindre reasoning explanations med sensitiv data
|
||||
- **Data residency:** Sørg for at judge model (Azure OpenAI deployment) er i EU-region
|
||||
|
||||
### Forvaltningsloven § 11a (automatiserte enkeltvedtak)
|
||||
|
||||
**Risiko:** Hvis agenten fatter enkeltvedtak, må evaluering dokumentere at systemet oppfyller kvalitetskrav.
|
||||
|
||||
**Mitigering:**
|
||||
- **Batch evaluation før deploy:** Dokumentér pass rate for TaskAdherence, IntentResolution (min. 80% i kritiske use cases)
|
||||
- **Continuous monitoring:** Løpende overvåking av agent performance i produksjon med alerts ved degradering
|
||||
- **Human-in-the-loop:** Ved vedtak: kombiner agent-forslag med manual review, log evaluation scores i vedtakssystemet
|
||||
|
||||
### AI Act (High-risk AI systems)
|
||||
|
||||
**Risiko:** Agenter i kritiske domener (helse, politi, offentlige ytelser) klassifiseres som high-risk → krav til testing og dokumentasjon.
|
||||
|
||||
**Mitigering:**
|
||||
- **Test dataset representativitet:** Sørg for at evaluation dataset dekker alle demografiske grupper (bias testing)
|
||||
- **Adversarial testing:** Bruk `IndirectAttackEvaluator` for jailbreak testing, `ContentSafetyEvaluator` for harmful content
|
||||
- **Evaluation audit trail:** Lagre alle evaluation runs i Foundry med timestamp, versioning, og results (compliance dokumentasjon)
|
||||
|
||||
### Schrems II
|
||||
|
||||
**Risiko:** Evaluation data sendes til Azure OpenAI i US-region (data transfer issue).
|
||||
|
||||
**Mitigering:**
|
||||
- **EU-based judge models:** Deploy Azure OpenAI judge model (gpt-4.1) i EU-region (France Central, Sweden Central)
|
||||
- **On-prem evaluation:** Vurder local evaluation for svært sensitive use cases (men mistet CI/CD-integrasjon)
|
||||
|
||||
## Kostnad og lisensiering
|
||||
|
||||
### Prismodell
|
||||
|
||||
| Komponent | Pricing model | Estimert cost (per 1000 evals) |
|
||||
|-----------|---------------|--------------------------------|
|
||||
| **Azure AI Evaluation SDK** | Gratis (open-source) | 0 NOK |
|
||||
| **Azure OpenAI judge model (gpt-4o)** | Pay-per-token (input + output) | ~200-500 NOK (avhengig av conversation length) |
|
||||
| **Azure OpenAI judge model (o3-mini)** | Pay-per-token + reasoning tokens | ~500-1200 NOK (høyere pga. extended thinking) |
|
||||
| **Application Insights** | Data ingestion + retention | ~50-100 NOK/måned (1M traces) |
|
||||
| **Foundry storage** | Evaluation results + traces | Inkludert i Azure AI Foundry project (ingen ekstra cost) |
|
||||
|
||||
### Cost optimization tips
|
||||
|
||||
1. **Reducer sampling rate i continuous eval:**
|
||||
- Development: 10-20% sampling
|
||||
- Production: 5-10% sampling (høyere for kritiske agenter)
|
||||
|
||||
2. **Velg billigere judge model for simple evaluators:**
|
||||
- Coherence, Fluency, Relevance → gpt-4o (ikke o-series)
|
||||
- TaskAdherence, ToolCallAccuracy → o3-mini (krever reasoning)
|
||||
|
||||
3. **Reduser conversation length i evaluation data:**
|
||||
- Inkluder kun siste 3-5 turns i `query` (ikke hele thread history)
|
||||
|
||||
4. **Batch evaluation i stedet for continuous:**
|
||||
- Pre-deployment testing: batch eval (1x før release)
|
||||
- Production: sample 5-10%, ikke 100%
|
||||
|
||||
5. **Reuse eval datasets:**
|
||||
- Lagre golden datasets i Foundry, ikke regenerer hver gang
|
||||
|
||||
### Lisensiering
|
||||
|
||||
| Komponent | Lisens | Krav |
|
||||
|-----------|--------|------|
|
||||
| **Azure AI Evaluation SDK** | MIT License (open-source) | Ingen |
|
||||
| **Azure AI Foundry** | Inkludert i Azure subscription | Azure subscription |
|
||||
| **Azure OpenAI** | Pay-as-you-go (per token) | Azure OpenAI access (申请 required) |
|
||||
| **Application Insights** | Pay-as-you-go (per GB ingested) | Azure subscription |
|
||||
|
||||
## For arkitekten (Cosmo)
|
||||
|
||||
### Spørsmål å stille under arkitekturgjennomgang
|
||||
|
||||
1. **Evaluation strategy:**
|
||||
- "Hvilken type evaluation kjører du? (local, batch, continuous)?"
|
||||
- "Hvor ofte evaluerer du agenten? (per PR, pre-deploy, kontinuerlig)?"
|
||||
- "Har dere golden dataset for regression testing?"
|
||||
|
||||
2. **Evaluator selection:**
|
||||
- "Hvilke evaluators bruker du? (quality, safety, custom)?"
|
||||
- "Bruker du reasoning models (o-series) som judges for komplekse evaluators?"
|
||||
- "Hvordan håndterer du tool call evaluation?"
|
||||
|
||||
3. **Cost management:**
|
||||
- "Hva er budsjettet for evaluation per måned?"
|
||||
- "Har dere optimalisert sampling rate i continuous eval?"
|
||||
- "Bruker dere billigere judge models for simple evaluators?"
|
||||
|
||||
4. **Compliance:**
|
||||
- "Hvor lagres evaluation data? (EU-region?)"
|
||||
- "Er PII fjernet fra test datasets?"
|
||||
- "Redacts dere reasoning explanations i continuous eval?"
|
||||
|
||||
5. **Production monitoring:**
|
||||
- "Er Application Insights satt opp for continuous eval?"
|
||||
- "Har dere alerts på pass rate degradation?"
|
||||
- "Hvordan debugger dere failed evaluations? (trace-kobling?)"
|
||||
|
||||
6. **Custom evaluators:**
|
||||
- "Har dere behov for custom evaluators utover built-in?"
|
||||
- "Er custom evaluators lagret i Foundry Evaluator Library?"
|
||||
- "Hvordan tester dere custom evaluators selv?"
|
||||
|
||||
7. **Agent framework:**
|
||||
- "Bruker dere Foundry Agent Service, Semantic Kernel, eller custom agents?"
|
||||
- "Støtter eders agent framework AIAgentConverter?"
|
||||
- "Må dere manuelt parse agent messages til OpenAI-style schema?"
|
||||
|
||||
8. **Safety validation:**
|
||||
- "Kjører dere adversarial testing (jailbreak, indirect attack)?"
|
||||
- "Er ContentSafetyEvaluator del av continuous eval?"
|
||||
- "Hvordan håndterer dere evaluation av harmful content?"
|
||||
|
||||
### Fallgruver
|
||||
|
||||
| Fallgruve | Konsekvens | Unngå ved |
|
||||
|-----------|-----------|-----------|
|
||||
| **Ingen continuous evaluation i prod** | Agent degraderer over tid uten at du vet det | Sett opp continuous eval med 5-10% sampling + alerts |
|
||||
| **Test dataset ikke representativt** | Agenten scorer høyt i test, lavt i prod | Bruk production data som test cases (anonymisert) |
|
||||
| **Ignorering av reasoning explanations** | Misforstår hvorfor agenten feiler | Les `{metric}_reason` field for å forstå root cause |
|
||||
| **Tool call mismatch** | ToolCallAccuracyEvaluator scorer lavt selv om agent fungerer | Sjekk at tool_definitions i evaluation matcher faktisk tool schema |
|
||||
| **Cost explosion i continuous eval** | Uventet høy faktura | Start med lav sampling (10%), bruk gpt-4o i stedet for o3-mini for simple metrics |
|
||||
| **Sensitive data i eval traces** | GDPR-brudd | Anonymiser test data, bruk `redact_score_properties=True` |
|
||||
| **Manglende system message i query** | Evaluators kan ikke parse agent context | Alltid inkluder system message som første melding i query |
|
||||
|
||||
### Anbefalinger per modenhetsnivå
|
||||
|
||||
#### Nivå 1: Prototype (ingen prod deployment)
|
||||
- **Local evaluation** med IntentResolution + TaskAdherence
|
||||
- Test på 5-10 manuelt skrevne test cases
|
||||
- Ingen continuous evaluation
|
||||
- Judge model: gpt-4o
|
||||
|
||||
#### Nivå 2: Pilot (begrenset prod bruk)
|
||||
- **Batch evaluation** før hver deploy (50-100 test cases)
|
||||
- Continuous evaluation i prod (10% sampling, kun ContentSafety + IntentResolution)
|
||||
- Application Insights oppsett
|
||||
- Judge model: gpt-4.1
|
||||
|
||||
#### Nivå 3: Production (full prod deployment)
|
||||
- **Batch evaluation** i CI/CD (200+ test cases, quality + safety evaluators)
|
||||
- Continuous evaluation (5-10% sampling, alle relevante evaluators)
|
||||
- Alerts på pass rate < 70%
|
||||
- Trace-debugger i Foundry for failed evals
|
||||
- Judge model: o3-mini for complex evaluators, gpt-4o for simple
|
||||
|
||||
#### Nivå 4: Mission-critical (high-risk AI system)
|
||||
- **Batch evaluation** med 1000+ test cases (inkludert adversarial)
|
||||
- Continuous evaluation (20-50% sampling, alle evaluators)
|
||||
- Custom evaluators for domain-specific metrics
|
||||
- Monthly red teaming med IndirectAttack + CodeVulnerability
|
||||
- Human-in-the-loop review av failed evaluations
|
||||
- Full evaluation audit trail (lagres i 5 år for AI Act compliance)
|
||||
- Judge model: o3-mini + custom fine-tuned judge for kritiske metrics
|
||||
|
||||
## Kilder og verifisering
|
||||
|
||||
### Microsoft Learn (MCP-verified)
|
||||
|
||||
1. **Evaluate your AI agents (preview)**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/agent-evaluate-sdk?view=foundry-classic
|
||||
*Confidence: Verified* — Hovedreferanse for Azure AI Evaluation SDK, evaluator types, model support
|
||||
|
||||
2. **Continuously evaluate your AI agents (preview)**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/continuous-evaluation-agents?view=foundry-classic
|
||||
*Confidence: Verified* — Continuous evaluation setup, sampling configuration, Application Insights integration
|
||||
|
||||
3. **Run evaluations in the cloud by using the Microsoft Foundry SDK**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/cloud-evaluation?view=foundry-classic
|
||||
*Confidence: Verified* — Cloud batch evaluation, CI/CD integration, dataset formats
|
||||
|
||||
4. **Tutorial: Idea to prototype - Build and evaluate an enterprise agent**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/tutorials/developer-journey-idea-to-prototype?view=foundry
|
||||
*Confidence: Verified* — End-to-end tutorial med cloud evaluation, built-in evaluators
|
||||
|
||||
5. **Test and evaluate AI workloads on Azure (Well-Architected Framework)**
|
||||
https://learn.microsoft.com/en-us/azure/well-architected/ai/test#validate-agentic-workflows
|
||||
*Confidence: Verified* — Agentic workflow testing strategy, tool call validation, security testing
|
||||
|
||||
6. **Observability in generative AI**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/observability
|
||||
*Confidence: Verified* — Built-in evaluators list, GenAIOps evaluation stages, simulators
|
||||
|
||||
7. **What are hosted agents? (Evaluate and test hosted agents)**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/hosted-agents?view=foundry#evaluate-and-test-hosted-agents
|
||||
*Confidence: Verified* — Hosted agent evaluation best practices, test dataset creation
|
||||
|
||||
8. **Agent evaluators**
|
||||
https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/evaluation-evaluators/agent-evaluators?view=foundry
|
||||
*Confidence: Verified* — Agent-specific evaluator details (Intent Resolution, Task Adherence, Tool Call Accuracy)
|
||||
|
||||
9. **Evaluate and monitor AI agents (MLflow 3 on Databricks)**
|
||||
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/
|
||||
*Confidence: Verified* — MLflow-based evaluation for cross-platform agents
|
||||
|
||||
10. **Run automated tests for agent quality and reliability (Copilot Studio)**
|
||||
https://learn.microsoft.com/en-us/power-platform/release-plan/2025wave1/microsoft-copilot-studio/run-automated-tests-agent-quality-reliability
|
||||
*Confidence: Verified* — Copilot Studio evaluation framework (2025 preview)
|
||||
|
||||
### Confidence levels per section
|
||||
|
||||
| Section | Confidence | Reason |
|
||||
|---------|-----------|--------|
|
||||
| Introduksjon | Verified | Basert på 3 MCP-kilder (agent-evaluate-sdk, observability, well-architected) |
|
||||
| Kjernekomponenter | Verified | Direkte fra agent-evaluate-sdk dokumentasjon + code samples |
|
||||
| Arkitekturmønstre | Verified | Fra cloud-evaluation + continuous-evaluation docs + code samples |
|
||||
| Beslutningsveiledning | Baseline + Verified | Decision tables basert på best practices (well-architected) + cost models |
|
||||
| Integrasjon med Microsoft-stakken | Verified | Fra Foundry, Semantic Kernel, Prompt Flow, Application Insights docs |
|
||||
| Offentlig sektor (Norge) | Baseline | GDPR/AI Act vurdering basert på modellkunnskap + Azure residency facts |
|
||||
| Kostnad og lisensiering | Baseline | Prisestimater basert på Azure OpenAI pricing (feb 2026) + observability costs |
|
||||
| For arkitekten (Cosmo) | Baseline | Synthesized fra verified sources + praktisk erfaring |
|
||||
|
||||
---
|
||||
|
||||
**Document metadata:**
|
||||
- **MCP calls:** 3 (microsoft_docs_search) + 2 (microsoft_docs_fetch) + 1 (microsoft_code_sample_search) = 6
|
||||
- **Unique sources:** 10 Microsoft Learn URLs
|
||||
- **Word count:** ~3200 ord
|
||||
- **File size:** ~29 KB
|
||||
Loading…
Add table
Add a link
Reference in a new issue