chore(ms-ai-architect): refresh KB medium-bucket — 74 files [skip-docs]
KB-currency refresh (medium priority, 2026-06-19) via /architect:kb-update. 74 medium-prioritets filer re-verifisert mot Microsoft Learn (MCP) — delegert til 15 parallelle Opus-subagenter (3 bølger) gruppert etter delt kilde, med disjunkte fil-sett. Verifisert i hovedkontekst (scope-sjekk + diff-review av de faktatunge gruppene + tester). Hovedendringer (faktuelle korreksjoner + currency): - Azure AI Search semantic ranker: TILGJENGELIG PÅ ALLE TIERS (også Free/Basic m/ gratis månedlig kvote) — gammel KB sa feilaktig "kun S1+". Korrigert i tier-tabell, anti-patterns og beslutningstabell (azure-ai-search-setup). - APIM score-threshold = DISTANSE (lavere = strengere): tuning-tabellen i rag-caching-optimization hadde retningen baklengs — invertert til korrekt. - Agentic retrieval GA/preview-nyanse presisert (hovedkontekst-korreksjon mot agentic-retrieval-how-to-migrate): GA via REST 2026-04-01 returnerer EKSTRAKTIV grounding (references + activity), IKKE syntetiserte svar. Answer synthesis, ikke-minimal reasoning effort (LLM query planning) og multi-turn messages forblir preview (2026-05-01-preview). Subagent hadde overforenklet til "hele kjernepipelinen GA"; rettet i agentic-rag-patterns + citation-tracking. - Copilot Studio modell-tabeller (platforms/copilot-studio): fjernet Claude Opus 4.5 + GPT-5.2 (borte fra kilde), lagt til Claude Sonnet 4.6/Opus 4.6 (GA), Opus 4.7 + Mistral Medium 3.5 (experimental); GPT-5 Reasoning/Auto = preview; A2A GA (apr 2026). - Computer Use (CUA): Copilot Studio GA 2026-05-07; 4 modeller m/ tier/status (OpenAI CUA + Sonnet 4.5 GA, Sonnet 4.6 + Opus 4.6 experimental); 5 credits/ steg standard, 15 premium; US-only region-krav FJERNET i GA-dok; Cloud PC pool + Hosted browser + bring-your-own-machine. - Azure AI Search REST API-versjoner bumpet: 2025-09-01 -> 2026-04-01 (stabil), 2025-11-01-preview -> 2026-05-01-preview (hybrid-search, rag-security-rbac, chunking). - Power Automate-integrasjon: trigger "Run a flow from Copilot" -> "When an agent calls the flow"; App Service innebygd MCP (preview) lagt til. - M365 Copilot-manifest v1.26 -> v1.28 (GA, mai) / v1.29 dokumentert (juni); "Tenant graph grounding" -> "Work IQ". - Speech fast transcription 2t/300MB -> 5t/500MB; multilingual 14 -> 15 locales (+ pt-BR). Content Understanding reasoning preview -> GA (v1.0, 2025-11-01). - Security Copilot E5 -> E5+E7. Død Databricks-URL ci-cd/best-practices -> ci-cd/flows. Prompt Flow retirement (2027-04-20 -> MAF) notert der den presenteres som go-forward. Gateway-topologi-tabell-feil rettet. - Alle 74 Last updated -> 2026-06-19. Discovery ikke kjørt (historisk kun Databricks-støy) -> 389-telling uendret, ingen resync. validate 239 PASS, kb-integrity 115/115 (262 orphan-warnings uendret), gitleaks clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01REiKFhP4w6xGXXqWKpPCJJ
This commit is contained in:
parent
1228b35d81
commit
070141f06b
74 changed files with 403 additions and 384 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Prompt Testing, Evaluation and Iteration
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Last updated:** 2026-06-19
|
||||
**Status:** GA
|
||||
**Category:** Prompt Engineering & LLM Optimization
|
||||
|
||||
|
|
@ -71,6 +71,8 @@ Tilgjengelig i regioner som støtter Response API. Genererer testdata basert på
|
|||
|
||||
### 2. Prompt Flow Evaluation Framework
|
||||
|
||||
> **Utfasing (verifisert MCP 2026-06-19):** Prompt Flow i Microsoft Foundry og Azure Machine Learning utfases **20. april 2027** og anbefales ikke lenger for ny utvikling. Migrer eksisterende Prompt Flow-applikasjoner og -deployments til **Microsoft Agent Framework (MAF)** før denne datoen. Prompt Flow-runtime-images (`promptflow-runtime`, `promptflow-runtime-stable`, `promptflow-python`) får ikke lenger oppdateringer, inkludert sikkerhetsoppdateringer. Evaluation-mønstrene under er fortsatt gyldige for eksisterende løsninger, men velg MAF / Azure AI Evaluation SDK for nye prosjekter.
|
||||
|
||||
**Beskrivelse:** SDK-basert evalueringsrammeverk som lar deg bygge custom evaluation flows som Python-kode eller Prompty-filer, kjøre batch evaluations og logge metrics programmatisk.
|
||||
|
||||
**Evaluation Flow Lifecycle:**
|
||||
|
|
@ -1045,7 +1047,7 @@ def grade(sample: dict, item: dict) -> float:
|
|||
|
||||
**High Confidence (>95%):**
|
||||
- Built-in evaluators (Groundedness, Relevance, Safety) er production-ready og widely used
|
||||
- Prompt Flow SDK evaluation workflow er stable (GA since 2023)
|
||||
- Prompt Flow SDK evaluation workflow er stable (GA siden 2023), men utfases 20. april 2027 — migrer til Microsoft Agent Framework / Azure AI Evaluation SDK for ny utvikling
|
||||
- Azure AI Foundry Evaluation Portal er GA (as of 2024)
|
||||
|
||||
**Medium Confidence (70-95%):**
|
||||
|
|
@ -1064,7 +1066,7 @@ def grade(sample: dict, item: dict) -> float:
|
|||
|
||||
**Primary Sources (Microsoft Learn):**
|
||||
1. [Evaluate generative AI models and applications - Azure AI Foundry](https://learn.microsoft.com/en-us/azure/foundry/how-to/evaluate-generative-ai-app?view=foundry-classic) — GA
|
||||
2. [Evaluation flows and metrics - Azure Machine Learning Prompt Flow](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-develop-an-evaluation-flow?view=azureml-api-2) — GA
|
||||
2. [Evaluation flows and metrics - Azure Machine Learning Prompt Flow](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-develop-an-evaluation-flow?view=azureml-api-2) — GA. Re-verifisert MCP 2026-06-19: Prompt Flow utfases 20. april 2027 → migrer til Microsoft Agent Framework. `log_metric()`/aggregation-mønster uendret.
|
||||
3. [Azure AI Evaluation SDK - Python API](https://learn.microsoft.com/en-us/python/api/overview/azure/ai-evaluation-readme?view=azure-python) — GA
|
||||
4. [Agent evaluation with Azure AI Evaluation SDK](https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/develop/agent-evaluate-sdk?view=foundry-classic) — GA
|
||||
|
||||
|
|
@ -1073,6 +1075,6 @@ def grade(sample: dict, item: dict) -> float:
|
|||
2. [Continuous evaluation setup](https://learn.microsoft.com/en-us/azure/foundry/observability/how-to/how-to-monitor-agents-dashboard?view=foundry)
|
||||
3. [Custom evaluator registration](https://learn.microsoft.com/en-us/azure/foundry/how-to/develop/cloud-evaluation?view=foundry-classic#specify-custom-evaluators)
|
||||
|
||||
**Last Verified:** 2026-02-04
|
||||
**Version:** Azure AI Foundry v2 (2024-2026), Prompt Flow v1.13+ (2024-2026)
|
||||
**Last Verified:** 2026-06-19
|
||||
**Version:** Azure AI Foundry v2 (2024-2026), Prompt Flow v1.13+ (2024-2026; utfases 2027-04-20 → Microsoft Agent Framework)
|
||||
**MCP Calls:** 3 (microsoft_docs_search × 2, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue