docs(architect): weekly KB update — 106 files refreshed (2026-04)

Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering,
ms-ai-governance, ms-ai-security, ms-ai-infrastructure.

Key changes:
- Language Services (Custom Text Classification, Text Analytics, QnA):
  retirement warning 2029-03-31, migration guides to Foundry/GPT-4o
- Agentic Retrieval: 50M free reasoning tokens/month (Public Preview)
- Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models
- Agent Registry: Risks column (M365 E7), user-shared/org-published types
- Declarative agents: schema v1.5 → v1.6, Store validation requirements
- MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code
- AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python)
- Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access
- Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned
- Fast Transcription API: phrase lists, 14-language multi-lingual transcription
- Azure Monitor Workbooks: Bicep support, RBAC specifics
- Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA)
- RAG security-rbac: 4-approach table (GA + 3 preview access control methods)
- IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns
- Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP)

All 106 files: Last updated 2026-04 | Verified: MCP 2026-04

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-10 09:13:24 +02:00
commit 6645e93205
104 changed files with 1986 additions and 520 deletions

View file

@ -1,6 +1,6 @@
# Multimodal Prompt Design with Images and Text
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Prompt Engineering & LLM Optimization
@ -187,12 +187,20 @@ messages = [
| Verbalization | Semantisk dybde, LLM-sitérbare beskrivelser | LLM-kall per bilde, høyere latency | Diagrammer, flowcharts, infografikk |
| Direct embeddings | Rask, ingen LLM-kall ved indexing | Ingen forklaring av relasjoner | Visual similarity, produktsøk |
**Azure AI Search multimodal pipeline:**
1. Document extraction (Document Extraction / Layout / Content Understanding skill)
2. Text chunking (Text Split skill)
3. Image verbalization (GenAI Prompt skill + LLM)
4. Embedding (Azure OpenAI / Foundry / Azure Vision)
5. Knowledge store (for image storage og retrieval)
**Azure AI Search multimodal pipeline (Verified MCP 2026-04):**
1. **Content extraction** — velg mellom:
- Document Extraction skill: rask prototyping, PDF-støtte
- Document Layout skill: presise sidetall, bounding boxes, RAG-optimalisert
- Azure Content Understanding skill: avansert — cross-page tabeller, semantisk chunking, DOCX/XLSX/PPTX
2. **Text chunking:** Text Split skill
3. **Image verbalization:** GenAI Prompt skill + LLM (phi-4, gpt-4o, gpt-5) → naturlig-språklig beskrivelse
4. **Embedding:** Azure OpenAI / Microsoft Foundry / Azure Vision multimodal embeddings
5. **Knowledge store:** Lagrer bilder for retrieval; image-lokasjon lagres i indeks for sitert visning
**To retrieval-stier:**
- Verbalized content → hybrid queries (text + vector). Gir semantisk dybde og LLM-siterbare beskrivelser.
- Direct multimodal embeddings (Azure Vision) → image-to-vector queries. Effektiv visual similarity uten LLM-kall ved indexing.
- Mange løsninger kombinerer begge: forklaringsrike visuals verbaliseres, foto/produktbilder embeddes direkte.
## Beslutningsveiledning
@ -445,8 +453,12 @@ Multimodal scenario?
├─ Volum > 10k bilder/dag
│ └─ Azure AI Search multimodal pipeline + Azure Vision embeddings
└─ Trengs søk over historiske bilder?
└─ Azure AI Search multimodal RAG (verbalization eller direct embeddings)
├─ Trengs søk over historiske bilder?
│ └─ Azure AI Search multimodal RAG (verbalization eller direct embeddings)
└─ RAG over PDF/Office-dokumenter med embedded diagrammer?
├─ Forklaringsrike visuals: Document Layout skill + GenAI Prompt verbalization
└─ Visual similarity: Azure Content Understanding + Azure Vision embeddings
```
### Red Flags
@ -532,7 +544,7 @@ AzureDiagnostics
**Microsoft Learn dokumentasjon (verifisert 2026-02):**
- [Use vision-enabled chat models](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/gpt-with-vision) — Offisiell how-to guide for GPT-4o/GPT-4 Turbo with Vision
- [Image prompt engineering techniques](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/gpt-4-v-prompt-engineering) — Best practices for multimodal prompting
- [Multimodal search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/multimodal-search-overview) — RAG-arkitektur med image verbalization og direct embeddings
- [Multimodal search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/multimodal-search-overview) (Re-verified MCP 2026-04) — RAG-arkitektur; extraction skill-sammenligning (Document Extraction vs Layout vs Content Understanding); verbalization vs direct embeddings; hybrid query-alternativ
- [Azure OpenAI models](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/models) — Modelloversikt og token-kostnader
- [Quickstart: Multimodal search in Azure portal](https://learn.microsoft.com/en-us/azure/search/search-get-started-portal-image-search) — Wizard-basert oppsett
- [Get started with multimodal vision chat apps](https://learn.microsoft.com/en-us/azure/developer/ai/get-started-app-chat-vision) — End-to-end sample app med Base64 encoding
@ -547,5 +559,5 @@ AzureDiagnostics
- ⚠️ **Medium confidence:** Kostberegninger i NOK (basert på jan 2026 pricing, kan variere)
- ⚠️ **Medium confidence:** Offentlig sektor use cases (inferert fra generelle patterns, ikke Microsoft-spesifikt)
**Sist verifisert:** 2026-02-04
**Neste review:** 2026-04 (eller ved nye GPT-modeller)
**Sist verifisert:** 2026-04-10
**Neste review:** 2026-07 (eller ved nye GPT-modeller/AI Search features)