docs(architect): weekly KB update — 106 files refreshed (2026-04)
Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering, ms-ai-governance, ms-ai-security, ms-ai-infrastructure. Key changes: - Language Services (Custom Text Classification, Text Analytics, QnA): retirement warning 2029-03-31, migration guides to Foundry/GPT-4o - Agentic Retrieval: 50M free reasoning tokens/month (Public Preview) - Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models - Agent Registry: Risks column (M365 E7), user-shared/org-published types - Declarative agents: schema v1.5 → v1.6, Store validation requirements - MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code - AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python) - Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access - Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned - Fast Transcription API: phrase lists, 14-language multi-lingual transcription - Azure Monitor Workbooks: Bicep support, RBAC specifics - Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA) - RAG security-rbac: 4-approach table (GA + 3 preview access control methods) - IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns - Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP) All 106 files: Last updated 2026-04 | Verified: MCP 2026-04 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
0eb30fa853
commit
6645e93205
104 changed files with 1986 additions and 520 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Multimodal Prompt Design with Images and Text
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Last updated:** 2026-04 | Verified: MCP 2026-04
|
||||
**Status:** GA
|
||||
**Category:** Prompt Engineering & LLM Optimization
|
||||
|
||||
|
|
@ -187,12 +187,20 @@ messages = [
|
|||
| Verbalization | Semantisk dybde, LLM-sitérbare beskrivelser | LLM-kall per bilde, høyere latency | Diagrammer, flowcharts, infografikk |
|
||||
| Direct embeddings | Rask, ingen LLM-kall ved indexing | Ingen forklaring av relasjoner | Visual similarity, produktsøk |
|
||||
|
||||
**Azure AI Search multimodal pipeline:**
|
||||
1. Document extraction (Document Extraction / Layout / Content Understanding skill)
|
||||
2. Text chunking (Text Split skill)
|
||||
3. Image verbalization (GenAI Prompt skill + LLM)
|
||||
4. Embedding (Azure OpenAI / Foundry / Azure Vision)
|
||||
5. Knowledge store (for image storage og retrieval)
|
||||
**Azure AI Search multimodal pipeline (Verified MCP 2026-04):**
|
||||
1. **Content extraction** — velg mellom:
|
||||
- Document Extraction skill: rask prototyping, PDF-støtte
|
||||
- Document Layout skill: presise sidetall, bounding boxes, RAG-optimalisert
|
||||
- Azure Content Understanding skill: avansert — cross-page tabeller, semantisk chunking, DOCX/XLSX/PPTX
|
||||
2. **Text chunking:** Text Split skill
|
||||
3. **Image verbalization:** GenAI Prompt skill + LLM (phi-4, gpt-4o, gpt-5) → naturlig-språklig beskrivelse
|
||||
4. **Embedding:** Azure OpenAI / Microsoft Foundry / Azure Vision multimodal embeddings
|
||||
5. **Knowledge store:** Lagrer bilder for retrieval; image-lokasjon lagres i indeks for sitert visning
|
||||
|
||||
**To retrieval-stier:**
|
||||
- Verbalized content → hybrid queries (text + vector). Gir semantisk dybde og LLM-siterbare beskrivelser.
|
||||
- Direct multimodal embeddings (Azure Vision) → image-to-vector queries. Effektiv visual similarity uten LLM-kall ved indexing.
|
||||
- Mange løsninger kombinerer begge: forklaringsrike visuals verbaliseres, foto/produktbilder embeddes direkte.
|
||||
|
||||
## Beslutningsveiledning
|
||||
|
||||
|
|
@ -445,8 +453,12 @@ Multimodal scenario?
|
|||
├─ Volum > 10k bilder/dag
|
||||
│ └─ Azure AI Search multimodal pipeline + Azure Vision embeddings
|
||||
│
|
||||
└─ Trengs søk over historiske bilder?
|
||||
└─ Azure AI Search multimodal RAG (verbalization eller direct embeddings)
|
||||
├─ Trengs søk over historiske bilder?
|
||||
│ └─ Azure AI Search multimodal RAG (verbalization eller direct embeddings)
|
||||
│
|
||||
└─ RAG over PDF/Office-dokumenter med embedded diagrammer?
|
||||
├─ Forklaringsrike visuals: Document Layout skill + GenAI Prompt verbalization
|
||||
└─ Visual similarity: Azure Content Understanding + Azure Vision embeddings
|
||||
```
|
||||
|
||||
### Red Flags
|
||||
|
|
@ -532,7 +544,7 @@ AzureDiagnostics
|
|||
**Microsoft Learn dokumentasjon (verifisert 2026-02):**
|
||||
- [Use vision-enabled chat models](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/gpt-with-vision) — Offisiell how-to guide for GPT-4o/GPT-4 Turbo with Vision
|
||||
- [Image prompt engineering techniques](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/gpt-4-v-prompt-engineering) — Best practices for multimodal prompting
|
||||
- [Multimodal search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/multimodal-search-overview) — RAG-arkitektur med image verbalization og direct embeddings
|
||||
- [Multimodal search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/multimodal-search-overview) (Re-verified MCP 2026-04) — RAG-arkitektur; extraction skill-sammenligning (Document Extraction vs Layout vs Content Understanding); verbalization vs direct embeddings; hybrid query-alternativ
|
||||
- [Azure OpenAI models](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/models) — Modelloversikt og token-kostnader
|
||||
- [Quickstart: Multimodal search in Azure portal](https://learn.microsoft.com/en-us/azure/search/search-get-started-portal-image-search) — Wizard-basert oppsett
|
||||
- [Get started with multimodal vision chat apps](https://learn.microsoft.com/en-us/azure/developer/ai/get-started-app-chat-vision) — End-to-end sample app med Base64 encoding
|
||||
|
|
@ -547,5 +559,5 @@ AzureDiagnostics
|
|||
- ⚠️ **Medium confidence:** Kostberegninger i NOK (basert på jan 2026 pricing, kan variere)
|
||||
- ⚠️ **Medium confidence:** Offentlig sektor use cases (inferert fra generelle patterns, ikke Microsoft-spesifikt)
|
||||
|
||||
**Sist verifisert:** 2026-02-04
|
||||
**Neste review:** 2026-04 (eller ved nye GPT-modeller)
|
||||
**Sist verifisert:** 2026-04-10
|
||||
**Neste review:** 2026-07 (eller ved nye GPT-modeller/AI Search features)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue