open/ms-ai-architect
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

chore(ms-ai-architect): refresh KB medium-bucket — 74 files [skip-docs]

Browse source 
KB-currency refresh (medium priority, 2026-06-19) via /architect:kb-update.
74 medium-prioritets filer re-verifisert mot Microsoft Learn (MCP) — delegert
til 15 parallelle Opus-subagenter (3 bølger) gruppert etter delt kilde, med
disjunkte fil-sett. Verifisert i hovedkontekst (scope-sjekk + diff-review av
de faktatunge gruppene + tester).

Hovedendringer (faktuelle korreksjoner + currency):
- Azure AI Search semantic ranker: TILGJENGELIG PÅ ALLE TIERS (også Free/Basic
  m/ gratis månedlig kvote) — gammel KB sa feilaktig "kun S1+". Korrigert i
  tier-tabell, anti-patterns og beslutningstabell (azure-ai-search-setup).
- APIM score-threshold = DISTANSE (lavere = strengere): tuning-tabellen i
  rag-caching-optimization hadde retningen baklengs — invertert til korrekt.
- Agentic retrieval GA/preview-nyanse presisert (hovedkontekst-korreksjon mot
  agentic-retrieval-how-to-migrate): GA via REST 2026-04-01 returnerer EKSTRAKTIV
  grounding (references + activity), IKKE syntetiserte svar. Answer synthesis,
  ikke-minimal reasoning effort (LLM query planning) og multi-turn messages
  forblir preview (2026-05-01-preview). Subagent hadde overforenklet til "hele
  kjernepipelinen GA"; rettet i agentic-rag-patterns + citation-tracking.
- Copilot Studio modell-tabeller (platforms/copilot-studio): fjernet Claude Opus
  4.5 + GPT-5.2 (borte fra kilde), lagt til Claude Sonnet 4.6/Opus 4.6 (GA),
  Opus 4.7 + Mistral Medium 3.5 (experimental); GPT-5 Reasoning/Auto = preview;
  A2A GA (apr 2026).
- Computer Use (CUA): Copilot Studio GA 2026-05-07; 4 modeller m/ tier/status
  (OpenAI CUA + Sonnet 4.5 GA, Sonnet 4.6 + Opus 4.6 experimental); 5 credits/
  steg standard, 15 premium; US-only region-krav FJERNET i GA-dok; Cloud PC pool
  + Hosted browser + bring-your-own-machine.
- Azure AI Search REST API-versjoner bumpet: 2025-09-01 -> 2026-04-01 (stabil),
  2025-11-01-preview -> 2026-05-01-preview (hybrid-search, rag-security-rbac,
  chunking).
- Power Automate-integrasjon: trigger "Run a flow from Copilot" -> "When an agent
  calls the flow"; App Service innebygd MCP (preview) lagt til.
- M365 Copilot-manifest v1.26 -> v1.28 (GA, mai) / v1.29 dokumentert (juni);
  "Tenant graph grounding" -> "Work IQ".
- Speech fast transcription 2t/300MB -> 5t/500MB; multilingual 14 -> 15 locales
  (+ pt-BR). Content Understanding reasoning preview -> GA (v1.0, 2025-11-01).
- Security Copilot E5 -> E5+E7. Død Databricks-URL ci-cd/best-practices ->
  ci-cd/flows. Prompt Flow retirement (2027-04-20 -> MAF) notert der den
  presenteres som go-forward. Gateway-topologi-tabell-feil rettet.
- Alle 74 Last updated -> 2026-06-19.

Discovery ikke kjørt (historisk kun Databricks-støy) -> 389-telling uendret,
ingen resync. validate 239 PASS, kb-integrity 115/115 (262 orphan-warnings
uendret), gitleaks clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REiKFhP4w6xGXXqWKpPCJJ
This commit is contained in:
Kjell Tore Guttormsen 2026-06-19 14:02:18 +02:00
commit 070141f06b
74 changed files with 403 additions and 384 deletions
Download patch file Download diff file Expand all files Collapse all files

28
skills/ms-ai-advisor/references/prompt-engineering/multimodal-prompt-design.md
View file

@ -1,6 +1,6 @@
# Multimodal Prompt Design with Images and Text
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Last updated:** 2026-06-19 | Verified: MCP 2026-06-19
**Status:** GA
**Category:** Prompt Engineering & LLM Optimization
@ -187,12 +187,12 @@ messages = [
| Verbalization | Semantisk dybde, LLM-sitérbare beskrivelser | LLM-kall per bilde, høyere latency | Diagrammer, flowcharts, infografikk |
| Direct embeddings | Rask, ingen LLM-kall ved indexing | Ingen forklaring av relasjoner | Visual similarity, produktsøk |
**Azure AI Search multimodal pipeline (Verified MCP 2026-04):**
1. **Content extraction** — velg mellom:
- Document Extraction skill: rask prototyping, PDF-støtte
- Document Layout skill: presise sidetall, bounding boxes, RAG-optimalisert
- Azure Content Understanding skill: avansert — cross-page tabeller, semantisk chunking, DOCX/XLSX/PPTX
2. **Text chunking:** Text Split skill
**Azure AI Search multimodal pipeline (Verified MCP 2026-06-19):**
1. **Content extraction** — to anbefalte innebygde skills:
- Document Extraction skill: rask prototyping/produksjon der eksakt posisjon ikke kreves; bilde-posisjonsmetadata kun for PDF; ingen innebygd chunking (bruk Text Split skill)
- Azure Content Understanding skill: avansert — cross-page tabeller, semantisk chunking (innebygd), AI-genererte bildebeskrivelser, og tekst-/bilde-posisjonsmetadata for PDF, DOCX, XLSX, PPTX
- (Document Layout skill er fortsatt støttet for *eksisterende* pipelines, men for nye skillsets anbefaler Microsoft Azure Content Understanding skill, som slår sammen ekstraksjon og chunking i én skill.)
2. **Text chunking:** Text Split skill (ikke nødvendig med Content Understanding, som chunker semantisk)
3. **Image verbalization:** GenAI Prompt skill + LLM (phi-4, gpt-4o, gpt-5) → naturlig-språklig beskrivelse
4. **Embedding:** Azure OpenAI / Microsoft Foundry / Azure Vision multimodal embeddings
5. **Knowledge store:** Lagrer bilder for retrieval; image-lokasjon lagres i indeks for sitert visning
@ -314,7 +314,7 @@ Can you tell me what the image depicts?
**Pipeline-steg (wizard):**
1. Data source: Azure Blob / ADLS Gen2
2. Content extraction: Document Extraction / Layout / Content Understanding skill
2. Content extraction: Document Extraction skill eller Azure Content Understanding skill (Document Layout skill kun for eksisterende pipelines)
3. Text chunking: Text Split skill
4. Image verbalization (optional): GenAI Prompt skill
5. Embedding: Azure OpenAI / Foundry / Azure Vision
@ -457,7 +457,7 @@ Multimodal scenario?
│ └─ Azure AI Search multimodal RAG (verbalization eller direct embeddings)
│
└─ RAG over PDF/Office-dokumenter med embedded diagrammer?
├─ Forklaringsrike visuals: Document Layout skill + GenAI Prompt verbalization
├─ Forklaringsrike visuals: Azure Content Understanding skill (eller Document Extraction) + GenAI Prompt verbalization
└─ Visual similarity: Azure Content Understanding + Azure Vision embeddings
```
@ -541,13 +541,13 @@ AzureDiagnostics
## Kilder og verifisering
**Microsoft Learn dokumentasjon (verifisert 2026-02):**
**Microsoft Learn dokumentasjon (re-verifisert MCP 2026-06-19):**
- [Use vision-enabled chat models](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/gpt-with-vision) — Offisiell how-to guide for GPT-4o/GPT-4 Turbo with Vision
- [Image prompt engineering techniques](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/gpt-4-v-prompt-engineering) — Best practices for multimodal prompting
- [Multimodal search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/multimodal-search-overview) (Re-verified MCP 2026-04) — RAG-arkitektur; extraction skill-sammenligning (Document Extraction vs Layout vs Content Understanding); verbalization vs direct embeddings; hybrid query-alternativ
- [Multimodal search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/multimodal-search-overview) (Re-verified MCP 2026-06-19) — RAG-arkitektur; to anbefalte extraction skills (Document Extraction og Azure Content Understanding; Document Layout kun for eksisterende pipelines); verbalization vs direct embeddings; image-to-vector-queries krever Azure Vision / AML multimodal embeddings-vectorizer; hybrid query-alternativ
- [Azure OpenAI models](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure) — Modelloversikt og token-kostnader
- [Quickstart: Multimodal search in Azure portal](https://learn.microsoft.com/en-us/azure/search/search-get-started-portal-image-search) — Wizard-basert oppsett
- [Get started with multimodal vision chat apps](https://learn.microsoft.com/en-us/azure/developer/ai/get-started-app-chat-vision) — End-to-end sample app med Base64 encoding
- [Get started with multimodal vision chat apps](https://learn.microsoft.com/en-us/azure/developer/ai/get-started-app-chat-vision) (Re-verified MCP 2026-06-19) — End-to-end sample app: Base64-enkoder opplastet bilde i frontend (FileReader), sender via Azure OpenAI Responses API (`input_image`/`input_text`, default gpt-4o), managed identity-autentisering, deploy til Azure Container Apps
**Code samples:**
- Azure-Samples/cognitive-services-sample-data-files (GitHub)
@ -559,5 +559,5 @@ AzureDiagnostics
- ⚠️ **Medium confidence:** Kostberegninger i NOK (basert på jan 2026 pricing, kan variere)
- ⚠️ **Medium confidence:** Offentlig sektor use cases (inferert fra generelle patterns, ikke Microsoft-spesifikt)
**Sist verifisert:** 2026-04-10
**Neste review:** 2026-07 (eller ved nye GPT-modeller/AI Search features)
**Sist verifisert:** 2026-06-19
**Neste review:** 2026-09-19 (eller ved nye GPT-modeller/AI Search features)