open/ktg-plugin-marketplace
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

docs(architect): weekly KB update — 106 files refreshed (2026-04)

Browse source 
Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering,
ms-ai-governance, ms-ai-security, ms-ai-infrastructure.

Key changes:
- Language Services (Custom Text Classification, Text Analytics, QnA):
  retirement warning 2029-03-31, migration guides to Foundry/GPT-4o
- Agentic Retrieval: 50M free reasoning tokens/month (Public Preview)
- Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models
- Agent Registry: Risks column (M365 E7), user-shared/org-published types
- Declarative agents: schema v1.5 → v1.6, Store validation requirements
- MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code
- AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python)
- Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access
- Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned
- Fast Transcription API: phrase lists, 14-language multi-lingual transcription
- Azure Monitor Workbooks: Bicep support, RBAC specifics
- Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA)
- RAG security-rbac: 4-approach table (GA + 3 preview access control methods)
- IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns
- Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP)

All 106 files: Last updated 2026-04 | Verified: MCP 2026-04

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-10 09:13:24 +02:00
commit ff6a50d14f
104 changed files with 1986 additions and 520 deletions
Download patch file Download diff file Expand all files Collapse all files

13
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/agent-365-governance-and-deployment.md
View file

@ -22,12 +22,13 @@ Agent Registry er det sentrale administrasjonspunktet for alle agenter i organis
| Komponent | Beskrivelse | Tilgang |
|-----------|-------------|---------|
| **Agent Inventory** | Full oversikt over Microsoft-bygde, partner-bygde og interne agenter | AI Admin, Global Admin, Global Reader (view-only) |
| **Agent Details** | Metadata (capabilities, data sources, actions, sensitivity labels) | Per agent-basis |
| **Agent Inventory** | Full oversikt over Microsoft-bygde, partner-bygde, user-shared og org-published agenter | AI Admin, Global Admin, Global Reader (view-only) |
| **Agent Details** | Metadata (capabilities, data sources, actions, sensitivity labels, permissions-tab) | Per agent-basis |
| **Security & Compliance** | Oversikt over sikkerhetsrisiko (Entra alerts), compliance gaps (Purview) | Integrert med Defender/Purview |
| **Ownerless Agent Management** | Identifisering av agenter uten aktiv eier (f.eks. etter at bruker er slettet) | Real-time oppdatering |
| **Ownerless Agent Management** | Identifisering av agenter uten aktiv eier. Dashboard viser total count, one-click filter, og real-time updates ved brukersletting. *(Verified MCP 2026-04)* | Real-time oppdatering |
| **Risks Column** | Aggregerte high-severity risks fra Entra, Defender og Purview per agent. Kun tilgjengelig med **Microsoft 365 E7-lisens**. *(Verified MCP 2026-04)* | AI Admin, Global Reader |
**Verified (Microsoft Learn, 2026-02)**
**Verified (Microsoft Learn, 2026-04)**
### Agent Lifecycle Actions
@ -38,7 +39,7 @@ Administratorer har 11 lifecycle management actions tilgjengelig i Admin Center:
| **Publish** | Gjør agent tilgjengelig for installasjon (krever AI Admin approval) | Kontrollert utrulling til spesifikke grupper |
| **Activate** | Tillater brukere å installere agenten og opprette instanser | Selvbetjent agent-onboarding |
| **Deploy** | Automatisk installasjon for brukere (ready-to-use) | Zero-touch deployment |
| **Pin** | Fremhev agent i Copilot-interface (opptil 3 administrator-pinned agents) | Prioritering av business-kritiske agenter |
| **Pin** | Fremhev agent i Copilot-interface (opptil 3 administrator-pinned agents per tenant; kun deployed agents kan pinnes; Researcher/Analyst kan ikke scopes — bruk Block for disse) *(Verified MCP 2026-04)* | Prioritering av business-kritiske agenter |
| **Block** | Sperr tilgang for hele organisasjonen | Akutt sikkerhetsrespons |
| **Remove** | Fjern fra tenant inventory (kan gjenopprettes fra store) | Midlertidig deaktivering |
| **Delete** | Permanent sletting (inkludert SharePoint Embedded containers) | Irreversibel cleanup (24t propagation) |
@ -353,7 +354,7 @@ New-MgIdentityGovernanceLifecycleWorkflow -BodyParameter $params
## Kilder og verifisering
### Microsoft Learn (Verified, 2026-02)
- [Agent Registry i Microsoft 365 Admin Center](https://learn.microsoft.com/en-us/microsoft-365/admin/manage/agent-registry) – **Confidence: Verified**
- [Agent Registry i Microsoft 365 Admin Center](https://learn.microsoft.com/en-us/microsoft-365/admin/manage/agent-registry) – **Confidence: Verified (2026-04)** — Oppdatert: Risks column (M365 E7), ownerless agent management, Researcher with Computer Use admin configuration, sensitivity labels for embedded files, GraphAPI for Agent Registry (preview)
- [Microsoft 365 Copilot Agents Deployment Blueprint](https://learn.microsoft.com/en-us/copilot/microsoft-365/agent-essentials/m365-agents-blueprint) – **Confidence: Verified**
- [Copilot Control System Management Controls](https://learn.microsoft.com/en-us/copilot/microsoft-365/copilot-control-system/management-controls) – **Confidence: Verified**
- [Microsoft Entra Agent ID and Agent Identity Platform](https://learn.microsoft.com/en-us/microsoft-agent-365/admin/capabilities-entra) – **Confidence: Verified**

31
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md
View file

@ -1,6 +1,6 @@
# Agent Evaluation and Testing Frameworks
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Azure AI Evaluation SDK), Preview (Agent-specific evaluators)
**Category:** Agent Orchestration & Automation
@ -276,6 +276,33 @@ evaluator = TaskAdherenceEvaluator(
- **KQL queries:** Flexible querying av evaluation metrics over tid
- **Alerts:** Sett opp alerts hvis pass rate dropper under threshold
### MLflow 3 (Databricks / Cross-platform)
MLflow 3 tilbyr komprehensiv GenAI-evaluering for agenter paa tvers av plattformer:
| Feature | Beskrivelse |
|---------|-------------|
| **Built-in LLM judges** | Innebygde dommere for kvalitetsmetrikker (relevance, groundedness, safety, etc.) |
| **Custom scorers** | Definer egne kvalitetsmetrikker med Python-funksjoner |
| **Eval harness** | Test GenAI-app mot eval-datasett under utvikling; sammenlign appversjoner |
| **Conversation evaluation** | Vurder multi-turn samtalekvaltiet (completeness, user frustration, dialogue coherence) |
| **Conversation simulation** | Generer syntetiske multi-turn samtaler for testing |
| **Production monitoring** | Kjoer scorers og judges paa produksjons-traces automatisk (Beta) |
| **Review App** | Samle ekspertfeedback og bygg eval-datasett |
MLflow Tracing gir real-time trace logging gjennom hele livssyklusen. Samme judges og scorers kan brukes i baade development og produksjon — konsistent evaluering.
```python
# MLflow 3 evaluation eksempel
import mlflow
results = mlflow.genai.evaluate(
data=eval_dataset,
predict_fn=my_agent,
scorers=[mlflow.genai.scorers.groundedness(), mlflow.genai.scorers.safety()]
)
```
### Prompt Flow
- **Evaluation flows:** Custom evaluation logic som Prompt Flow (deprecated approach — bruk Azure AI Evaluation SDK i stedet)
@ -488,7 +515,7 @@ evaluator = TaskAdherenceEvaluator(
9. **Evaluate and monitor AI agents (MLflow 3 on Databricks)**
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/
*Confidence: Verified* — MLflow-based evaluation for cross-platform agents
*Confidence: Verified* — MLflow 3 GenAI evaluation: built-in LLM judges og scorers, eval-harness for development, production monitoring (Beta), conversation evaluation (multi-turn), conversation simulation, Review App for human feedback, Genie Code for observability; integrert med MLflow Tracing paa tvers av development/test/produksjon; oppdatert 2026-04
10. **Run automated tests for agent quality and reliability (Copilot Studio)**
https://learn.microsoft.com/en-us/power-platform/release-plan/2025wave1/microsoft-copilot-studio/run-automated-tests-agent-quality-reliability

11
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/agent-memory-and-context-management.md
View file

@ -1,6 +1,6 @@
# Agent Memory and Context Management Strategies
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Managed Memory in Foundry Agent Service: Preview)
**Category:** Agent Orchestration & Automation
@ -189,9 +189,12 @@ var agent = new ChatClientAgent(
```
**Azure Copilot BYOS (Bring Your Own Storage):**
- Organisations-kontrollert Cosmos DB instance
- Audit trail av alle samtaler
- Managed identity-basert tilgang
- Organisasjonen velger og administrerer sin egen Azure Cosmos DB-instans
- Full audit trail av alle Azure Copilot-samtaler (user prompts + Copilot responses) for alle tenant-brukere
- System-assigned managed identity med `Cosmos DB Built-in Data Contributor`-rollen for sikker lese-/skrivetilgang
- Aktiveres via Azure Copilot admin center → Conversation storage
- **OBS:** Hvis BYOS aktiveres, mister brukere tilgang til samtaler lagret av Microsoft foer aktivering. Bytte av Cosmos DB-instans gir tilsvarende tap av tilgang til tidligere instans.
- **OBS:** BYOS deaktiverer for oeyeblikket migration agent-kapabiliteter i Azure Copilot
**Fordeler:**
- Full data control og compliance

30
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/agent-to-agent-a2a-protocol.md
View file

@ -1,6 +1,6 @@
# Agent2Agent (A2A) Protocol — Åpen Standard for Agent-Interoperabilitet
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** Preview (Microsoft-implementasjoner) / GA (protokollspesifikasjon v0.3)
**Category:** Agent Orchestration & Automation
@ -210,6 +210,8 @@ A2A og MCP (Model Context Protocol) løser forskjellige problemer og er kompleme
| **Transparens** | Intern logikk er ugjennomsiktig for kallende agent | Orkestrator ser og kontrollerer all verktøybruk |
| **Beste for** | Agenter eid av forskjellige team/org, kompleks delegering | Enkelt, kontrollert tilgang til APIer og data |
**A2A-melding inkluderer rik metadata:** Hver A2A-melding inneholder et unikt `contextId`, `messageId`, locale-info, full chat-historikk (ikke bare siste melding), og content parts (tekst, tool calls, etc.). Downstream agenter kan bruke denne metadata til routing, kontekst og kontinuitet.
**Typisk kombinert bruk:**
```
@ -263,12 +265,24 @@ with AIProjectClient(endpoint=endpoint, credential=DefaultAzureCredential()) as
Copilot Studio kan konsumere A2A-agenter direkte:
1. Gå til **Agents** → **Add an agent** → **Connect to an external agent** → velg **Agent2Agent**
2. Angi endepunkt-URL (ikke URL for agent card, men kommunikasjonsendepunktet)
3. Copilot Studio henter automatisk navn og beskrivelse fra `/.well-known/agent.json`
4. Velg autentiseringsmetode
1. Gå til **Agents**-siden → **Add an agent** → **Connect to an external agent** → velg **Agent2Agent**
2. Angi endepunkt-URL (kommunikasjonsendepunktet, IKKE URL for agent card)
3. Copilot Studio henter automatisk navn og beskrivelse fra `/.well-known/agent.json` (standard well-known-URL). Hvis automatisk populering feiler, angi navn og beskrivelse manuelt
4. Velg autentiseringsmetode: **None**, **API key**, eller **OAuth 2.0**
5. Velg eller opprett connection, deretter **Add and configure**
**Viktig:** Copilot Studio er ansvarlig for å vurdere datadeling, sikkerhet og compliance for tilkoblede eksterne agenter.
**A2A vs HTTP connector — valg av integrasjonstype:**
| Behov | Anbefalt |
|-------|----------|
| Agenter som er bygget på eksterne rammeverk | A2A |
| Agenter hostet utenfor Copilot Studio | A2A |
| Multi-turn interaksjoner med domenespesifikk resonnering | A2A |
| Enkle API-kall eller HTTP-tjenester | Custom connectors |
| MCP-verktøy og ressurser | MCP-servere |
| Microsoft 365 Agents SDK-agenter | Activity Protocol |
**Viktig:** Tilkobling til A2A-agenter utenfor Copilot Studio gir brukeransvar for datadeling, sikkerhet, compliance og kvalitetssikring.
### Semantic Kernel
@ -614,11 +628,11 @@ app.MapA2A(agent, "/a2a/my-agent", agentCard: new()
3. **Copilot Studio — Connect A2A Agent**
- https://learn.microsoft.com/microsoft-copilot-studio/add-agent-agent-to-agent
- Confidence: **Verified** (offisiell guide, februar 2026)
- Confidence: **Verified** (offisiell guide, oppdatert 2026-04: oppsettstrinn, autentiseringsalternativer, A2A vs HTTP connector-tabell)
4. **Multi-agent Patterns — MCP vs A2A**
- https://learn.microsoft.com/microsoft-copilot-studio/guidance/architecture/multi-agent-patterns
- Confidence: **Verified** (Copilot Studio arkitekturguide, februar 2026)
- Confidence: **Verified** (Copilot Studio arkitekturguide, oppdatert 2026-04: MCP vs A2A capability-matrise, hybrid workflow-anbefalinger, Agent 365 control plane)
5. **Azure API Management — A2A Agent API**
- https://learn.microsoft.com/azure/api-management/agent-to-agent-api

14
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/agent-to-agent-communication.md
View file

@ -1,6 +1,6 @@
# Agent-to-Agent Communication Protocols
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Agent Orchestration & Automation
@ -237,8 +237,14 @@ eventBus.Subscribe<AgentCompletedEvent>(async evt =>
| Topologi | Beskrivelse | Use Case |
|----------|-------------|----------|
| **Broker Topology** | Agenter broadcaster events, andre agenter reagerer eller ignorerer | Dynamiske workflows, ingen sentral koordinering |
| **Mediator Topology** | En mediator styrer event flow og state | Komplekse workflows med error handling og state management |
| **Broker Topology** | Agenter broadcaster events, andre agenter reagerer eller ignorerer. Hoey dekobling, men mangler innebygd error handling og distributed transaction-stoette | Dynamiske workflows, ingen sentral koordinering |
| **Mediator Topology** | En mediator styrer event flow og state, dispatcher commands til dedikerte kanaler. Bedre feilhaandtering og datakonsistens, men oekt kobling og potensielt bottleneck | Komplekse workflows med error handling og state management |
**Event-driven utfordringer ved agent-kommunikasjon:**
- **Eventual consistency:** Data paa tvers av agenter er ikke umiddelbart konsistent. Design for dette bevisst.
- **Ordering:** Ved skalering kan events motttas i feil rekkefoelge. Bruk partisjonsnoekler og idempotent processing.
- **Observability:** Inkluder correlation ID i alle events fra start — retrofit er vanskelig.
- **Schema evolution:** Definer versjonsstrategi tidlig. Design consumers til aa haandtere ukjente event-versjoner.
**Når brukes dette:**
- Lang-levende workflows (timer/dager)
@ -538,7 +544,7 @@ Trenger dere agent discovery?
4. **Event-Driven Architecture (Azure)**
- https://learn.microsoft.com/en-us/azure/architecture/guide/architecture-styles/event-driven
- Confidence: **Verified** (Azure Architecture Center, 2024)
- Confidence: **Verified** (Azure Architecture Center, oppdatert 2026-04: broker vs mediator topology, eventual consistency, ordering, observability, schema evolution)
5. **Azure Service Bus Integration**
- https://learn.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/integration-event-based-microservice-communications

22
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/computer-using-agents-cua.md
View file

@ -1,6 +1,6 @@
# Computer-Using Agents (CUA)
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** Preview (sep 2025 — Foundry Agent Service; mai 2025 — Copilot Studio)
**Category:** Agent Orchestration & Automation
@ -109,12 +109,15 @@ Copilot Studio tilbyr CUA som et lavkode **Computer Use Tool** — ingen koding
### Oppsett
1. Gå til **Tools** i agenten → **Add tool** → **Computer use**
2. Velg modell: OpenAI Computer-Using Agent eller Anthropic Claude Sonnet 4.5
3. Skriv instruksjoner på naturlig norsk/engelsk
1. Gå til **Tools** i agenten → **Add tool** → **New tool** → **Computer use**
2. Velg modell: **OpenAI Computer-Using Agent** eller **Anthropic Claude Sonnet 4.5** (preview, pågående regionutrulling — krever at administrator har aktivert external models)
3. Skriv instruksjoner på naturlig norsk/engelsk (se "Best practices for instructions" under)
4. Konfigurer **Machine** (hvor CUA kjøres):
- **Hosted browser** (Windows 365 for Agents) — rask start, kun web, ikke anbefalt for produksjon
- **Dedikert Windows-maskin** — gir full desktop-tilgang, anbefalt for produksjon
- Velg målmaskin fra listen, eller opprett ny via Power Automate Portal
- **Hosted browser**: rask start, kun web — ikke anbefalt for produksjon
- **Dedikert Windows-maskin**: gir full desktop-tilgang, anbefalt for produksjon
**Merk:** Tilgangskontroll (access control) hindrer kun at modellen *utfører handlinger* på ikke-autoriserte nettsider/apper — ikke at de åpnes. Eksempel: Bing kan åpnes fra Edge-søkebaren selv om kun microsoft.com er på allowlisten, men interaksjon med Bing vil feile.
### Credentials og tilgangskontroll
@ -122,12 +125,13 @@ Copilot Studio tilbyr CUA som et lavkode **Computer Use Tool** — ingen koding
|---------------|-------------|
| **Maker-provided credentials** | Agenten bruker makerens innloggingsinfo (for autonome agenter) |
| **End user credentials** | Brukeren logger inn selv (for konversasjonelle agenter) |
| **Intern Power Platform-lagring** | Kryptert intern lagring — ingen forhåndskonfigurasjon nødvendig |
| **Azure Key Vault** | Passord lagres i Key Vault — anbefalt for produksjonsmiljøer |
| **Access control** | Begrens hvilke nettsider/applikasjoner CUA kan operere på |
### Lisensiering (Copilot Studio, preview)
Faktureres som Agent action: **5 Copilot Credits per steg**.
Faktureres som Agent action: **5 Copilot Credits per steg** (hvert steg kan inneholde én eller flere lavnivå-handlinger som klikk, skriving eller navigering).
Eksempel — utfylling av timeregistreringsskjema (4 steg = 20 Copilot Credits):
1. Åpne nettleser og naviger til URL
@ -481,7 +485,7 @@ Kostnader basert på:
2. **Automate web and desktop apps with computer use — Copilot Studio**
- https://learn.microsoft.com/microsoft-copilot-studio/computer-use
- Confidence: **Verified** (offisiell Copilot Studio preview-dokumentasjon, 2025)
- Confidence: **Verified** (offisiell Copilot Studio preview-dokumentasjon, oppdatert 2026-04: støttede modeller, credentials, access control-semantikk)
3. **Configure where computer use runs**
- https://learn.microsoft.com/microsoft-copilot-studio/configure-where-computer-use-runs
@ -497,7 +501,7 @@ Kostnader basert på:
6. **FAQ for the computer use tool**
- https://learn.microsoft.com/microsoft-copilot-studio/faqs-computer-use
- Confidence: **Verified** (offisiell FAQ, inkl. 80%/35% suksessrater)
- Confidence: **Verified** (offisiell FAQ, inkl. 80%/35% suksessrater, human supervision-detaljer, oppdatert 2026-04)
7. **Computer Use Release Plan (2025 Wave 1)**
- https://learn.microsoft.com/power-platform/release-plan/2025wave1/microsoft-copilot-studio/automate-web-desktop-apps-computer-use

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/foundry-workflows-visual-orchestration.md
View file

@ -1,6 +1,6 @@
# Foundry Workflows — Visuell Multi-Agent Orkestrering
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** Public Preview (announced MS Ignite november 2025)
**Category:** Agent Orchestration & Automation
@ -133,6 +133,8 @@ Human-in-the-loop er et førsteklasses konsept i Foundry Workflows. Workflowen *
falsePath: return_for_revision
```
**Agent Framework HITL (pro-code):** Bruk `RequestPort` (C#) eller `ctx.request_info()` (Python) for HITL i egendefinerte workflows. For agent orchestrations (sequential, concurrent, group chat): bruk `ToolApprovalRequestContent` — agenten kan markere tools som approval-required, workflow pauser og emitter `RequestInfoEvent`. Checkpoints bevarer pending requests ved gjenopptak.
### Detaljer: Loop-noden (For each)
```yaml
@ -598,7 +600,7 @@ Foundry Workflows' visuelle designer gir offentlig sektor-organisasjoner en unik
5. **Declarative Workflows — Overview (Agent Framework)**
- https://learn.microsoft.com/agent-framework/workflows/declarative
- Confidence: **Verified** (YAML patterns, looping, HITL, error handling)
- Confidence: **Verified** (YAML action types tabell: Variable Management, Control Flow, Agent/Tool Invocation, HITL, Conversation — C# og Python; oppdatert 2026-04)
6. **Human-in-the-Loop Workflows**
- https://learn.microsoft.com/agent-framework/workflows/human-in-the-loop

29
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/multi-agent-orchestration-patterns.md
View file

@ -1,6 +1,6 @@
# Multi-Agent Orchestration Patterns and Topologies
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Agent Orchestration & Automation
@ -429,11 +429,24 @@ await Task.WhenAll(taskA, taskB); // Checkpoint ensures no replay
- Anbefales for inter-platform orchestration
**Multi-Agent Pattern Recommendations (Microsoft Copilot Studio):**
1. Prefer platform-native orchestration for internal flows
2. Use MCP for tool/data access (M365 services)
3. Use A2A for cross-platform messaging
4. Integrate mature agents via MCP or A2A
5. Enforce policy/auditing via Agent 365 control plane
1. Prefer platform-native orchestration for internal flows with subagents
2. Use MCP for tool and data access (M365 services) — enterprise-grade security, authentication, auditing
3. Use A2A for cross-platform agent-to-agent messaging — design for capability discovery and task contracts
4. Integrate mature or abstracted agents via MCP or A2A for reuse and end-to-end traceability
5. Enforce policy and auditing at control-plane layer with Agent 365
6. Use least-privileged scope when calling MCP-hosted tools
7. Design for parallelism, limit inter-agent context to strictly necessary, use short-term memory
8. Include users in workflow — require human approvals for high-impact cross-agent actions
**MCP vs A2A — nar bruke hva (oppdatert fra Copilot Studio multi-agent-patterns):**
| Kapabilitet | MCP | A2A |
|-------------|-----|-----|
| Multimodalitet | Krever at MCP host stoetter det | Annonserer stoettede medietyper |
| Multi-turn interaksjoner | Valgfri elicitation. Kontekst hos host | contextId haandterer kontekst paa tvers av agenter |
| Orkestrering | Host orkestrerer hvilke tools som kalles | Invokert agent bruker sin egen chain-of-thought |
| Forhandling | Krever klientoppdatering | Dynamisk forhandling, robust mot serviceoppdateringer |
| Beste for | Full kontroll over resonnering, kontrollerte scenarios | Opake agenter, cross-org, ekstern agent |
### Semantic Kernel + Agent Framework
@ -638,7 +651,7 @@ scope.RecordOutputMessages(new[] { output });
1. **AI agent orchestration patterns** (Azure Architecture Center)
https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns
*Confidence: Verified* — Definitive guide til alle 5 patterns
*Confidence: Verified* — Definitive guide til alle 5 patterns, spektrum av kompleksitet (direct model call → single agent → multi-agent), oppdatert 2026-04
2. **Microsoft Agent Framework Workflows Orchestrations**
https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/overview
@ -658,7 +671,7 @@ scope.RecordOutputMessages(new[] { output });
6. **Multi-agent patterns (Copilot Studio)**
https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/architecture/multi-agent-patterns
*Confidence: Verified* — A2A protocol, MCP integration
*Confidence: Verified* — A2A protocol, MCP integration, capability matrise, hybrid workflow diagram, oppdatert 2026-04
7. **Build agent platforms on Azure** (Microsoft for Startups)
https://learn.microsoft.com/en-us/microsoft-for-startups/build/build-agent

10
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/semantic-kernel-agents-implementation.md
View file

@ -1,6 +1,6 @@
# Semantic Kernel and Microsoft Agent Framework - Implementation Patterns
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Agent Orchestration: Experimental)
**Category:** Agent Orchestration & Automation
@ -235,6 +235,12 @@ Trenger du OpenAI Assistants API features (code interpreter, retrieval)?
### Velg Orchestration Pattern
**Foer du velger multi-agent pattern:** Evaluer om scenariet faktisk krever det. Hvert kompleksitetsnivaa introduserer koordinerings-overhead, latens og kostnad. Bruk laveste kompleksitetsnivaa som tilfredsstillende loser problemet:
1. **Direct model call** — klassifisering, oppsummering, oversettelse (ingen agent)
2. **Single agent med tools** — varierte spoerringsmaal innen ett domene (ofte riktig default)
3. **Multi-agent orchestration** — cross-domain, ulike sikkerhetsbegrensninger, eller oppgaver som drar nytte av parallell spesialisering
| Scenario | Anbefalt Pattern | Hvorfor |
|----------|------------------|---------|
| Uavhengige subtasks | Concurrent | Parallell utførelse, redusert total tid |
@ -459,7 +465,7 @@ Kernel agentKernel = sharedKernel.Clone(); // Rebruk AI service connections
5. [Microsoft Agent Framework Overview](https://learn.microsoft.com/en-us/agent-framework/overview/agent-framework-overview) — Confidence: Verified (2026-02)
6. [Magentic Orchestration Pattern](https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/magentic) — Confidence: Verified (2026-02)
7. [Microsoft 365 Agents SDK - Semantic Kernel Integration](https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/using-semantic-kernel-agent-framework) — Confidence: Verified (2026-02)
8. [AI Agent Orchestration Patterns (Azure Architecture)](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns) — Confidence: Verified (2026-02)
8. [AI Agent Orchestration Patterns (Azure Architecture)](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns) — Confidence: Verified (oppdatert 2026-04: start med riktig kompleksitetsnivaa — direct model call, single agent med tools, multi-agent; guidance om naar multi-agent er hensiktsmessig)
### Kodeeksempler (Verified via MCP Code Search)

40
plugins/ms-ai-architect/skills/ms-ai-engineering/references/agent-orchestration/tool-use-and-function-calling-patterns.md
View file

@ -1,6 +1,6 @@
# Tool Use and Function Calling - Advanced Patterns
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Agent Orchestration & Automation
@ -207,19 +207,35 @@ var result = await mainAgent.RunAsync("Hvordan er været i Oslo?");
**Implementering (AG-UI + Agent Framework):**
```python
# AG-UI middleware for approval
def approval_middleware(tool_call):
if tool_call.name in ["delete_record", "send_email"]:
user_response = input(f"Approve {tool_call.name}? (yes/no): ")
return user_response.lower() == "yes"
return True # Auto-approve andre funksjoner
AG-UI backend tool rendering stoetter HITL via to mekanismer:
# Agent med approval-workflow
agent = ChatAgent(chat_client=client, tools=[delete_record, send_email])
# Koble approval_middleware til agent runtime
**C# - ApprovalRequiredAIFunction:**
```csharp
// Tool som krever human approval
var approvalTool = ApprovalRequiredAIFunction.Create(DeleteRecord);
// Workflow emitter RequestInfoEvent med ToolApprovalRequestContent
await foreach (var evt in workflow.WatchStreamAsync()) {
if (evt is RequestInfoEvent req && req.Data is ToolApprovalRequestContent tc) {
bool approved = await AskUserApproval(tc.ToolName);
await handle.SendResponseAsync(req.Request.CreateResponse(approved));
}
}
```
**Python - @tool med approval_mode:**
```python
@tool(approval_mode="always_require")
def delete_record(record_id: str) -> str:
# Sletter en post - krever alltid menneskelig godkjenning
return db.delete(record_id)
# Workflow pauser og emitter function_approval_request event
# Klient-loop maa haandtere og respondere
```
**Backend tool events streames til klient i sanntid:** TOOL_CALL_START, TOOL_CALL_ARGS, TOOL_CALL_END, TOOL_CALL_RESULT.
---
## Beslutningsveiledning
@ -412,7 +428,7 @@ def update_citizen_record(ssn: str, field: str, value: str) -> str:
1. [Azure OpenAI Function Calling](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/function-calling) — **Verified 2026-02**
2. [Semantic Kernel Agent Functions](https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/agent-functions) — **Verified 2026-02**
3. [Agent Framework - Agent as Function Tool](https://learn.microsoft.com/en-us/agent-framework/tutorials/agents/agent-as-function-tool) — **Verified 2026-02**
4. [AG-UI Backend Tool Rendering](https://learn.microsoft.com/en-us/agent-framework/integrations/ag-ui/backend-tool-rendering) — **Verified 2026-02**
4. [AG-UI Backend Tool Rendering](https://learn.microsoft.com/en-us/agent-framework/integrations/ag-ui/backend-tool-rendering) — **Verified 2026-04** (backend tool streaming, ApprovalRequiredAIFunction C#, @tool(approval_mode) Python, TOOL_CALL_* events)
5. [Azure OpenAI Assistants Function Calling](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/assistant-functions) — **Verified 2026-02**
6. [Structured Outputs](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/structured-outputs) — **Verified 2026-02**

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/ai-services-api-best-practices.md
View file

@ -1,6 +1,6 @@
# Azure AI Services - API Design and Best Practices
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Azure AI Services (Foundry Tools)

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/ai-services-enterprise-architecture.md
View file

@ -1,5 +1,5 @@
# Azure AI Services - Enterprise Architecture Patterns
**Last updated:** 2026-04
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Azure AI Services (Foundry Tools)
@ -16,7 +16,7 @@ Azure AI Services (tidligere Cognitive Services) krever robuste enterprise-arkit
- Network isolation og security perimeter
- Kvotestyring og throttling-håndtering
**Scope:** Dette dokumentet fokuserer på arkitekturmønstre for Azure OpenAI (del av Foundry Models), Azure AI Search, og støttetjenester som Azure API Management og Azure Front Door. Mønstrene gjelder både Foundry-baserte løsninger og standalone AI Services.
**Scope:** Dette dokumentet fokuserer på arkitekturmønstre for Azure OpenAI (del av Foundry Models, tidligere "OpenAI in Azure" — nå konsolidert under Foundry Tools), Azure AI Search, og støttetjenester som Azure API Management og Azure Front Door. Mønstrene gjelder både Foundry-baserte løsninger og standalone AI Services.
---
@ -549,7 +549,7 @@ TOTAL: ~46 700 NOK/måned (høyere cost, men forutsigbar)
3. [Baseline Foundry Chat Architecture (Foundry Agent Service + Microsoft Agent Framework)](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/architecture/baseline-microsoft-foundry-chat) — Verified (MCP 2026-04)
4. [Azure API Management - AI Gateway Capabilities](https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities)
5. [Reliability in Azure AI Search](https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search)
6. [Multi-Backend Gateway Guide](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-multi-backend)
6. [Multi-Backend Gateway Guide](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-multi-backend) — Verified MCP 2026-04: Dokumentet bekrefter fire gateway-topologier (single instance/multiple deployments, multi-instance same region/subscription, multi-instance multi-region). Tagger nå eksplisitt "Foundry Tools" og "Azure OpenAI in Foundry Models".
7. [Load Balancing Options - Azure Architecture](https://learn.microsoft.com/en-us/azure/architecture/guide/technology-choices/load-balancing-overview)
**GitHub Samples (Microsoft-verified):**

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/ai-services-monitoring-logging.md
View file

@ -1,6 +1,6 @@
# Azure AI Services - Monitoring, Logging and Diagnostics
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Azure AI Services (Foundry Tools)

5
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/ai-services-networking-security.md
View file

@ -1,7 +1,10 @@
# Azure AI Services - Networking, Security and Private Endpoints
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
> **Navngivning 2026-04:** Microsoft har omdøpt "Azure Cognitive Services" til **"Azure AI Foundry Tools"** (eller kortform "Foundry Tools") i dokumentasjonen. API-endepunkter, RBAC-roller og ARM-ressurstyper beholder "CognitiveServices" i identifikatoren. Alle nye dokumentasjonsreferanser bruker "Foundry Tools".
**Category:** Azure AI Services (Foundry Tools)
---

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/ai-services-vs-foundry-tools-selection.md
View file

@ -1,6 +1,6 @@
# Azure AI Services vs Foundry Tools - Platform Selection Guide
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Azure AI Services (Foundry Tools)

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/azure-ai-vision-image-analysis.md
View file

@ -1,6 +1,6 @@
# Azure AI Vision - Image Analysis and Tagging
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Generally Available)
**Category:** Azure AI Services (Foundry Tools)

5
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/azure-ai-vision-ocr-processing.md
View file

@ -1,7 +1,10 @@
# Azure AI Vision - OCR and Document Processing
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
> **Oppdatering 2026-04:** For OCR kombinert med semantisk analyse, bruk **Azure AI Content Understanding** (GA). Azure AI Vision OCR (Read API) er fortsatt det beste valget for ren tekst-ekstraksjon, men Content Understanding gir overlegent resultat for dokumenter der layout, tabeller og kontekstuell forståelse er viktig.
**Category:** Azure AI Services (Foundry Tools)
---

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/content-understanding-multimodal-analysis.md
View file

@ -1,6 +1,6 @@
# Content Understanding - Multimodal Analysis and Video Intelligence
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** Preview (GA for core features, Limited Access for face description)
**Category:** Azure AI Services (Foundry Tools)

5
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/document-intelligence-prebuilt-models.md
View file

@ -1,7 +1,10 @@
# Document Intelligence - Prebuilt Models for Forms and Invoices
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
> **Oppdatering 2026-04:** Azure AI Content Understanding er nå **fullt GA** og er anbefalt startpunkt for de fleste IDP-scenarier. Content Understanding dekker og utvider Document Intelligence-funksjonalitet med multimodal analyse. Velg Document Intelligence for spesifikke prebuilt-skjemaer (regnskap, ID-dokumenter, kvitteringer) der disse gir direkte verdi uten tilpasning. For generell dokumentanalyse og semantisk chunking — start med Content Understanding.
**Category:** Azure AI Services (Foundry Tools)
---

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/language-services-custom-text-classification.md
View file

@ -1,7 +1,9 @@
# Language Services - Custom Text Classification and NER
**Last updated:** 2026-02
**Status:** GA
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA — avvikles 31. mars 2029
> **ADVARSEL — TJENESTE AVVIKLES:** Custom Text Classification og Custom Named Entity Recognition (NER) avvikles **31. mars 2029**. Migrer til Azure AI Foundry-modeller (prompt-basert klassifisering og NER med GPT-4o eller GPT-4.1). Se [migrasjonsveiledning](https://learn.microsoft.com/azure/ai-services/language-service/custom-text-classification/how-to/migrate-azure-openai) for detaljer.
**Category:** Azure AI Services (Foundry Tools)
---

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/language-services-question-answering.md
View file

@ -1,7 +1,9 @@
# Language Services - Question Answering and Knowledge Mining
**Last updated:** 2026-02
**Status:** GA
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA — avvikles 31. mars 2029
> **ADVARSEL — TJENESTE AVVIKLES:** Custom Question Answering (CQA) avvikles **31. mars 2029**. Migrer til Azure AI Foundry-baserte løsninger: Agentic Retrieval, Azure AI Search + GPT-4o RAG-pipeline, eller AI Foundry Knowledge Retrieval. Se [migrasjonsveiledning](https://learn.microsoft.com/azure/ai-services/language-service/question-answering/how-to/migrate-qnamaker-to-question-answering) for detaljer.
**Category:** Azure AI Services (Foundry Tools)
---

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/language-services-text-analytics.md
View file

@ -1,7 +1,9 @@
# Language Services - Text Analytics for Sentiment and Key Phrases
**Last updated:** 2026-02
**Status:** GA
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (deler avvikles 2029-03-31)
> **ADVARSEL — TJENESTER AVVIKLES (2029-03-31):** Sentiment Analysis, Opinion Mining og Custom Text Classification avvikles 31. mars 2029. Migrer til Azure AI Foundry-modeller. PII Detection, Key Phrase Extraction og Language Detection er ikke berørt.
**Category:** Azure AI Services (Foundry Tools)
---

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/speech-services-speech-to-text.md
View file

@ -1,6 +1,6 @@
# Speech Services - Speech-to-Text and Real-time Transcription
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** Azure AI Services (Foundry Tools)

5
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/speech-services-text-to-speech.md
View file

@ -1,7 +1,10 @@
# Speech Services - Text-to-Speech and Neural Voices
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
> **Status 2026-04:** Azure Neural TTS og Custom Neural Voice er begge bekreftet GA og aktivt vedlikeholdt. `nb-NO-PernilleNeural` og `nb-NO-FinnNeural` er de primære norske stemmene. Custom Neural Voice Pro tilbyr ytterligere tilpasning for enterprise-bruk.
**Category:** Azure AI Services (Foundry Tools)
---

2
plugins/ms-ai-architect/skills/ms-ai-engineering/references/azure-ai-services/translator-document-translation.md
View file

@ -1,6 +1,6 @@
# Translator Service - Document Translation and Multi-language Support
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (General Availability) + Preview features
**Category:** Azure AI Services (Foundry Tools)

8
plugins/ms-ai-architect/skills/ms-ai-engineering/references/data-engineering/microsoft-purview-governance.md
View file

@ -231,9 +231,11 @@ Purview Data Owner Policies muliggjør sentralisert tilgangsstyring:
### Governance Domains og OKR-er
Governance Domains er nå den sentrale organiseringsenhet for glossary terms i Unified Catalog. Workflow: opprett term (Draft) → rediger → publiser. Glossary terms kan flyttes mellom domains (begge domains krever Data Steward-rolle). Termer kan linkes til data products og critical data elements på tvers av domains. *(Verified MCP 2026-04)*
```
Governance Domain: "AI og Maskinlæring"
├── Glossary Terms
├── Glossary Terms (Data Steward-rolle påkrevd)
│ ├── "Treningsdata" -- Definisjon og bruksregler
│ ├── "Feature Store" -- Standard for feature-lagring
│ └── "Ground Truth" -- Krav til merkede datasett
@ -243,7 +245,7 @@ Governance Domain: "AI og Maskinlæring"
├── OKRs
│ ├── "90% av AI-datasett klassifisert innen Q2"
│ └── "100% lineage-dekning for ML-pipelines"
└── Data Products
└── Data Products (kan linkes til glossary terms)
├── "Customer 360 Feature Set"
└── "Trafikkdata for ML"
```
@ -344,7 +346,7 @@ Microsoft Purview gir nå governance-dekning for Fabric Copilots og agenter —
- [How to get lineage from Microsoft Fabric items into Microsoft Purview](https://learn.microsoft.com/en-us/purview/data-map-lineage-fabric) -- Lineage fra Fabric
- [Data lineage in classic Data Catalog](https://learn.microsoft.com/en-us/purview/data-gov-classic-lineage) -- Lineage-konsepter
- [Learn about sensitivity labels in Data Map](https://learn.microsoft.com/en-us/purview/data-map-sensitivity-labels) -- Sensitivitetsmerking
- [Create and manage glossary terms](https://learn.microsoft.com/en-us/purview/unified-catalog-glossary-terms-create-manage) -- Business glossary
- [Create and manage glossary terms](https://learn.microsoft.com/en-us/purview/unified-catalog-glossary-terms-create-manage) -- Business glossary *(Verified MCP 2026-04)* — Ny funksjonalitet: bulk edit opptil 50 terms, flytt terms mellom governance domains, custom attributes med filter, Data Steward-rolle påkrevd for opprettelse. Terms opprettes i Draft state, må publiseres for å bli synlige. Governance domain MÅ publiseres FØR terms publiseres.
- [Glossary terms in Unified Catalog](https://learn.microsoft.com/en-us/purview/unified-catalog-glossary-terms) -- Aktive glossary-termer
- [Learn about Microsoft Purview Unified Catalog](https://learn.microsoft.com/en-us/purview/unified-catalog) -- Oversikt over Unified Catalog
- [Set up data quality for Fabric Lakehouse data](https://learn.microsoft.com/en-us/purview/unified-catalog-data-quality-fabric-lakehouse) -- Datakvalitet for Fabric

35
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/ab-testing-llm-applications.md
View file

@ -1,11 +1,14 @@
# A/B Testing and Experimentation for AI Models
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
A/B-testing og eksperimentering er kritiske teknikker for å validere og optimalisere AI-modeller i produksjon. I motsetning til tradisjonell programvareutvikling, hvor funksjonalitet er binær (fungerer/fungerer ikke), er AI-modeller probabilistiske — ytelsen deres varierer med data, kontekst og bruksmønster. A/B-testing gjør det mulig å sammenligne modelversjoner, fine-tuning-strategier, prompt-varianter eller RAG-konfigurasjoner under reelle forhold, med ekte brukere og reell trafikk.
@ -438,3 +441,35 @@ Krever:
- Shadow deployment patterns
**Antall unike kilder:** 7 (Microsoft Learn) + 3 (baseline concepts) = **10 kilder**
### A/B Testing with Azure ML Managed Online Endpoints + MLflow 3 (2026)
**Traffic splitting via managed online endpoints**:
```bash
# Deploy challenger model with 10% traffic
az ml online-deployment create --name challenger --endpoint my-endpoint
az ml online-endpoint update --name my-endpoint --traffic control=90 challenger=10
# Monitor with MLflow 3 scorers — same metrics for both variants
# Use RelevanceToQuery, Correctness, custom business scorers
```
**MLflow 3 A/B evaluation pattern**:
- Use `mlflow.genai.evaluate()` on traces from each variant
- Compare scorers: `Correctness`, `RelevanceToQuery`, `ToolCallEfficiency`
- Statistical significance: MLflow tracks Cohen's Kappa against human baseline
- Aliases in Prompt Registry: `@control` and `@challenger` for prompt A/B testing
**Azure ML safe rollout progression**:
1. **Shadow testing**: Mirror X% of traffic to new model (no user impact)
2. **Canary**: Route 10% live traffic, monitor bake time (hours/days)
3. **Progressive**: 10% → 50% → 100% with health gate at each step
4. **Rollback trigger**: Automatic halt on health signal degradation
**Evaluation metrics for LLM A/B tests**:
- Quality: Groundedness, Relevance, Correctness (MLflow judges)
- Latency: P50, P90, P99 response times
- Cost: Token usage per request
- Business: Task completion rate, user satisfaction

58
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/azure-ml-pipelines-orchestration.md
View file

@ -1,11 +1,14 @@
# Azure ML Pipelines - Orchestration and Automation
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
Azure Machine Learning pipelines representerer et komplett orkestreringsrammeverk for machine learning-arbeidsflyter. En pipeline automatiserer en komplett ML-oppgave ved å dele den inn i flere håndterbare steg (components), hvor hvert steg kan utvikles, optimaliseres, konfigureres og automatiseres uavhengig. Azure ML håndterer dependencies mellom steg automatisk, og legger til rette for parallellisering, caching og gjenbruk.
@ -18,6 +21,61 @@ Fra et kostnads- og effektivitetsperspektiv gir pipelines betydelige fordeler: d
### Pipeline Components (v2)
### Azure ML Pipelines — Python SDK v2 (Tutorial 2026)
**Key benefits**: Standardized MLOps, scalable team collaboration, training efficiency, cost reduction.
**Pipeline creation pattern** (SDK v2):
```python
from azure.ai.ml import MLClient, dsl, Input, Output, command
from azure.identity import DefaultAzureCredential
ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)
# 1. Create reusable components
data_prep_component = command(
name="data_prep",
inputs={"data": Input(type="uri_folder"), "test_train_ratio": Input(type="number")},
outputs={"train_data": Output(type="uri_folder"), "test_data": Output(type="uri_folder")},
code="./components/data_prep",
command="python data_prep.py --data ${{inputs.data}} ...",
environment=f"{env.name}:{env.version}",
)
# Register for reuse
ml_client.create_or_update(data_prep_component.component)
# 2. Define pipeline with @dsl.pipeline decorator
@dsl.pipeline(compute="serverless", description="E2E training pipeline")
def training_pipeline(data_input, test_train_ratio, learning_rate, model_name):
prep_job = data_prep_component(data=data_input, test_train_ratio=test_train_ratio)
train_job = train_component(
train_data=prep_job.outputs.train_data,
test_data=prep_job.outputs.test_data,
learning_rate=learning_rate,
registered_model_name=model_name,
)
# 3. Submit pipeline
pipeline_job = ml_client.jobs.create_or_update(
training_pipeline(data_input=..., ...),
experiment_name="e2e_pipeline"
)
```
**Component lifecycle**:
1. Write YAML spec or create programmatically (`CommandComponent`)
2. Register with name+version in workspace or registry
3. Load and compose into pipeline
4. Submit via `ml_client.jobs.create_or_update()`
**Compute options**: `serverless` (recommended), named compute cluster, or per-step compute override.
**Environment**: Curated environments (`azureml://registries/azureml/environments/sklearn-1.5/labels/latest`) or custom conda/Docker.
**Output types**: `uri_folder` (data), `mlflow_model` (model), `uri_file` (file).
**MLflow integration**: Use `mlflow.start_run()` in scripts for automatic experiment tracking (metrics, parameters, models).
| Komponent-type | Beskrivelse | Bruksområde |
|----------------|-------------|-------------|
| **Command component** | Kjører et shell-script eller Python-script | Data prep, training, scoring, evaluation |

29
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/ci-cd-for-ml-models.md
View file

@ -1,6 +1,7 @@
# CI/CD Pipelines for Machine Learning Models
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
@ -288,6 +289,34 @@ Disse signalene indikerer at din ML CI/CD ikke er production-ready:
### GitHub Actions Integration
### GitHub Actions with Azure Machine Learning (2026 Update)
The recommended authentication approach is **OpenID Connect (OIDC) with federated credentials** — eliminates long-lived secrets.
**Workflow structure** (`/.github/workflows/`):
```yaml
permissions:
id-token: write
jobs:
build:
steps:
- uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
- run: az ml job create --file pipeline.yml
```
**MLOps v2 GitHub setup** (recommended end-to-end):
1. Fork `Azure/mlops-v2-gha-demo` template repo
2. Set GitHub secrets: `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_SUBSCRIPTION_ID`, `ARM_TENANT_ID`
3. Deploy infrastructure via `tf-gha-deploy-infra.yml` workflow
4. Run `deploy-model-training-pipeline` and `deploy-online-endpoint-pipeline` workflows
**Pipeline stages**: Prepare Data → Train Model → Evaluate Model → Register Model → Deploy Endpoint
**Setup:**
- Opprett `.github/workflows/` directory i repo
- Konfigurer GitHub Secrets for Azure credentials (eller OIDC)

31
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/data-drift-monitoring-detection.md
View file

@ -1,11 +1,14 @@
# Data Drift Monitoring and Detection
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
Data drift er endringer i statistisk fordeling av modellinput-data over tid som kan føre til forringet modellprestasjon. For machine learning-modeller er kontinuerlig overvåking av data drift avgjørende for å opprettholde produksjonskvalitet. Azure Machine Learning tilbyr innebygd drift detection som sammenligner produksjonsdata mot baseline-datasett (typisk treningsdata eller nylig produksjonsdata) og beregner statistiske avstandsmål.
@ -363,3 +366,31 @@ Hvis kunden bruker legacy `DataDriftDetector` (azureml-datadrift SDK):
**MCP Calls:** 5 (3 × microsoft_docs_search, 1 × microsoft_docs_fetch, 1 × microsoft_code_sample_search)
**Unique Sources:** 12 Microsoft Learn URLs
### Azure ML Model Monitoring — Data Drift Detection (2026)
**Model monitoring signals** (out-of-box for online endpoints):
| Signal | What it detects |
|--------|----------------|
| **Data quality** | Null values, out-of-range values, type mismatches in input features |
| **Data drift** | Statistical distribution change: training data vs production data |
| **Prediction drift** | Distribution shift in model output predictions |
| **Feature attribution drift** | Changes in which features drive predictions |
| **Custom signals** | User-defined metrics via Python scripts |
**Setup options**:
- **Out-of-box**: Automatically configured for Azure ML online endpoints (no configuration required)
- **Advanced**: Custom monitoring for models deployed outside Azure ML (batch endpoints, external)
- **Azure Event Grid integration**: Route monitoring alerts for automated response
**Statistical methods used**:
- Jensen-Shannon divergence for categorical features
- Wasserstein distance (Earth Mover's Distance) for numerical features
- Population Stability Index (PSI) for feature stability
**Reference dataset**: Training dataset used as baseline; monitoring compares production distribution against it.
**Alerting**: Configure thresholds per signal; integrate with Azure Monitor alerts and Action Groups.

31
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/feedback-loops-continuous-improvement.md
View file

@ -4,6 +4,8 @@
**Dato:** 2026-02-04
**Confidence:** HIGH (basert på offisiell Microsoft-dokumentasjon)
**Verified:** MCP 2026-04
## Introduksjon
Feedback loops og kontinuerlig forbedring er kritiske komponenter i moderne AI-operasjoner. I motsetning til tradisjonell programvare, hvor funksjonalitet er deterministisk, kan AI-modeller vise kvalitetsdrift eller uventet oppførsel når de møter reelle data. Et velfungerende feedback-system sikrer at modeller forblir nøyaktige, relevante og trygge gjennom hele sin livssyklus.
@ -738,3 +740,32 @@ Dette dokumentet dekker hele feedback loop-syklusen for både classical ML og Ge
5. **Kostnad:** Threshold-based retraining kan spare 50-70% compute vs daily retraining
Bruk arkitekturmønstrene til å visualisere løsningen for kunden. Påpek at MLflow Tracing + Agent Evaluation gir "free" observability (built-in i Databricks).
### MLflow 3 Evaluation & Feedback Loop (2026)
MLflow 3 introduces a unified evaluation-monitoring lifecycle for GenAI feedback loops:
**Iterative workflow**:
1. **Trace** production requests (MLflow Tracing — end-to-end observability)
2. **Evaluate** against scorers during development (`mlflow.genai.evaluate()`)
3. **Monitor** production with same scorers (consistent quality measurement)
4. **Gather human feedback** via Review App (expert annotations)
5. **Improve** prompts/models based on evaluation datasets
**Azure ML Model Monitoring signals**:
- Data quality: null values, out-of-range, type mismatch
- Data drift: statistical distribution changes between training and production data
- Prediction drift: distribution shift in model outputs
- Feature attribution drift: changes in feature importance
- Custom signals: user-defined metrics via custom scripts
**Monitoring setup**:
```python
# Set up out-of-box monitoring for Azure ML online endpoints
# Monitors data drift, prediction drift automatically
# Integrates with Azure Event Grid for alerting
```
**Continuous improvement cycle**: Production traces → MLflow evaluation datasets → Scorer alignment → Prompt/model update → A/B test → Production rollout

251
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/genaiops-llm-specific-practices.md
View file

@ -1,6 +1,7 @@
# GenAIOps - LLM-Specific MLOps Practices
**Dato:** 2026-02-04
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Kategori:** MLOps & GenAIOps
**Konfidensgrad:** Høy (basert på 18 MCP-kilder fra Microsoft Learn)
@ -10,252 +11,16 @@
GenAIOps (Generative AI Operations), også kalt LLMOps, beskriver operasjonelle praksiser og strategier for håndtering av store språkmodeller (LLMs) i produksjon. Mens tradisjonell MLOps fokuserer på å trene og deploye diskriminative modeller, handler GenAIOps om å **velge, tilpasse, orkestrere og overvåke** eksisterende foundation models.
### Forskjell mellom MLOps og GenAIOps
| Dimensjon | Tradisjonell MLOps | GenAIOps (LLMOps) |
|-----------|-------------------|-------------------|
| **Primært fokus** | Trene nye modeller fra scratch | Konsumere og fine-tune eksisterende foundation models |
| **Artefakter** | Trainede modeller (pkl, ONNX) | Prompts, orchestrators, agents, chains, grounding data |
| **Evaluering** | Accuracy, precision, recall (deterministiske) | Groundedness, relevance, coherence, fluency (LLM-as-judge) |
| **Infrastruktur** | Modell-serving endepunkter | Orchestrators, vector stores, API gateways, LLM endpoints |
| **Deployment** | Modellversjonering | Modell + prompt + grounding data + orchestrator |
| **Monitoring** | Model drift, data drift | Data drift + prompt effectiveness + content safety + token usage |
**Konfidensgrad:** 95% — Microsoft dokumentasjon definerer eksplisitt disse forskjellene.
### MLflow 3 Tracing — GenAI Observability
MLflow Tracing provides end-to-end observability for GenAI applications:
- Records inputs, outputs, intermediate steps, and metadata
- Supports complex agent-based systems and multi-turn conversations
- Integrates with Genie Code for natural language trace analysis
- Enables: debugging, performance monitoring, cost optimization, auditability
- Production monitoring reuses same scorers as development evaluation (consistent lifecycle)
---
## Kjernekomponenter
### 1. Prompt Engineering og Prompt Registry
**Hva:** Strukturert håndtering av system- og user prompts som versjonerte artefakter.
**Hvorfor:** Prompts er den primære "koden" i GenAI-løsninger. Endringer i prompts påvirker output like mye som kodeendringer.
**Hvordan (Azure):**
- **MLflow Prompt Registry** (Databricks): Versjonert prompt-håndtering med aliaser (f.eks. `production`, `staging`)
- **Azure AI Foundry Prompt Flow**: Visuell prompt designer med versjonering og CI/CD-integrasjon
- **Semantic Kernel Prompt Functions**: Prompts som code-artefakter i `.txt`-filer med Handlebars-syntax
```python
# MLflow Prompt Registry eksempel
import mlflow
prompt = mlflow.genai.register_prompt(
name="mycatalog.myschema.customer_support",
template="You are a helpful assistant. Answer this question: {{question}}",
commit_message="Initial customer support prompt"
)
mlflow.genai.set_prompt_alias(
name="mycatalog.myschema.customer_support",
alias="production",
version=1
)
# I applikasjon
prompt = mlflow.genai.load_prompt(
name_or_uri="prompts:/mycatalog.myschema.customer_support@production"
)
response = llm.invoke(prompt.format(question="How do I reset my password?"))
```
**Konfidensgrad:** 90% — Prompt Registry er dokumentert, men adoption rates varierer.
### 2. Orchestration Layer
**Hva:** Systemet som håndterer logikk, kaller datakilder/agenter, genererer prompts og kaller LLM-modeller.
**Hvorfor:** Generative AI-løsninger er ikke bare modellen — de er komplekse workflows som krever orkestrering.
**Microsoft-alternativer:**
- **Azure AI Foundry Agent Service**: Low-code agent-orkestrering
- **Microsoft Agent Framework SDK (Semantic Kernel)**: Code-first orkestrering med C#/Python
- **Prompt Flow**: Visuell workflow-designer for LLM-chains
- **LangChain/LlamaIndex**: Open source (støttes av Azure ML)
**Deployment:**
- Azure App Service (containerized orchestrator)
- Azure Container Apps (serverless orchestrator)
- Azure Kubernetes Service (high-scale orchestrator)
- Azure Machine Learning Managed Online Endpoints
**Konfidensgrad:** 85% — Mange deployment-alternativer, best practice varierer med use case.
### 3. Vector Stores og Grounding Data
**Hva:** Datalagringsløsninger for RAG (Retrieval-Augmented Generation) som støtter vektor-søk.
**Azure-alternativer:**
- **Azure AI Search**: Hybrid search (full-text + vector + semantic)
- **Azure Cosmos DB for MongoDB vCore**: Vector search capabilities
- **Azure Database for PostgreSQL (pgvector)**: Open source vector extension
- **Databricks Vector Search**: Delta table-basert, auto-syncing
**DataOps-utvidelser for GenAIOps:**
- **Chunking pipelines**: Split dokumenter i semantisk meningsfulle chunks (Azure Machine Learning pipelines)
- **Embedding generation**: Batch-generering av embeddings (Azure OpenAI text-embedding-ada-002 / text-embedding-3-small)
- **Index maintenance**: Incremental updates vs. full rebuilds (compliance: right-to-be-forgotten)
- **Data freshness**: Real-time vs. batch refresh (business requirements)
**Konfidensgrad:** 90% — Dokumentert arkitektur, men chunking-strategier er eksperimentelle.
### 4. Evaluation Framework
**Hva:** LLM-spesifikke evalueringsmetrikker og human-in-the-loop feedback.
**Azure AI Foundry Evaluation SDK:**
```python
from azure.ai.evaluation import evaluate, RelevanceEvaluator, CoherenceEvaluator
model_config = {
"azure_endpoint": os.environ.get("AZURE_OPENAI_ENDPOINT"),
"api_key": os.environ.get("AZURE_OPENAI_KEY"),
"azure_deployment": os.environ.get("AZURE_OPENAI_DEPLOYMENT"),
}
result = evaluate(
data="test_data.jsonl",
evaluators={
"relevance": RelevanceEvaluator(model_config=model_config),
"coherence": CoherenceEvaluator(model_config=model_config),
},
evaluator_config={
"relevance": {
"column_mapping": {
"query": "${data.query}",
"ground_truth": "${data.ground_truth}",
"response": "${outputs.response}"
}
}
},
azure_ai_project=azure_ai_project,
output_path="./evaluation_results.json"
)
```
**Evaluerings-dimensjoner:**
| Use case | Metrikker |
|----------|-----------|
| **RAG** | Groundedness, relevance, coherence, fluency |
| **Summarization** | ROUGE, BLEU, BERTScore, METEOR |
| **Translation** | BLEU |
| **Classification** | Precision, recall, accuracy, F1 |
| **Content Safety** | Hate/violence/sexual/self-harm scores (Azure AI Content Safety) |
**Human Feedback Loop:**
- **Mosaic AI Agent Framework Review App** (Databricks): UI for human reviewers
- **Application Insights**: Thumbs up/down fra sluttbrukere
- **Custom feedback APIs**: Integrasjon i enterprise workflows
**Konfidensgrad:** 95% — Built-in evaluators er godt dokumentert.
### 5. CI/CD for GenAIOps
**GenAIOps Prompt Flow Template** (Microsoft-anbefalt):
- **Repository**: [microsoft/genaiops-promptflow-template](https://github.com/microsoft/genaiops-promptflow-template)
- **CI/CD**: GitHub Actions eller Azure DevOps Pipelines
- **Lifecycle**: Feature branch → PR → Dev → Staging → Production
**Pipeline-faser:**
1. **PR Pipeline** (CI):
- Flow validation
- Unit testing av custom Python code
- Variant experimentation
- Evaluation runs mot test data
2. **Dev Pipeline** (CI + CD):
- Batch testing
- Model/prompt registration (conditional)
- Human-in-the-loop approval gate
- Deployment til dev/staging endpoints
3. **Production Pipeline** (CD):
- Blue-green deployment
- A/B testing (traffic splitting)
- Canary deployment
- Rollback capabilities
**Azure DevOps-integrasjon:**
```yaml
# Eksempel: Prompt Flow evaluation i Azure Pipelines
- task: AzureCLI@2
displayName: 'Run Prompt Flow Evaluation'
inputs:
azureSubscription: 'AzureML-ServiceConnection'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
az ml job create --file evaluation-job.yaml \
--workspace-name $(ML_WORKSPACE) \
--resource-group $(RESOURCE_GROUP)
```
**Konfidensgrad:** 85% — Template er aktiv (2025), men requires customization.
### 6. Monitoring og Observability
**LLM-spesifikke overvåkningsdimensjoner:**
| Dimensjon | Hva overvåkes | Azure-verktøy |
|-----------|---------------|---------------|
| **Operational** | Latency, token usage, 429 errors, endpoint availability | Azure Monitor, Application Insights |
| **Quality** | Groundedness, relevance, coherence, fluency (sampled) | Azure Machine Learning Model Monitoring (Generation Quality Signal) |
| **Safety** | Harmful content detection (hate, violence, sexual, self-harm) | Azure AI Content Safety (real-time filtering) |
| **Cost** | Token consumption per user/session, quota utilization | Azure Cost Management, API Management gateway logs |
| **Data drift** | Changes in user query patterns, grounding data staleness | Azure ML Data Drift monitors |
| **Feedback** | User ratings (thumbs up/down), session abandonment rate | Custom telemetry (Application Insights) |
**MLflow Tracing for GenAI:**
```python
import mlflow
# Automatisk tracing av OpenAI calls
mlflow.openai.autolog()
# Custom trace decorators
@mlflow.trace
def my_rag_app(query: str):
context = retrieve_from_vector_store(query)
prompt = format_prompt(query, context)
response = llm.invoke(prompt)
return response
```
**Azure AI Foundry Monitoring (SDK v2):**
```python
from azure.ai.ml.entities import (
MonitorSchedule, GenerationSafetyQualitySignal,
GenerationTokenStatisticsSignal
)
# Quality monitoring
gsq_signal = GenerationSafetyQualitySignal(
connection_id=aoai_connection_id,
metric_thresholds={
"groundedness": {"aggregated_groundedness_pass_rate": 0.7},
"relevance": {"aggregated_relevance_pass_rate": 0.7},
},
production_data=[production_data],
sampling_rate=1.0
)
# Token monitoring
token_signal = GenerationTokenStatisticsSignal()
monitor = MonitorSchedule(
name="genai-monitor",
trigger=CronTrigger(expression="15 10 * * *"),
create_monitor=MonitorDefinition(
monitoring_signals={"quality": gsq_signal, "tokens": token_signal}
)
)
```
**Konfidensgrad:** 90% — Monitoring capabilities er dokumentert, men sampling rates må justeres for cost.
---
## Arkitekturmønstre
### 1. Fine-Tuning Pattern

35
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/inferencing-optimization-caching.md
View file

@ -4,6 +4,8 @@
**Dato:** 2026-02-04
**Forfattet av:** Cosmo Skyberg, Senior Microsoft AI Solution Architect
**Verified:** MCP 2026-04
## Introduksjon
Inferencing optimization og caching representerer kritiske teknikker for å maksimere ytelse og minimere kostnader når AI-modeller skal serve prediksjoner i produksjon. Mens model training handler om å oppnå høy accuracy, handler inferencing om å levere disse prediksjonene raskt, pålitelig og kostnadseffektivt til brukere og systemer.
@ -1011,3 +1013,36 @@ Diagnostikk:
- Monitor **cache hit rate** og **autoscaling metrics** kontinuerlig
**Confidence nivå: HIGH** — Denne referansen er basert på 12 MCP-kall til offisiell Microsoft-dokumentasjon og kodeeksempler.
### ONNX Inferencing Optimization for Computer Vision (Azure ML AutoML 2026)
ONNX (Open Neural Network Exchange) enables cross-framework interoperability and inference optimization:
**Supported AutoML computer vision tasks**:
- Image classification (binary and multi-class)
- Object detection
- Instance segmentation
**ONNX inference workflow**:
1. Download ONNX model files from AutoML training run
2. Understand model inputs/outputs (image format requirements)
3. Preprocess data to required input format
4. Run inference with ONNX Runtime Python API (`onnxruntime`)
5. Post-process predictions (bounding boxes for detection, masks for segmentation)
**Python ONNX Runtime**:
```python
import onnxruntime as rt
sess = rt.InferenceSession("model.onnx")
# Works across languages: Python, C++, C#, Java, JavaScript
```
**Cross-platform benefits**:
- Deploy on any platform without framework dependencies
- Reduced inference latency vs Python framework
- Edge deployment: Azure IoT Edge, on-premises
- Language flexibility post-export
**SDK**: `azure-ai-ml v2 (current)` — use AutoML image tasks to generate ONNX models automatically

38
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md
View file

@ -4,6 +4,8 @@
**Kategori:** MLOps & GenAIOps
**Forfatter:** Cosmo Skyberg, Senior Microsoft AI Solution Architect
**Verified:** MCP 2026-04
## Introduksjon
Infrastructure as Code (IaC) er en fundamental MLOps-praksis der infrastruktur defineres og deployes gjennom kode fremfor manuelle konfigurasjoner. Dette er kritisk viktig for AI/ML-prosjekter fordi det sikrer reproducerbarhet, konsistens og versjonskontroll av hele ML-miljøet — fra development til production.
@ -539,6 +541,42 @@ resource mlWorkspace 'Microsoft.MachineLearningServices/workspaces@2024-01-01-pr
### IaC-verktøy kostnader
### IaC Design for MLOps — Azure Well-Architected (OE:05) 2026
**Core principle** (Well-Architected OE:05): Standardized IaC approach with declarative syntax, consistent styles, appropriate modularization, quality assurance.
**Declarative over imperative** (recommended):
- Bicep / ARM templates: Azure-native, JSON/DSL declarative
- Terraform: Industry-standard, multi-cloud declarative
- Avoid: imperative scripts for infrastructure state management
**Azure-native tools**:
```bash
# Bicep — deploy Azure ML workspace
az deployment group create --template-file ml-workspace.bicep
# Terraform — integrated into GitHub Actions / Azure Pipelines
terraform init && terraform apply
```
**Layered IaC pipeline approach for MLOps**:
- **Low-touch** (networking, VNet, ACR): Rarely changes, stable baseline
- **Medium-touch** (compute clusters, storage, AKS): Occasional changes
- **High-touch** (model endpoints, deployments): Continuous delivery
**IaC best practices**:
- Treat IaC artifacts the same as application code (version control, PR reviews, testing)
- Use parameters/variables for multi-environment support (dev/test/prod)
- Collocate IaC with application code for synchronized deployments
- Scan IaC repos for secrets (Microsoft Defender for Cloud: IaC vulnerability scanning)
- Immutable infrastructure preferred for business-critical workloads
**AI opportunity** (2026): AI tools (GitHub Copilot) can review IaC templates, identify misconfigurations, suggest security improvements, and generate templates from natural language.
**MLOps v2 infrastructure**: `tf-gha-deploy-infra.yml` workflow in `Azure/mlops-v2-gha-demo` deploys full Azure ML infrastructure via Terraform + GitHub Actions.
| Verktøy | Lisens | Kostnad |
|---------|--------|---------|
| **Bicep** | Open source (MIT) | Gratis |

34
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/llm-evaluation-production.md
View file

@ -6,6 +6,8 @@
---
**Verified:** MCP 2026-04
## Introduksjon
LLM-evaluering i produksjonsmiljø er fundamentalt forskjellig fra tradisjonell ML-evaluering. Mens klassiske ML-modeller evalueres med deterministiske metrikker på statiske test-sett, krever generative AI-applikasjoner kontinuerlig evaluering av åpne, ikke-deterministiske output i dynamiske produksjonsscenarioer.
@ -558,6 +560,38 @@ project = AIProjectClient.from_connection_string(
### MLflow 3 + Databricks Unity Catalog
### MLflow 3 LLM Evaluation Framework (2026)
MLflow 3 (SDK `mlflow[databricks]>=3.1`) introduces a unified evaluation model:
**Core architecture**: Traces → Scorers → Feedback
- Traces from `mlflow.genai.evaluate()` or production monitoring service
- Scorers parse traces, assess quality, return `Feedback` objects
- Same scorers used in development AND production (consistent lifecycle)
**Built-in LLM judges** (research-validated):
| Judge | Needs Ground Truth | Evaluates |
|-------|-------------------|-----------|
| `RelevanceToQuery` | No | Response relevance to user request |
| `RetrievalGroundedness` | No | Hallucination detection |
| `Safety` | No | Harmful/toxic content |
| `Correctness` | Yes | Accuracy vs ground truth |
| `Completeness` | Yes | All questions addressed |
| `ToolCallCorrectness` | Yes | Tool calls and arguments |
| `ToolCallEfficiency` | No | Redundant tool usage |
| `Guidelines` | No | Custom natural-language rules |
**Multi-turn judges** (conversation-level): `ConversationCompleteness`, `UserFrustration`, `KnowledgeRetention`, `ConversationalSafety`
**Production monitoring**: Automatically runs scorers on production traces; uses Databricks-hosted LLM judges (EU workspaces: EU-hosted models). No prompts stored with Azure OpenAI (Abuse Monitoring opt-out).
**Custom judges**: Full control over evaluation criteria, scores (numerical/categorical/boolean), human feedback alignment via `align_judges()`.
**Key note**: MLflow 3 replaced Agent Evaluation SDK — migrate with `mlflow.genai.*` functions.
**Enterprise governance for AI:**
```

46
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/mlops-fundamentals-overview.md
View file

@ -1,11 +1,14 @@
# MLOps Fundamentals - Lifecycle and Principles
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
Machine Learning Operations (MLOps) er anvendelse av DevOps-prinsipper på machine learning-prosjekter. Målet er å automatisere og effektivisere hele ML-livssyklusen – fra eksperimentering og trening, via deployment, til overvåking og retrening. MLOps bygger på etablert DevOps-praksis som continuous integration (CI), continuous deployment (CD), version control, og infrastructure as code (IaC), men legger til ML-spesifikke utfordringer som data versioning, model tracking, feature engineering, og drift detection.
@ -292,6 +295,49 @@ jobs:
### DevOps-verktøy
### DevOps for Machine Learning — Azure DevOps Integration (2026)
**Azure Pipelines + Azure ML** (how-to-devops-machine-learning):
Automate the ML lifecycle via Azure DevOps pipelines:
1. Data preparation (ETL)
2. On-demand scale-out training
3. Model deployment (public/private web service)
4. Monitoring (performance, data drift)
**Azure DevOps pipeline YAML pattern**:
```yaml
- task: AzureCLI@2
inputs:
azureSubscription: $(service-connection)
inlineScript: |
job_name=$(az ml job create --file pipeline.yml -g $(resource-group) -w $(workspace) --query name -o tsv)
echo "##vso[task.setvariable variable=JOB_NAME;isOutput=true;]$job_name"
- job: WaitForJobCompletion
pool: server # Server job — no agent costs
steps:
- task: AzureMLJobWaitTask@1 # From Azure ML extension
inputs:
serviceConnection: $(service-connection)
azureMLJobName: $(azureml_job_name)
```
**Authentication options**:
- Azure Resource Manager service connection (recommended with Azure ML extension)
- Generic service connection (uses InvokeRESTAPI task)
**MLOps maturity model**: Manual → Partial automation → Full CI/CD → Full MLOps with monitoring
**Key automation operations** (Azure DevOps):
- Infrastructure deployment (Terraform / Bicep)
- Component registration and versioning
- Model training on compute clusters
- Online/batch endpoint deployment
- Production monitoring alerts
| Verktøy | Kostnad | Anbefaling |
|---------|---------|-----------|
| **Azure DevOps** | Gratis for 5 brukere + 1800 min/mnd pipeline | Bruk Basic plan for mindre team |

30
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md
View file

@ -1,7 +1,8 @@
# Security and Access Control in MLOps
**Kategori:** MLOps & GenAIOps
**Dato:** 2026-02-04
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Dato:** 2026-04-10
**Confidence:** HIGH — Basert på offisiell Microsoft Learn dokumentasjon (8 MCP-oppslag, 16 kilder)
---
@ -747,3 +748,30 @@ AmlComputeClusterNodeEvent
- ✅ HIGH confidence: Offisiell dokumentasjon + kodeeksempler fra Microsoft Learn
- ⚠️ MEDIUM confidence: Utledet fra best practices og architecture patterns
- ❓ LOW confidence: Ikke aktuelt (alle påstander er verifisert mot offisiell dokumentasjon)
### Azure Machine Learning VNet Security (2026 Update)
**Managed Virtual Networks** (recommended approach): Azure ML handles network isolation automatically.
Use `az ml workspace update` with managed network settings instead of manual VNet configuration.
**Private Endpoint for Workspace**:
- Connects workspace via private IP addresses within your VNet
- Requires securing all dependent resources: Storage, Key Vault, Container Registry
- Private endpoint alone does NOT ensure end-to-end security — all components must be secured
**Storage Account Security**:
- Private endpoint (recommended): Blob, File, Queue, Table subresources
- Service endpoint: Must be same VNet and subnet as compute
- Set `Microsoft.MachineLearningServices/Workspace` as trusted resource type
**Required outbound traffic service tags**:
- `AzureActiveDirectory` (TCP 443) — authentication
- `AzureMachineLearning` (TCP 443, 18881, UDP 5831)
- `Storage.region` (TCP 443) — data access
- `MicrosoftContainerRegistry.region` (TCP 443) — Docker images
**Secure connectivity options**: Azure VPN Gateway (Point-to-site/Site-to-site), ExpressRoute, Azure Bastion (jump box)
**ACR requirements**: Premium SKU required for private endpoints; ACR must be in same VNet or peered VNet.

28
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md
View file

@ -5,6 +5,8 @@
**Kilde:** Microsoft Learn, Azure Architecture Center
**Konfidensgradering:** ⭐⭐⭐⭐⭐ (Verifisert mot offisiell Microsoft-dokumentasjon)
**Verified:** MCP 2026-04
## Introduksjon
Vellykkede MLOps-implementeringer krever samarbeid mellom flere teamroller med ulike verktøy, arbeidsflyter og ansvar. Denne referansen dekker hvordan ulike personas samarbeider gjennom machine learning-livssyklusen, hvilke verktøy som støtter samarbeid, og hvordan organisasjoner kan strukturere teamarbeid for maksimal effektivitet.
@ -123,6 +125,32 @@ MLOps-miljøer opererer med distinkte roller som hver har spesifikke ansvarsomr
**Konfidensmarkør:** ⭐⭐⭐⭐⭐ Azure Boards er core DevOps-plattform med native Azure DevOps-integrasjon.
#### Azure DevOps / GitHub Actions
### Azure DevOps — Integrated MLOps Platform (2026)
Azure DevOps provides end-to-end project management for ML teams:
| Service | ML Use Case |
|---------|-------------|
| **Azure Boards** | Sprint planning for model iterations, bug tracking, backlog management |
| **Azure Repos** | Git repositories for model code, notebooks, IaC; branch policies + PR reviews |
| **Azure Pipelines** | CI/CD for ML (build, test, train, deploy); integrates with Azure ML via `AzureMLJobWaitTask@1` |
| **Azure Test Plans** | Manual testing of model outputs, test case management |
| **Azure Artifacts** | Package feeds (NuGet, pip, conda) for ML libraries and shared components |
**Azure DevOps MCP Server**: Natural language queries for project management — `Summarize sprint status`, `List blocked work items`, `Show pipeline success rates` (2026 feature).
**GitHub Actions integration** (alternative to Azure Pipelines):
- OIDC authentication (recommended, no long-lived secrets)
- `azure/login@v2` + `az ml job create` pattern
- MLOps v2 solution accelerator: `Azure/mlops-v2-gha-demo`
**Databricks CI/CD best practices**:
- Feature branching with short-lived branches
- Automated notebook testing before merge
- MLflow experiment tracking integrated into PR workflows
**Formål:** CI/CD automation for ML lifecycle
**Nøkkelkapabiliteter:**
- Pipeline-basert workflow automation

35
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md
View file

@ -5,6 +5,8 @@
**Målgruppe:** Arkitekter som planlegger ML-modellutplassering i produksjon
**Konfidensgrad:** ⚡️⚡️⚡️ Høy (basert på Microsoft Learn + offisielle code samples)
**Verified:** MCP 2026-04
## Introduksjon
Model deployment strategies handler om hvordan man trygt og effektivt ruller ut nye ML-modeller eller modellversjoner til produksjon uten å forårsake nedetid eller forringet brukeropplevelse. Azure Machine Learning tilbyr flere deployment patterns som støtter **progressive exposure**, **traffic routing**, og **rollback-mekanismer**.
@ -1050,3 +1052,36 @@ Denne kunnskapsreferansen er basert på følgende Microsoft Learn-artikler og co
**Sist oppdatert:** 2026-02-04
**Neste review:** 2026-05-04 (eller ved større endringer i Azure ML deployment capabilities)
### Safe Rollout / Blue-Green Deployment (Azure Well-Architected 2026)
Azure ML managed online endpoints support blue-green (safe rollout) deployments natively:
```bash
# Deploy green deployment with 0% traffic initially
az ml online-deployment create --name green --endpoint my-endpoint --traffic-allocation 0
# Test green deployment in isolation (direct routing)
az ml online-endpoint invoke --name my-endpoint --deployment-name green
# Mirror 10% of live traffic to green for shadow testing
# Then progressively shift: 10% → 50% → 100%
az ml online-endpoint update --name my-endpoint --traffic blue=90 green=10
```
**Azure Well-Architected SDP principles (OE:11)**:
- **Progressive exposure**: Canary → Blue-Green → Deployment Stamps
- **Health models**: Pass health checks before each rollout phase
- **Bake time**: Hours/days between phases (not minutes) to capture time-zone usage patterns
- **Failure detection**: Automatic halt + investigation when health signals degrade
- **Recovery options**: Roll back (revert), roll forward (hotfix), or redeploy last known good
**Azure facilitation**:
- `Azure Pipelines` + `GitHub Actions` support multi-stage deployments with approval gates
- `Azure App Configuration` for feature flag management
- `Azure Load Balancers` for traffic routing and health monitoring
- Point-in-time restore available for Azure SQL, Cosmos DB, MySQL, PostgreSQL
**Emergency SDP**: Prescriptive protocols for hotfix acceleration — approval stage and bake time reduction — with explicit approval criteria.

39
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/model-drift-performance-degradation.md
View file

@ -1,11 +1,14 @@
# Model Drift and Performance Degradation Detection
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
Model drift og performance degradation er kritiske fenomener som oppstår når en maskinlæringsmodells ytelse forverres over tid i produksjon. Dette skjer fordi virkeligheten endrer seg – input-data får andre distribusjoner, forretningslogikk endres, sensorer kalibreres feil, eller brukernes atferd endrer seg. Uten kontinuerlig overvåking kan modeller raskt bli utdaterte og levere feil prediksjoner som undergraver forretningsmål eller skaper compliance-problemer i regulerte sektorer.
@ -633,3 +636,39 @@ Email Alerts + Azure Monitor Dashboard
### Sist oppdatert
**2026-02** – Basert på Microsoft Learn-dokumentasjon (azure-ai-ml SDK v2, API version 2).
### Azure ML Model Drift & Performance Degradation Monitoring (2026)
**Model monitoring** provides continuous tracking of production model performance:
**Degradation signals**:
- **Prediction drift**: Output distribution shifts away from training baseline
- **Feature attribution drift**: Feature importance changes indicate concept drift
- **Data quality degradation**: Input data quality issues upstream
- **Performance metric degradation**: Track against ground truth when labels available
**Monitoring configuration**:
```python
# Set up monitoring for deployed models on online endpoints
# Azure ML handles data collection and signal computation
# Monitoring jobs run on schedule (default: daily)
```
**Alert thresholds** (recommended):
- Data drift coefficient > 0.1: Investigate
- Data drift coefficient > 0.3: Retrain trigger
- Prediction drift > 15%: Production alert
- Unusable nodes > 0: Infrastructure alert (Azure Monitor)
**Continuous learning loop**:
1. Monitor signals → detect drift early
2. Critically evaluate inherent model risks
3. Identify hidden problems before business impact
4. Trigger retraining or model update workflow
5. Validate new model before rollout (blue-green/canary)
**Integration**: Azure Event Grid for alerting → Logic Apps / Functions → automated retraining trigger
**For GenAI/LLM**: MLflow 3 production monitoring reuses development scorers (Groundedness, Relevance) on production traces — consistent quality measurement throughout lifecycle.

39
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md
View file

@ -1,11 +1,14 @@
# Model Evaluation Frameworks and Metrics
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
Evaluering av AI-modeller, spesielt generative AI-applikasjoner, krever en helt annen tilnærming enn tradisjonell maskinlæring. Mens tradisjonell ML fokuserer på deterministiske metrikker som accuracy og precision, må GenAI-evaluering håndtere multi-turn-samtaler, kontekstuell relevans, sikkerhet og subjektiv kvalitet. Microsoft tilbyr et omfattende rammeverk for modellevaluering gjennom Azure AI Foundry, Azure Machine Learning Prompt Flow og MLflow 3, som dekker hele utviklingsløpet fra modellvalg til produksjonsovervåking.
@ -107,6 +110,42 @@ print(f"Foundry URL: {result.get('studio_url')}")
### MLflow 3 Evaluation & Monitoring
### MLflow 3 Evaluation Framework (2026)
MLflow 3 provides the evaluation framework for both traditional ML and GenAI applications on Databricks:
**Scorer types** (unified interface for all evaluation):
| Type | Customization | Use Case |
|------|--------------|---------|
| Built-in judges | Minimal | Quick evaluation: `Correctness`, `RetrievalGroundedness`, `Safety` |
| Guidelines judges | Moderate | Custom natural-language rules (pass/fail) |
| Custom LLM judges | Full | Domain-specific criteria, detailed scoring |
| Code-based scorers | Full | Deterministic: exact match, format validation, business logic |
**Key evaluation functions**:
```python
import mlflow
# Development evaluation
results = mlflow.genai.evaluate(
data=eval_dataset,
scorers=[RelevanceToQuery(), RetrievalGroundedness(), Correctness()]
)
# Production monitoring — same scorers as development
# Automatically applied to production traces
```
**Judge accuracy**: Databricks validates with Cohen's Kappa, accuracy, F1 score against human expert judgment.
**Traditional ML evaluation** (Azure ML):
- Data quality signals: null rate, out-of-bounds, type errors
- Statistical drift: Jensen-Shannon divergence, Wasserstein distance
- Custom metrics via Python scripts in monitoring jobs
MLflow 3 integrerer evaluering og production monitoring i én workflow. Samme LLM judges og scorers kan brukes i development, testing og production.
**Hovedkomponenter:**

41
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md
View file

@ -1,11 +1,14 @@
# Model Versioning and Registry Management
**Last updated:** 2026-02
**Verified:** MCP 2026-04
**Status:** GA
**Category:** MLOps & GenAIOps
---
**Verified:** MCP 2026-04
## Introduksjon
Model versioning og registry management er fundamentale komponenter i MLOps-livssyklusen som sikrer sporbarhet, reproduserbarhet og effektiv styring av maskinlæringsmodeller gjennom hele deres levetid. Azure Machine Learning tilbyr to primære tilnærminger: workspace model registry for team-intern bruk og Azure Machine Learning registry for tverrorganisatorisk deling. Begge støtter MLflow som standardformat, noe som gir portabilitet og integrasjon med et bredt økosystem av verktøy.
@ -31,6 +34,44 @@ Azure Machine Learning skiller seg fra tradisjonelle Git-baserte tilnærminger v
### Registry-typer sammenlignet
### Azure Machine Learning Cross-Workspace Registry (2026)
**Azure ML Registry** enables model, component, and environment sharing across workspaces and subscriptions:
**Two primary scenarios**:
1. **Cross-workspace MLOps**: Train in `dev` → deploy to `test`/`prod` with full lineage (code, data, environment)
2. **Cross-team sharing**: Publish models/components to central catalog for reuse across teams
**Registry operations** (CLI v2 / Python SDK v2):
```bash
# Create model in registry (from local files)
az ml model create --name nyc-taxi-model --version 1 --type mlflow_model --path ./artifacts/model/ --registry-name <registry-name>
# Share model from workspace to registry (preserves training lineage)
az ml model share --name nyc-taxi-model --version 1 --registry-name <registry-name> --share-with-name <new-name> --share-with-version 1
# Deploy model from registry to any workspace
# (model: azureml://registries/<registry-name>/models/<name>/versions/<v>)
az ml online-deployment create --file deploy.yml --all-traffic
```
**Python SDK pattern**:
```python
ml_client_registry = MLClient(credential=credential, registry_name="<REGISTRY_NAME>")
ml_client_workspace = MLClient(credential=credential, workspace_name="<WS_NAME>", ...)
# Create in registry
ml_client_registry.models.create_or_update(mlflow_model)
# Deploy from registry to workspace
ml_client_workspace.online_deployments.begin_create_or_update(deployment)
```
**Lineage tracking**: Models registered from job outputs link back to training job, code, data, and environment.
**Access control**: ACR token-based access (workspace compute has `AcrPull` via registry's managed identity).
**MLflow format**: Required for no-code deployment with built-in scoring server.
| Egenskap | Workspace Registry | Azure ML Registry |
|----------|-------------------|-------------------|
| **Scope** | Enkelt workspace | Multi-workspace, cross-subscription |

41
plugins/ms-ai-architect/skills/ms-ai-engineering/references/mlops-genaiops/monitoring-observability-ml-systems.md
View file

@ -7,6 +7,8 @@
---
**Verified:** MCP 2026-04
## Introduksjon
Monitoring og observability for ML-systemer handler om kontinuerlig overvåkning av modeller i produksjon for å sikre ytelse, kvalitet og pålitelighet. Azure tilbyr et komplett økosystem for ML-overvåkning gjennom Azure Machine Learning Model Monitoring og Azure Monitor, som til sammen gir innsikt i både **modellytelse** (data science-perspektiv) og **operasjonell helse** (infrastruktur-perspektiv).
@ -297,6 +299,45 @@ create_monitor:
### Azure Monitor
### Azure Machine Learning Monitoring Architecture (2026)
**Azure Monitor integration**:
- All metrics in namespace: `Machine Learning Service Workspace`
- Platform metrics collected automatically, no configuration needed
- Route resource logs to Log Analytics for querying with KQL
**Key Kusto (KQL) queries**:
```kusto
# Failed jobs last 5 days
AmlComputeJobEvent
| where TimeGenerated > ago(5d) and EventType == "JobFailed"
| project TimeGenerated, ClusterId, EventType, ExecutionState, ToolType
# Failed online endpoint requests
AmlOnlineEndpointTrafficLog
| where TimeGenerated > ago(1d) and ResponseCode != 200
| project TimeGenerated, EndpointName, DeploymentName, ResponseCode
```
**Recommended alert rules**:
| Alert | Condition | Threshold |
|-------|-----------|-----------|
| Model Deploy Failed | Total > 0 | Any failure |
| Quota Utilization | Average > 90% | High utilization |
| Unusable Nodes | Total > 0 | Any unusable |
**Application Insights integration**: Live metrics, Transaction search, Failures, Performance analysis.
Use workspace-based Application Insights (default for new workspaces) + Azure Monitor Private Link for VNet isolation.
**Data storage layers**:
- Metrics database: Platform metrics (near real-time)
- Log Analytics: Resource logs + Activity log (queryable with KQL)
- Azure Storage / Event Hubs: Long-term export
**Cross-workspace monitoring**: Use single Log Analytics workspace for multiple Azure ML workspaces to query across all resources simultaneously.
**Application Insights** (📊 For endpoints):
```python
# Enable for online endpoint

23
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/agentic-rag-patterns.md
View file

@ -1,6 +1,6 @@
# Agentic RAG Patterns — Agent-styrt retrieval
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Semantic Kernel), Preview (Azure AI Search agentic retrieval)
**Category:** RAG Architecture & Semantic Search
@ -285,3 +285,24 @@ agent = chat_client.as_agent(
| AI Agent Design Patterns | **Verified** | [learn.microsoft.com](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns) |
| Semantic Kernel Agent Orchestration | **Verified** | [learn.microsoft.com](https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/agent-orchestration/) |
| Multi-agent performance (34% accuracy) | **Baseline** | Community source (ragaboutit.com) |
### Azure AI Search Agentic Retrieval (Public Preview — oppdatert 2026-04)
Azure AI Search agentic retrieval er en managed multi-query pipeline for komplekse spørsmål i chat og copilot-apper:
**Funksjonalitet:**
- LLM (gpt-4o/4.1/5-serien) bryter ned komplekse queries til fokuserte subqueries
- Subqueries kjøres **parallelt** med semantisk reranking per query
- Resultater slås sammen til ett grounding data-sett med query plan og source documents
- Leser inn chat history for kontekstuell query planning
**Prising:**
- Free plan: **50 millioner gratis reasoning tokens/mnd** (alle tiers)
- Standard plan: pay-as-you-go etter fri kvote
- Avhenger av semantic ranker (premium feature)
**Arkitektur:** Knowledge Base + Knowledge Source(s) + Azure OpenAI LLM + Azure AI Search index
**AI Agent Design Patterns (Azure Architecture Center):**
Agentic RAG plasseres i et spektrum fra single model call → single agent with tools → multi-agent orchestration. Start med laveste nødvendige kompleksitetsnivå. Mønstre: sequential (pipeline), parallel fanout, supervisor, og autonomous loop. Multi-agent krever koordineringsoverhead og økt latency — bruk kun når single-agent RAG ikke er tilstrekkelig.

18
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/azure-ai-search-setup.md
View file

@ -1,6 +1,6 @@
# Azure AI Search - Configuration and Deployment
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -34,6 +34,10 @@ For offentlig sektor er Azure AI Search spesielt relevant fordi den støtter dat
- **Semantic Ranker:** Kun Standard og høyere (S1+)
- **Private Link:** Standard (S1+), Storage Optimized
> **SKU-oppdatering 2026-04:** Search services opprettet etter 3. april 2024 har større partisjoner og høyere vector kvoter på nesten alle tiers. Basic støtter 3 replicas for SLA. Standard (S1-S3) er standard valg. S3 HD er hosting mode for mange små indekser (multitenancy). Storage Optimized (L1/L2) gir lavere pris/TB for sjeldent oppdaterte, store indekser — med høyere query latency. Tier-bytte er nå mulig mellom Basic og Standard (S1/S2/S3) uten å gjenoppbygge indeksen fra scratch i mange tilfeller.
### Indekseringsstrategier
Azure AI Search støtter tre indekseringsmodeller:
@ -462,3 +466,15 @@ Azure AI Search prises per **search unit** (SU = 1 partition × 1 replica).
---
**For Cosmo:** Denne referansen brukes når kunden snakker om "RAG-implementasjon", "søkeløsning", "Azure AI Search setup", eller spør om SKU-valg. Kombiner med **RAG Core Patterns** for arkitekturveiledning og **Hybrid Search - Full-Text and Vector Combined** for query-optimalisering.
### Hybrid Search (oppdatert 2026-04)
Hybrid search kombinerer full-text search og vector search i én enkelt forespørsel mot en søkeindeks med både tekstlig og vektorisert innhold:
- Kjører full-text og vector search **parallelt**
- Merger resultater med **Reciprocal Rank Fusion (RRF)**
- Støtter filtrering, faceting, sortering, scoring profiles og semantic ranking i én request
- `maxTextRecallSize` (preview) kontrollerer antall BM25-resultater inn til RRF-ranker (default 1000, max 10000)
- Benchmark testing viser at hybrid retrieval med semantic ranker gir signifikant bedre søkerelevans enn enkelt-modalitet
**Query-struktur:** `search` for full-text, `vectorQueries` for vector (kan ha flere), valgfri `queryType=semantic` for L2-reranking.

13
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/citation-tracking.md
View file

@ -1,6 +1,6 @@
# Citation Tracking and Source Attribution
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (classic RAG), Preview (agentic retrieval)
**Category:** RAG Architecture & Semantic Search
@ -292,3 +292,14 @@ result = groundedness_eval(
- Norsk lovgivning (Forvaltningsloven, Offentleglova, Arkivloven)
- Kostnadsoptimerings-anbefalinger
- Modenhetsnivå-tabellen
### Agentic Retrieval — Citation Tracking (oppdatert 2026-04)
Azure AI Search agentic retrieval (preview) returnerer et tre-delt svar som gjør citation tracking robust:
1. **Merged content** — samlet grounding data for LLM
2. **Source references** — kildereferanser for inspeksjon og citation
3. **Activity plan** — query execution-detaljer (subqueries, sources, parameters)
Agentic retrieval bruker LLM til å rive ned komplekse queries til subqueries som kjøres parallelt, med semantisk reranking av hvert delresultat. Dette gir bedre grounding data enn klassisk RAG for komplekse spørsmål. Source references med full provenance tracking støtter transparenskrav i norsk offentlig sektor.

17
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/contextual-retrieval.md
View file

@ -1,6 +1,6 @@
# Contextual Retrieval — Kontekstuell berikelse av chunks
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (custom skill pattern), Preview (agentic retrieval)
**Category:** RAG Architecture & Semantic Search
@ -282,3 +282,18 @@ Hvis contextual retrieval reduserer irrelevante LLM-kall med 30%:
| Custom skill interface (Azure AI Search) | **Verified** | [learn.microsoft.com](https://learn.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-interface) |
| Custom skill example (Python) | **Verified** | [learn.microsoft.com](https://learn.microsoft.com/en-us/previous-versions/azure/search/cognitive-search-custom-skill-python) |
| Retrieval failure benchmarks | **Baseline** | Anthropic research, validert av Microsoft |
### Custom Skill Interface (oppdatert 2026-04)
Custom skills integreres i Azure AI Search enrichment pipeline via `#Microsoft.Skills.Custom.WebApiSkill`.
**Interface-krav:**
- HTTPS endpoint (Azure Functions, containers, eller annen Azure-hosted tjeneste)
- Aksepterer JSON batch: `{"values": [{"recordId": "...", "data": {...}}, ...]}`
- Returnerer JSON batch: `{"values": [{"recordId": "...", "data": {...}, "errors": [], "warnings": []}]}`
- Timeout: default 30s, maks 230s (`PT230S`)
- Autentisering: API-key i URI/header, eller managed identity med `authResourceId`
**Kontekstuell berikelse via custom skill:**
Custom skills brukes for contextual retrieval der hver chunk berikes med kontekst fra omliggende tekst (f.eks. "Dette er fra kapittel 3 om sikkerhet..."). Custom skill kaller LLM med original dokument + chunk, og returnerer kontekstualisert chunk for bedre embedding-kvalitet.

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/graphrag-knowledge-graphs.md
View file

@ -1,6 +1,6 @@
# GraphRAG - Knowledge Graphs and Relationship Extraction
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** Preview
**Category:** RAG Architecture & Semantic Search
@ -23,7 +23,7 @@ GraphRAG-systemet består av flere integrerte lag som sammen muliggjør relasjon
| Komponent | Beskrivelse | Microsoft-teknologi |
|-----------|-------------|---------------------|
| **Entity Extraction** | Identifiserer og trekker ut navngitte entiteter (personer, organisasjoner, lokasjoner) fra tekst | Azure AI Language Service (NER v3), GenAI Prompt skill |
| **Relationship Graphs** | Representerer entiteter som nodes og relasjoner som edges i en graph-struktur | Azure Cosmos DB (graph API), Kusto Query Language (KQL) graph semantics |
| **Relationship Graphs** | Representerer entiteter som nodes og relasjoner som edges i en graph-struktur | Azure Cosmos DB (graph API), Microsoft Fabric Graph (Labeled Property Graph — LPG model, Public Preview), Kusto Query Language (KQL) graph semantics |
| **Graph Indexing** | Lagrer og indekserer graph-strukturen for effektiv traversal og søk | Azure Cosmos DB indexing, Azure AI Search (hybrid indexing) |
| **Traversal Queries** | Søkemekanismer for å navigere graph-strukturen (pattern matching, shortest path, neighborhood search) | KQL `graph-match`, `graph-shortest-paths`, Labeled Property Graphs (LPG) |
| **Entity Linking** | Forbinder ekstraherte entiteter med eksisterende knowledge bases (f.eks. Wikipedia) for normalisering og anrikning | Azure AI Language Entity Linking skill |
@ -39,6 +39,8 @@ Entity extraction-prosessen transformerer ustrukturert tekst til strukturerte en
### Graph Database Modeller
> **Microsoft Fabric Graph (Preview):** Fabric Graph bruker Labeled Property Graph (LPG)-modellen for rask traversal og analytics. RDF-formatet støttes ikke. LPG egner seg for enterprise analytics og fraud detection der semantisk web-integrasjon ikke er nødvendig.
Microsoft Fabric og Azure støtter **Labeled Property Graphs (LPG)** som standard graph-modell:
- **Nodes (entiteter)**: Har labels (typer), properties (attributter) og unique IDs

26
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/hybrid-search-configuration.md
View file

@ -1,6 +1,6 @@
# Hybrid Search - Full-Text and Vector Combined
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -223,3 +223,27 @@ Bruk `debug: "vector"` eller `debug: "semantic"` i API-kallet for å pakke ut su
### Baseline (modellkunnskap)
- Kostnadsoptimerings-tips basert på generell Azure-erfaring
- Offentlig sektor-anbefalinger basert på norsk kontekst
### Hybrid Search — Konfigurasjon og Tuning (oppdatert 2026-04)
**Anbefalt startpunkt:** Balanced hybrid med `k=30-50`, `top=10-20`, semantic ranking etter relevans-test.
**maxTextRecallSize (preview):** Kontrollerer BM25-bidrag til RRF
- Default: 1000, Max: 10000
- Reduser hvis vector dominerer; øk for store indekser der default ikke gir nok dekning
- `countAndFacetMode: "countRetrievableResults"` scope-r teller til maxTextRecallSize-vinduet
**Ytelsesmønstre:**
- Recall-first: øk `maxTextRecallSize` gradvis
- Precision-first: lav `k` og `top`, unngå unødvendig semantic ranker
**Filter-moduser i hybrid:**
- `preFilter` (default) — filtrerer FØR query, reduserer søkerom for begge subqueries
- `postFilter` — filtrerer ETTER, men kan gi <50 docs til semantic ranker
- `strictPostFilter` (preview) — strengeste modus, ikke anbefalt med semantic ranker
- `filterOverride` (preview) — per-vectorQuery filter, overstyrer globalt filter
**SDK-støtte:** Python (`azure-search-documents`), C# (`Azure.Search.Documents`), Java, JavaScript.
**API-versjoner:** `2025-09-01` (stable), `2025-11-01-preview` (maxTextRecallSize, filterOverride, etc.)

27
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/metadata-management-filtering.md
View file

@ -1,6 +1,6 @@
# Metadata Management and Filtered Search
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -523,3 +523,28 @@ $filter=personalDataCategories/any(c: c eq 'Helseopplysninger') and legalBasis n
**For Cosmo:**
Metadata management er ofte undervurdert i RAG-prosjekter. Kunder fokuserer på embeddings og vector search, men glemmer at 70% av queries i produksjon inneholder strukturerte filter-kriterier ("bare fra min avdeling", "kun siste år", "høyeste klassifikasjon"). Design metadata-schema tidlig, test med reelle cardinality-tall, og prioritér normalizers og security trimming fra dag 1. I offentlig sektor er compliance non-negotiable — bygg audit trail og retention policies inn fra start.
### OData Filter Syntax (oppdatert 2026-04)
Azure AI Search bruker OData `$filter`-syntaks for metadata-filtrering:
**Operatorer:**
- Sammenligning: `eq`, `ne`, `gt`, `lt`, `ge`, `le`
- Logisk: `and`, `or`, `not` (høyest presedens: `not` > sammenlignings-ops > `and` > `or`)
- Samling: `any()`, `all()` for collection-felt
- Geo-spatial: `geo.distance()`, `geo.intersects()`
- Full-text i filter: `search.ismatch()`, `search.ismatchscoring()`
- Effektiv liste-matching: `search.in(field, "val1,val2", ",")`
**Viktig:** `search.in()` teller som én klausul (bedre enn lange OR-kjeder for filter size limits).
**Eksempel — sikkerhetstrimming:**
```odata
$filter=search.in(UserGroup, "GroupA,GroupB", ",")
```
**Hybrid search med filter:**
- `preFilter` (default) — anvendes FØR query execution, reduserer søkerom
- `postFilter` — anvendes ETTER, trimmer resultater (bedre med semantic ranker)
- `filterOverride` (preview) — per-vectorQuery filter som overstyrer globalt filter

6
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/multi-index-federation.md
View file

@ -1,6 +1,6 @@
# Multi-Index Federation and Cross-Search
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (single-index), Not supported (native cross-index)
**Category:** RAG Architecture & Semantic Search
@ -16,6 +16,10 @@ For scenarioer der data logisk tilhører separate indekser (ulike skjemaer, comp
**Viktig funn:** Microsofts offisielle FAQ sier eksplisitt: *"Can I search across multiple indexes? No. A query is always scoped to a single index."*
**Multi-region støtte:** Azure AI Search er en single-region service, men du kan oppnå høyere reliability ved å deploye identiske search services i flere regioner. Data synkroniseres via push eller pull (indexer) APIer. Load balancing håndteres av Azure Front Door, Traffic Manager, eller Application Gateway. Data residency: innhold lagres i regionen du velger, uten kryssregional dataflyt uten eksplisitt autorisasjon.
**Multi-vector fields (Preview):** Azure AI Search støtter nå multiple vektorer i ett dokumentfelt via `Collection(Edm.ComplexType)` — opp til 100 vektorer per dokument. Nyttig for multimodal data og lange dokumenter. `perDocumentVectorLimit`-parameteren kontrollerer antall matchende vektorer per dokument i query-resultater.
## Kjernekomponenter
### Hvorfor Azure AI Search ikke støtter cross-index queries

16
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/multimodal-rag.md
View file

@ -1,6 +1,6 @@
# Multimodal RAG — Bilder, tabeller og dokumenter i RAG
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Document Intelligence, Content Understanding), Preview (multimodal embeddings)
**Category:** RAG Architecture & Semantic Search
@ -309,3 +309,17 @@ chartFormat=markdown
| Multimodal RAG with Vision (ISE DevBlog) | **Verified** | [devblogs.microsoft.com](https://devblogs.microsoft.com/ise/multimodal-rag-with-vision/) |
| RAG Time Journey 4: Advanced Multimodal Indexing | **Verified** | [techcommunity.microsoft.com](https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/rag-time-journey-4-advanced-multimodal-indexing/4397300) |
| Azure-Samples/multimodal-rag-code-execution | **Baseline** | [github.com](https://github.com/Azure-Samples/multimodal-rag-code-execution) |
### Azure AI Search Multimodal Pipeline (oppdatert 2026-04)
Azure AI Search multimodal pipeline (GA) støtter nå en fullstendig 5-stegs prosess:
1. **Ekstraksjon** — Document Extraction, Document Layout, eller Content Understanding skill
2. **Tekst-chunking** — Text Split skill for håndterbare biter
3. **Bildebeskriving** — GenAI Prompt skill verbaliserer bilder via LLM
4. **Embedding** — Azure OpenAI, Microsoft Foundry, eller Azure Vision embedding
5. **Bildestoring** — Knowledge store lagrer ekstraherte bilder for annotation i klientapp
Hybrid queries kombinerer full-text search, vector search, og semantic ranking for å svare på spørsmål der svaret befinner seg i et innebygd diagram i en PDF.
**Querytidsstøtte:** GenAI Prompt skill-baserte pipelines støtter hybrid queries over tekst og verbaliserte bilder. For bilde-til-vektor-queries (søk med bilde som input), bruk Azure Vision multimodal embedding skill med en tilsvarende vectorizer.

22
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-caching-optimization.md
View file

@ -1,6 +1,6 @@
# RAG Caching and Performance Optimization
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -503,3 +503,23 @@ cache_key = f"user:{user_id}:tenant:{tenant_id}:query_hash:{hash(prompt)}"
**Totalt antall kilder:** 9 unike Microsoft Learn URLer
**MCP calls:** 6 (4 docs_search + 2 docs_fetch + 1 code_sample_search)
**Sist verifisert:** 2026-02-03
### Azure Managed Redis — Arkitektur (oppdatert 2026-04)
Azure Managed Redis (basert på Redis Enterprise) er anbefalt for AI-workloads vs. Azure Cache for Redis (community edition):
| Egenskap | Azure Cache for Redis | Azure Managed Redis |
|---------|----------------------|---------------------|
| Threading | Single-threaded | Multi-threaded (Redis Enterprise) |
| Arkitektur | Primary + replica (2 nodes) | Multiple shards per node, distributed primaries |
| Performance | Begrenset av single thread | Nær-lineær skalering med vCPUs |
| Clustering | Valgfritt | Alltid aktivert (OSS, Enterprise, eller Non-Clustered policy) |
| Active geo-replication | Nei | Ja |
**Cluster policies:**
- **OSS policy** — anbefalt for de fleste. Klienten kobles direkte til shards, laveste latency, best throughput
- **Enterprise policy** — enkelt endpoint, bakoverkompatibelt, men enkelt-node proxy kan bli bottleneck. Påkrevd for RediSearch
- **Non-Clustered** — kun ≤25 GB, for migrering fra ikke-shardede miljøer
**Flash Optimized tier:** 20% RAM + 80% NVMe Flash. Optimal for read-heavy workloads med subset av hot keys.

24
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-core-patterns.md
View file

@ -1,6 +1,6 @@
# RAG Core Patterns and Architecture
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -417,3 +417,25 @@ results = search_client.search(
---
**For Cosmo:** Når kunde spør om RAG, start med "Naive vs Advanced vs Agentic"-beslutningstreet. Identifiser data source, query complexity, og latency-krav først. Hvis offentlig sektor: alltid spør om GDPR/Schrems II/AI Act compliance før du foreslår arkitektur. Hvis customer mangler evaluation strategy: stopp og definer retrieval recall/precision targets før du går videre med implementation.
### Hybrid Search — Kjernemønster (oppdatert 2026-04)
Hybrid search er standardmønsteret for RAG i Azure AI Search:
```json
{
"search": "historisk hotell nær restauranter",
"vectorQueries": [{"kind": "vector", "vector": [...], "k": 50, "fields": "DescriptionVector"}],
"queryType": "semantic",
"semanticConfiguration": "my-semantic-config"
}
```
**Hvorfor hybrid:**
- Vector search: finner konseptuelt like dokumenter uten nøyaktige nøkkelord-treff
- Full-text search: presis matching for produktkoder, navn, datoer
- RRF merger: normaliserer scores fra BM25 og HNSW/eKNN
- Semantic ranker (L2): re-ranker opp til 50 resultater med maskinlesningsforståelse
**Best practice:** Sett `k=50` ved bruk av semantic ranker.

14
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-document-preprocessing.md
View file

@ -1,6 +1,6 @@
# Document Preprocessing and Pipeline Automation
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -776,3 +776,15 @@ Images (JPEG/PNG)?
**Totalt antall MCP-kilder:** 3 docs_search calls + 2 docs_fetch calls = **5 MCP-kall**
**Totalt antall unike URLer:** 8 Microsoft Learn-artikler + 4 GitHub-repos = **12 kilder**
**Konfidensnivå totalt:** 95% Verified (fra MCP), 5% Baseline (norske forhold og priser)
### Kognitiv søk — bildeprosessering (oppdatert 2026-04)
Azure AI Search støtter tre tilnærminger til bildeinnhold i RAG:
1. **Vektorisering** — Azure Vision genererer bildere presentasjoner som søkbare vektorer
2. **Verbalisering** — GenAI Prompt skill sender bilde til LLM-chat-modell for naturlig tekstbeskrivelse (bedre for RAG-grounding)
3. **Analyse/OCR** — Image Analysis skill (tags, description) og OCR skill (tekst fra bilder)
`imageAction: generateNormalizedImages` er påkrevd for bildebehandling. Maks 1000 bilder ekstraheres per dokument. Kostnader påløper ved `imageAction != none`.
**Skillset tutorial (oppdatert):** Skillsets bygges med OCR, språkdeteksjon, entity recognition og key phrase extraction i pipeline. Output field mappings mapper enriched document tree til søkeindeksfelt.

21
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-hallucination-mitigation.md
View file

@ -1,6 +1,6 @@
# RAG Hallucination Mitigation Strategies
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -400,3 +400,22 @@ Hvis AI-systemet gir feil informasjon som fører til skade:
**Sist verifisert:** 2026-02-03
**Neste revisjon:** 2026-05 (eller ved oppdatering av Azure AI Content Safety API)
### Azure AI Content Safety — Groundedness Detection (oppdatert 2026-04)
**Breaking change:** API-feltnavn er endret:
- `correction` → `mitigating` (deteksjons-modus)
- `correctedText` → `correctionText` (output-felt med korrigert tekst)
**Deteksjonsmoduser:**
- **Non-Reasoning mode** — rask binær deteksjon (grounded/ungrounded), lav latency for produksjon
- **Reasoning mode** — detaljerte forklaringer på ungrounded segmenter, bruk under utvikling/debugging
**Domenestøtte:** `MEDICAL` (medisinsk/vitenskapelig) og `GENERIC` (generelt formål)
**Task typer:** `Summarization` og `QnA`
**Grounding correction (preview):** API kan automatisk korrigere ikke-grounded tekst basert på dine grounding sources. Respons inkluderer `correctionText`-felt med korrigert innhold.
**Begrensninger:** Kun engelsk tekst støttes. Tilgjengelig i utvalgte Azure-regioner.

24
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-iterative-refinement.md
View file

@ -1,6 +1,6 @@
# Iterative RAG and Multi-Turn Refinement
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -439,3 +439,25 @@ public async Task ApplyRetentionPolicyAsync()
**Forfatter:** Cosmo Skyberg (AI Architect)
**For:** KTG Microsoft AI Architect Plugin
**MCP Research:** microsoft-learn (4 searches, 2 fetches), microsoft-code-samples (1 search)
### .NET AI IChatClient-interface (oppdatert 2026-04)
`IChatClient` er .NET-standarden for interaksjon med AI chat services (Microsoft.Extensions.AI). Støtter stateless og stateful samtaler, tool calling, streaming, caching, og OpenTelemetry.
```csharp
// Stateful iterativ RAG med IChatClient
List<ChatMessage> history = [];
while (true) {
history.Add(new(ChatRole.User, userInput));
ChatResponse response = await client.GetResponseAsync(history);
history.AddMessages(response); // akkumulerer kontekst
}
```
**Viktige egenskaper:**
- `ConversationId` støtter stateful tjenester (slipper å sende full historikk)
- `FunctionInvokingChatClient` gir automatisk tool invocation for agentic loops
- `DistributedCachingChatClient` wrapper cacher identiske historikker
- `ChatHistoryTruncationReducer` / `ChatHistorySummarizationReducer` håndterer context window limits
- Pipeline-komposisjon: `ChatClientBuilder` stacker cache, tool invocation, og telemetri

21
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-query-understanding.md
View file

@ -1,6 +1,6 @@
# Query Understanding and Expansion
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA
**Category:** RAG Architecture & Semantic Search
@ -576,3 +576,22 @@ Multi-Query RAG med 3 varianter = ~3x søkekostnad + 1 LLM-kall for generering.
- **ROI-tall (15-30% precision improvement):** Baseline — industry benchmarks, ikke Microsoft-spesifikk data
**Totalt antall kilder:** 7 Microsoft Learn URLer (Verified) + 1 GitHub repo (Verified) = 8 kilder
### Simple Query Syntax for RAG (oppdatert 2026-04)
Azure AI Search simple query syntax er default parser for full-text søk i RAG:
**Boolske operatorer (tegn-basert):**
- `+` — AND (påkrevd term)
- `|` — OR (alternativ term)
- `-` — NOT (ekskluder term) — `searchMode=all` anbefales for presis NOT-atferd
**Prefix queries:** `lingui*` — matcher "linguistic", "linguini" etc.
**Phrase search:** `"eksakt frase"` — krever eksakt ordrekkefølge
**Begrensninger:** Ingen fuzzy search, ingen suffix/infix wildcard (bruk full Lucene syntax for det).
**Bruk i RAG query expansion:**
Simple syntax egner seg for keyword-delen av hybrid queries. For agentic RAG bruker LLM query planning til å generere fokuserte subqueries som kombinerer full-text + vector search parallelt.

23
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/rag-security-rbac.md
View file

@ -1,6 +1,6 @@
# RAG Security - RBAC, Filtering, and Access Control
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** Preview (native ACL/RBAC), GA (security filters)
**Category:** RAG Architecture & Semantic Search
@ -531,3 +531,24 @@ Authorization: Bearer <user-token>
**GA features:** Security filters (alle API-versjoner)
**Note:** Preview features kan endre seg. Konsulter alltid nyeste dokumentasjon før produksjon.
### Dokumentnivå-tilgangskontroll — oppdatering 2026-04
**4 tilnærminger (oppdatert):**
| Tilnærming | Status | Beskrivelse |
|-----------|--------|-------------|
| **Security filters** | GA | String-sammenligning med `search.in()` — API-agnostisk |
| **POSIX-like ACL/RBAC scopes** | Preview | Microsoft Entra ID-autentisering mot dokument-ACLer (ADLS Gen2) |
| **Microsoft Purview sensitivity labels** | Preview | Entra-token + Purview policy enforced ved query-tid |
| **SharePoint M365 ACLs** | Preview | SharePoint-tilganger ekstraheres av indexer og håndheves ved søk |
**ADLS Gen2 ACL/RBAC (preview):**
- RBAC: container-nivå (grov tilgangskontroll for alle dokumenter i container)
- ACL: fil/mappe-nivå (finkornet per-dokument tilgangskontroll)
- ABAC: **ikke støttet** i Azure AI Search
- Tilgangsevaluering: RBAC sjekkes først, deretter ACL. Tilgang gis om én av dem tillater det
- Permissions synkroniseres ved: første full indexer-kjøring, nye dokumenter, eller manuell trigger via `/resync` (preview)
**Query-enforcement:** `x-ms-query-source-authorization`-header med Entra-token aktiverer automatisk trimming.

24
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/semantic-ranker-reranking.md
View file

@ -1,6 +1,6 @@
# Semantic Ranker and Reranking Models
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (core), Preview (query rewrite, prerelease models)
**Category:** RAG Architecture & Semantic Search
@ -276,3 +276,25 @@ Preview-funksjon (2025) som integrerer iterativ søk med semantic ranking.
### Baseline (modellkunnskap)
- Cross-encoder-eksempler basert på Sentence Transformers-dokumentasjon
- Offentlig sektor-anbefalinger basert på norsk kontekst
### Semantic Ranker i Hybrid Search (oppdatert 2026-04)
Semantic ranker (L2 reranking) fungerer optimalt i hybrid search-kontekst:
- Aksepterer opp til **50 resultater** fra RRF-merger som input
- Bruker maskinlesningsforståelse (MRC) for å re-ranke basert på semantisk relevans
- `@search.rerankerScore` erstatter `@search.score` som primær rankingmetrikk
- Valgfritt: `captions` (ekstraktiv) og `answers` (ekstraktiv) fra verbatim tekst
**Konfigurasjon:**
```json
{
"queryType": "semantic",
"semanticConfiguration": "min-konfig",
"captions": "extractive",
"answers": "extractive"
}
```
**Viktig:** Sett `k=50` i vectorQueries — semantic ranker trenger tilstrekkelig input. Pre-filtre som er for strenge kan redusere antall input-dokumenter og svekke reranking-kvaliteten.

17
plugins/ms-ai-architect/skills/ms-ai-engineering/references/rag-architecture/vector-indexing-techniques.md
View file

@ -1,6 +1,6 @@
# Vector Indexing - Techniques and Configuration
**Last updated:** 2026-02
**Last updated:** 2026-04 | Verified: MCP 2026-04
**Status:** GA (Hybrid search), Preview (Scalar quantization)
**Category:** RAG Architecture & Semantic Search
@ -410,3 +410,18 @@ Azure AI Search krever ingen spesifikk Microsoft 365-lisens, men:
- [Semantic ranking in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/semantic-search-overview) — **Verified** (2025-11)
**Konfidensnivå:** Verified (90%) — All info basert på offisiell Microsoft-dokumentasjon og prising per feb 2026.
### Vector Indexing i Hybrid Search-kontekst (oppdatert 2026-04)
Vector fields og tekstfelt coeksisterer i hybrid search-indekser:
- **HNSW** (Hierarchical Navigable Small World) — standard ANN-algoritme, `efSearch` og `maxConnections` tunable
- **eKNN** (exhaustive K-Nearest Neighbors) — fullstendig søk, brukes med `"exhaustive": true` i query
- **Multi-vector fields (preview)** — `Collection(Edm.ComplexType)` støtter opp til 100 vektorer per dokument
- `perDocumentVectorLimit: 1` — én vektor per dokument i resultater
- `perDocumentVectorLimit: 0` — ubegrenset (alle matchende vektorer)
- Nyttig for multimodal data (scene embeddings i video, fragmenter i lange dokumenter)
**Tuning ved overbelastning:**
Reduser `efSearch` (f.eks. 800 → 128-192) og `maxConnections` (64 → 32) FØR du skalerer ut med flere replicas. Hybrid queries med aggressive vector-innstillinger + semantic ranker øker CPU/minne-press betydelig.