chore(ms-ai-architect): refresh KB critical-bucket — 15 files [skip-docs]
KB-currency refresh (critical priority, 2026-06-19) via /architect:kb-update. 15 critical cost/arkitektur-filer re-verifisert mot Microsoft Learn (MCP). Hovedendringer: - Azure AI Search Serverless (Preview) prismodell (rag-cost, rag-query, vector-storage, rag-enterprise-scale) - Agentic retrieval GA-split (2026-04-01 REST GA, portal preview) + Foundry IQ - AI gateway multi-provider (Anthropic v2 / Google Vertex), unified model API (preview), Foundry-integrasjon (enterprise-arch, ai-services-cost, multi-model, cost-tracking) - FOCUS-skjema + Cost Mgmt -> ADLS Gen2 -> Fabric -> Power BI pipeline (cost-management, cost-allocation) - FinOps hub AI-agent via Azure MCP / Copilot Studio (budget-forecasting) - OTel trace-based sampling for logs (observability) - Korreksjoner: semantic-cache score-threshold = AVSTAND (0.15, ikke 0.85 cosine-likhet); llm-emit-token-metric maks 5 custom dimensions; Foundry RBAC-renames (Owner/Contributor = kun mgmt); Researcher/Analyst er ikke agenter - Alle 15 Last updated -> 2026-06-19 High-bucket (49 filer) UTSATT til egen sesjon (operatoer-valg). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01REiKFhP4w6xGXXqWKpPCJJ
This commit is contained in:
parent
1ae5655156
commit
41b390b38d
15 changed files with 69 additions and 30 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Semantic Caching for AI Workloads
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Last updated:** 2026-06-19
|
||||
**Status:** GA
|
||||
**Category:** Cost Optimization & FinOps for AI
|
||||
|
||||
|
|
@ -65,6 +65,8 @@ Semantic caching består av fire hovedkomponenter:
|
|||
- `0.70-0.84`: Liberal matching, høyere cache hit rate men lavere presisjon
|
||||
- **Start med 0.85 og juster basert på cache hit rate og user feedback** (MEDIUM confidence)
|
||||
|
||||
**Viktig — `score-threshold` i APIM-policyen er en AVSTAND, ikke cosine-likhet (MCP 2026-06):** Tabellen over gjelder generell cosine-**likhet** (høyere = strengere), som brukes av application-level vector-stores. Men APIM-policyene `llm-semantic-cache-lookup` / `azure-openai-semantic-cache-lookup` bruker et `score-threshold` som er en semantisk **avstand**: prompts med score *over* terskelen bruker IKKE cachen, så **lavere terskel = strengere matching**. Microsofts eget eksempel bruker `score-threshold="0.15"`. Sett derfor en LAV verdi (~0.1–0.2) i APIM-policyene under — IKKE 0.85.
|
||||
|
||||
---
|
||||
|
||||
## Arkitekturmønstre
|
||||
|
|
@ -88,7 +90,7 @@ Client → APIM (semantic cache policies) → Azure Managed Redis (RediSearch)
|
|||
```xml
|
||||
<!-- Inbound: Cache Lookup -->
|
||||
<azure-openai-semantic-cache-lookup
|
||||
score-threshold="0.85"
|
||||
score-threshold="0.15"
|
||||
embeddings-backend-id="embeddings-backend"
|
||||
embeddings-backend-auth="system-assigned"
|
||||
ignore-system-messages="true"
|
||||
|
|
@ -313,7 +315,7 @@ L3: Azure OpenAI → 500-2000ms latency
|
|||
|
||||
| Parameter | Verdi | Forklaring |
|
||||
|-----------|-------|------------|
|
||||
| `score-threshold` | 0.85 (anbefalt) | Minimum similarity for cache hit (0-1) |
|
||||
| `score-threshold` | 0.15 (lav!) | Semantisk **avstand**-terskel: prompts over terskelen bruker IKKE cachen → **lavere = strengere**. IKKE cosine-likhet. MS-eksempel: 0.15 |
|
||||
| `embeddings-backend-id` | "embeddings-backend" | Backend ID for embeddings deployment |
|
||||
| `embeddings-backend-auth` | "system-assigned" | Bruker APIM managed identity |
|
||||
| `ignore-system-messages` | true | Ignorer system messages i similarity-beregning |
|
||||
|
|
@ -398,7 +400,7 @@ Azure AI Foundry models (via Model Inference API) støttes med generic LLM polic
|
|||
```xml
|
||||
<!-- Use llm-semantic-cache-lookup instead of azure-openai-semantic-cache-lookup -->
|
||||
<llm-semantic-cache-lookup
|
||||
score-threshold="0.85"
|
||||
score-threshold="0.15"
|
||||
embeddings-backend-id="embeddings-backend"
|
||||
embeddings-backend-auth="system-assigned">
|
||||
<vary-by>@(context.Subscription.Id)</vary-by>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue