From 0c2fd3d8a6777a445a9f710bc8dcad98511d6a6d Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Tue, 23 Jun 2026 22:45:19 +0200 Subject: [PATCH] =?UTF-8?q?docs(ms-ai-architect):=20KB-refresh=20critical?= =?UTF-8?q?=20cost=204/8=20=E2=80=94=20inference-endpoint=20verifisert=20m?= =?UTF-8?q?ot=20Foundry=20priority-processing=20+=20deployment-types?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verifisert mot live Microsoft Learn (foundry/openai/concepts/priority-processing, foundry/foundry-models/concepts/deployment-types, foundry/concepts/manage-costs): - Priority processing lagt til (deployment-tabell + Nivå 4). ÆRLIG framing: delt doc-signal — provisioned-throughput-siden lister den som fullverdig pay-per-token-kategori m/ latensmål per modell (gpt-5.5/5.4/5.2/5.1/4.1), mens deployment-types/enable-siden markerer preview/invitasjon. Skrevet som tier på GlobalStandard/DataZoneStandard (modellv. 2025-12-01+), m/ «rulles ut»-caveat. - VERIFY meter-skille bekreftet: «Foundry Models sold by Azure billed via Azure meters as First Party Consumption Services» under Foundry-ressurs; partner/community «billed through Azure Marketplace» → Global resources, format model-name-GUID. Oppdatert Cost-tracking-bullet (slo feil sammen før). - Phi-3→Phi-4-familien (2 steder): katalog viser kun Phi-4-familien (Phi-4, -mini-instruct, -multimodal, -reasoning) i sky-serverless; Phi-3 superseded. Estimat-priser beholdt m/ (estimat). - Quota 200k TPM/1k RPM: lagt til «verifiser i quota-docs (varierer per modell/type/region)». Korreksjons-disiplin (operatør-godkjent): footer-dato-inkonsistens fikset — header 2026-04→2026-06, footer «Sist oppdatert 2026-02»→2026-06 (561-seksjonsstempel urørt, ikke re-verifisert). 2 kilde- rader stemplet Verified MCP 2026-06 (faktisk re-fetchet). Disclaimed priser urørt. validate 239/0. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../inference-endpoint-cost-optimization.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/skills/ms-ai-security/references/cost-optimization/inference-endpoint-cost-optimization.md b/skills/ms-ai-security/references/cost-optimization/inference-endpoint-cost-optimization.md index b4833d6..f097bf0 100644 --- a/skills/ms-ai-security/references/cost-optimization/inference-endpoint-cost-optimization.md +++ b/skills/ms-ai-security/references/cost-optimization/inference-endpoint-cost-optimization.md @@ -1,6 +1,6 @@ # Managed Inference Endpoints: Cost Optimization -**Last updated:** 2026-04 +**Last updated:** 2026-06 **Status:** GA **Category:** Cost Optimization & FinOps for AI @@ -23,8 +23,11 @@ Hovedutfordringen for de fleste organisasjoner er å balansere tre faktorer: kos | **Managed Online Endpoint** | VM-timer (per instance, per hour) | Konsistent, forutsigbar trafikk | Fast timekostnad uavhengig av requests | | **Serverless API Endpoint** | Pay-per-token + pay-per-request | Variabel, uforutsigbar trafikk | Kun kostnad ved faktisk bruk | | **Provisioned Throughput (PTU)** | Fast månedskostnad for reservert kapasitet | Stable workloads med høy throughput | Lavere enhetskostnad for høy bruk | +| **Priority Processing**¹ (tier på Standard serverless) | Pay-per-token (priority-tier-rate) | Latens-sensitiv serverless uten langtidsbinding | Premium enhetspris vs. Standard, men definert latensmål (SLA) per modell | | **Low-Priority VMs** | 50-80% rabatt vs. dedicated VMs | Batch inference, ikke-kritiske workloads | Betydelig kostnadsbesparing med preemption-risiko | +*¹ Priority processing aktiveres på GlobalStandard-/DataZoneStandard-deployments (modellversjon 2025-12-01+) og gir lav, forutsigbar latens på pay-as-you-go. Rulles ut — enkelte doc-flater (deployment-types/enable-siden) markerer den fortsatt som preview/invitasjon per 2026-06.* + ### Autoscaling-konfigurasjonskomponenter | Parameter | Beskrivelse | Kostnadspåvirkning | @@ -279,7 +282,7 @@ START **Serverless endpoints:** - Provisjoneres via AI Foundry Portal eller SDK (`ServerlessEndpoint`) -- Støtter Microsoft-modeller (Phi-3, m.fl.) og Azure Marketplace-modeller +- Støtter Microsoft-modeller (Phi-4-familien, m.fl.) og Azure Marketplace-modeller - Kostnadsoppfølging via Azure Cost Management med marketplace-meters **Managed compute (via Azure ML integration):** @@ -343,7 +346,7 @@ mon_client.autoscale_settings.create_or_update( **Cost tracking:** - Managed endpoints: Tag-basert kostnadssporing (`azuremlendpoint`, `azuremldeployment`) -- Serverless: Meters i Azure Cost Management (separate for Microsoft vs. Marketplace-modeller) +- Serverless: I Cost Management vises Models sold by Azure (inkl. Azure OpenAI/Microsoft) som meters under selve Foundry-ressursen, mens partner-/Marketplace-modeller vises under **Global resources** med format `model-name-GUID` - Budsjett-alerts for proaktiv kostnadskontroll **Metrics for optimalisering:** @@ -421,10 +424,10 @@ Total kostnad = (Instance hours × Instance price) **Token-basert prising:** - Pris per 1M tokens (input og output prises separat) - Pris per 1000 API requests -- Quota: 200k tokens/min og 1k requests/min per deployment (standard) +- Quota: typisk 200k tokens/min og 1k requests/min per deployment (standard) — verifiser gjeldende grenser i quota-docs (varierer per modell, deployment-type og region) **Microsoft-modeller (direkte fra Azure):** -- Phi-3: ~10 kr per 1M input tokens, ~30 kr per 1M output tokens (estimat) +- Phi-4-familien: ~10 kr per 1M input tokens, ~30 kr per 1M output tokens (estimat) - Priser vises i "Pricing and terms" tab ved deployment **Marketplace-modeller (tredjepart):** @@ -542,6 +545,7 @@ Total kostnad = (Instance hours × Instance price) **Nivå 4: Enterprise-skala (100+ modeller, millioner requests/dag)** - Vurder **Provisioned Throughput (PTU)** for høy-volum modeller (Azure OpenAI) +- Vurder **Priority processing** (pay-per-token priority-tier med definert latensmål) for latens-sensitive serverless-workloads uten PTU-binding — kan kombineres med PTU for steady-state-kapasitet - Implementer multi-region deployment for geo-distribusjon og cost arbitrage - Bruk custom autoscaling-metrics (business KPIs, ikke bare CPU) - Dedikert FinOps-team for kontinuerlig optimalisering @@ -574,6 +578,8 @@ Hvis inference-kostnad per prediction >10% av business value per prediction, er - [Deploy models as serverless API deployments (AI Foundry Portal)](https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/deploy-models-serverless?view=foundry-classic) — **Verified** - [Plan and manage costs for Microsoft Foundry](https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs?view=foundry-classic) — **Verified** - [Plan to manage costs for Azure OpenAI in Foundry Models](https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs) — **Verified** +- [Enable priority processing for Microsoft Foundry models](https://learn.microsoft.com/azure/foundry/openai/concepts/priority-processing) — **Verified MCP 2026-06** +- [Deployment types for Microsoft Foundry Models](https://learn.microsoft.com/azure/foundry/foundry-models/concepts/deployment-types) — **Verified MCP 2026-06** **Cost Governance:** - [Govern Azure platform services (PaaS) for AI](https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/platform/governance) — **Verified** @@ -598,6 +604,6 @@ Hvis inference-kostnad per prediction >10% av business value per prediction, er --- -**Sist oppdatert:** 2026-02 +**Sist oppdatert:** 2026-06 **Versjon:** 1.0 **Forfatter:** Cosmo Skyberg (AI-generert kunnskapsbase via MCP-research)