open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity

docs(ms-ai-architect): KB-refresh LOW 21/52 — genai-gateway-policies: token-quota-period sek→Daily, KQL ModelDeployment→DeploymentName, llm-token-limit Consumption Nei, semantic-cache tier-matrise korrigert

Browse source
This commit is contained in:
Kjell Tore Guttormsen 2026-06-24 12:15:37 +02:00
commit 4b4fd24702
1 changed files with 16 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

26
skills/ms-ai-engineering/references/api-management/genai-gateway-policies.md
View file

@ -1,6 +1,6 @@
# GenAI-Specific APIM Policies & Rules # GenAI-Specific APIM Policies & Rules
**Last updated:** 2026-02 **Last updated:** 2026-06-24
**Status:** GA **Status:** GA
**Category:** API Management & AI Gateway **Category:** API Management & AI Gateway
@ -322,11 +322,10 @@ Sett token-kvoter per dag, uke eller måned:
<!-- Daglig token-kvote per avdeling --> <!-- Daglig token-kvote per avdeling -->
<llm-token-limit <llm-token-limit
counter-key="@(context.Request.Headers.GetValueOrDefault("X-Department", "default"))" counter-key="@(context.Request.Headers.GetValueOrDefault("X-Department", "default"))"
tokens-per-minute="0"
token-quota="100000" token-quota="100000"
token-quota-period="86400" token-quota-period="Daily"
estimate-prompt-tokens="true" estimate-prompt-tokens="true"
remaining-tokens-variable-name="dailyRemaining" /> remaining-quota-tokens-variable-name="dailyRemaining" />
<!-- Legg til gjenværende kvote i respons-header --> <!-- Legg til gjenværende kvote i respons-header -->
<set-header name="X-Daily-Tokens-Remaining" exists-action="override"> <set-header name="X-Daily-Tokens-Remaining" exists-action="override">
@ -389,7 +388,8 @@ For å oppnå global rate limiting, bruk `quota-by-key` i stedet for `llm-token-
| `TimeGenerated` | Tidspunkt for request | 2026-02-11T10:30:00Z | | `TimeGenerated` | Tidspunkt for request | 2026-02-11T10:30:00Z |
| `CorrelationId` | Unik request-ID | abc-123-def | | `CorrelationId` | Unik request-ID | abc-123-def |
| `OperationName` | API-operasjon | ChatCompletions | | `OperationName` | API-operasjon | ChatCompletions |
| `ModelDeployment` | Deployment-navn | gpt-4o | | `DeploymentName` | Deployment-navn | gpt-4o |
| `ModelName` | Modellnavn | gpt-4o |
| `PromptTokens` | Antall prompt-tokens | 150 | | `PromptTokens` | Antall prompt-tokens | 150 |
| `CompletionTokens` | Antall completion-tokens | 250 | | `CompletionTokens` | Antall completion-tokens | 250 |
| `TotalTokens` | Totalt token-forbruk | 400 | | `TotalTokens` | Totalt token-forbruk | 400 |
@ -409,7 +409,7 @@ ApiManagementGatewayLlmLog
| project | project
TimeGenerated, TimeGenerated,
CorrelationId, CorrelationId,
Model = ModelDeployment, Model = DeploymentName,
PromptTokens, PromptTokens,
CompletionTokens, CompletionTokens,
Prompt = tostring(RequestArray.content), Prompt = tostring(RequestArray.content),
@ -426,9 +426,15 @@ ApiManagementGatewayLlmLog
```kusto ```kusto
// Finn uvanlig høy token-bruk per bruker // Finn uvanlig høy token-bruk per bruker
// Brukeridentitet finnes ikke i ApiManagementGatewayLlmLog — join mot
// ApiManagementGatewayLogs på CorrelationId for UserId/ApimSubscriptionId.
ApiManagementGatewayLlmLog ApiManagementGatewayLlmLog
| where TimeGenerated > ago(7d) | where TimeGenerated > ago(7d)
| extend UserId = tostring(CustomDimensions["UserId"]) | join kind=leftouter (
ApiManagementGatewayLogs
| where TimeGenerated > ago(7d)
| distinct CorrelationId, UserId
) on CorrelationId
| summarize | summarize
AvgTokens = avg(toint(TotalTokens)), AvgTokens = avg(toint(TotalTokens)),
MaxTokens = max(toint(TotalTokens)), MaxTokens = max(toint(TotalTokens)),
@ -598,9 +604,9 @@ Outbound (fra topp til bunn):
| Policy | Classic | V2 | Consumption | Self-hosted | Workspace | | Policy | Classic | V2 | Consumption | Self-hosted | Workspace |
|--------|---------|-----|-------------|-------------|-----------| |--------|---------|-----|-------------|-------------|-----------|
| `llm-content-safety` | Ja | Ja | Ja | Ja | Ja | | `llm-content-safety` | Ja | Ja | Ja | Ja | Ja |
| `llm-token-limit` | Ja | Ja | Ja | Ja | Ja | | `llm-token-limit` | Ja | Ja | Nei | Ja | Ja |
| `llm-semantic-cache-lookup` | Ja | Ja | Nei | Nei | Ja | | `llm-semantic-cache-lookup` | Ja | Ja | Ja | Ja | Nei |
| `llm-semantic-cache-store` | Ja | Ja | Nei | Nei | Ja | | `llm-semantic-cache-store` | Ja | Ja | Ja | Ja | Nei |
| `llm-emit-token-metric` | Ja | Ja | Ja | Ja | Ja | | `llm-emit-token-metric` | Ja | Ja | Ja | Ja | Ja |
--- ---