ms-ai-architect/skills/ms-ai-governance/references/monitoring-observability/token-usage-tracking-attribution.md

# Token Usage Tracking and Attribution

**Kategori:** Monitoring & Observability
**Dato:** 2026-04-09
**Versjon:** 1.0

## Introduksjon

Token usage tracking og cost attribution er kritiske kapabiliteter for å styre kostnader, implementere chargeback-modeller, og optimalisere ressursbruk i Microsoft AI-løsninger. Denne referansen dekker teknikker for nøyaktig token-måling, brukerattribuering, og kostnadsrapportering.

## Token Counting og Logging

### Basis Token Tracking

Azure OpenAI API returnerer token usage i response-objektet:

```python
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Your prompt"}]
)

# Token-data fra response
input_tokens = response.usage.prompt_tokens
output_tokens = response.usage.completion_tokens
total_tokens = response.usage.total_tokens
```

**Viktig:** Token-telling varierer per modell og er basert på modell-spesifikk tokenizer (GPT-2 tokenizer som baseline).

### Token Estimering (Pre-call)

For å estimere tokens før API-kall:

```python
import tiktoken

class TokenEstimator(object):
    GPT2_TOKENIZER = tiktoken.get_encoding("gpt2")

    def estimate_tokens(self, text: str) -> int:
        return len(self.GPT2_TOKENIZER.encode(text))

# Bruk
estimator = TokenEstimator()
token_count = estimator.estimate_tokens(input_text)
```

**Bruksområder:**
- Pre-validering mot rate limits
- Kostnadsestimering før kall
- Budsjett-gating i applikasjoner

### Azure Monitor Platform Metrics

Azure OpenAI samler automatisk token-baserte metrics:

**Tilgjengelige metrics:**
- `TokenTransaction` — Total token count (input + output)
- `PromptTokens` — Input tokens
- `CompletionTokens` — Output tokens
- `ProcessedPromptTokens` — Tokens faktisk prosessert (kan avvike ved caching)

**Aksess via:**
- Azure Portal → Azure OpenAI resource → Metrics
- Azure Monitor Metrics Explorer
- REST API (`/metrics` endpoint)

**Dashboards:**
Azure OpenAI tilbyr out-of-box dashboards med "Tokens-Based Usage" kategori som viser:
- Token consumption over tid
- Breakdown per modell
- Comparison mot quota limits

## Usage Attribution per Applikasjon/Bruker

### Utfordring: Native Telemetri-begrensninger

**Problem:**
Azure OpenAI logger IP-adresse med siste oktet masket (f.eks. `192.168.1.xxx`), noe som gjør det vanskelig å knytte token-bruk til spesifikk applikasjon eller business unit.

**Løsning:** Introduser gateway-pattern for fullstendig attributering.

### Gateway-basert Attribution (Azure API Management)

**Arkitektur:**
```
Client → API Management Gateway → Azure OpenAI
  ↓
  Token usage logged med:
  - Client IP (full adresse)
  - Microsoft Entra ID identity
  - Custom business unit/app identifier
```

**Fordeler:**
1. **Fullstendig IP-adresse** — Identifiser klient-applikasjon
2. **Identity-data** — Entra ID user/app principal
3. **Custom metadata** — Business unit, cost center, tenant ID
4. **Sentralisert logging** — Aggreger data fra multiple Azure OpenAI instances

**Kusto Query for Usage Monitoring (APIM):**

```kusto
ApiManagementGatewayLogs
| where tolower(OperationId) in ('completions_create','chatcompletions_create')
| extend model = tostring(parse_json(BackendResponseBody)['model'])
| extend prompttokens = parse_json(parse_json(BackendResponseBody)['usage'])['prompt_tokens']
| extend completiontokens = parse_json(parse_json(BackendResponseBody)['usage'])['completion_tokens']
| extend totaltokens = parse_json(parse_json(BackendResponseBody)['usage'])['total_tokens']
| extend ip = CallerIpAddress
| summarize
    sum(todecimal(prompttokens)),
    sum(todecimal(completiontokens)),
    sum(todecimal(totaltokens)),
    avg(todecimal(totaltokens))
    by ip, model
```

**Output:** Tabell med IP, model, sum(prompt tokens), sum(completion tokens), sum(total tokens).

### Application Insights Telemetry Enrichment

For applikasjoner uten gateway, bruk Application Insights med custom telemetry:

```python
import logging
from azure.monitor.opentelemetry import configure_azure_monitor

# Sett opp Application Insights
configure_azure_monitor()

logger = logging.getLogger(__name__)

def log_token_usage(response, user_id, business_unit):
    usage = response.usage

    # Log med custom properties for attribution
    logger.info(
        "Token usage",
        extra={
            "custom_dimensions": {
                "user_id": user_id,
                "business_unit": business_unit,
                "model": response.model,
                "prompt_tokens": usage.prompt_tokens,
                "completion_tokens": usage.completion_tokens,
                "total_tokens": usage.total_tokens
            }
        }
    )
```

**Advarsel:** Application Insights bruker [sampling](https://learn.microsoft.com/en-us/azure/azure-monitor/app/sampling) i high-volume scenarios, noe som ikke er egnet for nøyaktig billing/metering. For billing-data, bruk dedikert data store (Event Hubs + Stream Analytics).

### Resource Tags for Attribution

For deployment-level attribution:

```bash
# Tag Azure OpenAI resource
az openai account update \
  --name myopenai \
  --resource-group myrg \
  --tags CostCenter=Finance AppName=ChatBot Environment=Prod
```

**Bruk i Azure Cost Management:**
Filtrer kostnadsanalyse per tag for å allokere Azure-kostnader til business units.

**Begrensning:** Dette gir deployment-level attribution, ikke per-request granularitet.

## Budget Monitoring og Alerts

### Azure Monitor Budget Alerts

**Oppsett:**

1. **Opprett diagnostic setting** for Azure OpenAI resource:
   - Send metrics til Log Analytics workspace
   - Velg `AllMetrics` kategori

2. **Sett opp metric alert** på token usage:
   ```
   Metric: ProcessedPromptTokens
   Aggregation: Sum
   Threshold: 1000000 (1M tokens)
   Period: 1 hour
   Action: Send email / webhook
   ```

3. **Opprett budget i Cost Management:**
   - Scope: Azure OpenAI resource eller subscription
   - Budget amount: NOK 10,000/måned
   - Alert thresholds: 50%, 80%, 100%, 120%

**Kusto query for budget monitoring:**

```kusto
AzureMetrics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where MetricName == "TokenTransaction"
| summarize TotalTokens = sum(Total) by bin(TimeGenerated, 1h), Resource
| extend EstimatedCost = TotalTokens * 0.0001 // Eksempel: $0.0001 per token
| project TimeGenerated, Resource, TotalTokens, EstimatedCost
```

### Programmatic Budget Enforcement

**API-level rate limiting:**

```python
import logging

logger = logging.getLogger(__name__)

# Konfigurasjon
MONTHLY_TOKEN_BUDGET = 10_000_000
DAILY_TOKEN_BUDGET = 500_000
ITPM_LIMIT = 100_000  # Input tokens per minute
OTPM_LIMIT = 50_000   # Output tokens per minute

def log_token_usage(response, current_usage):
    usage = response.usage

    # Log current usage
    logger.info(f"Input tokens: {usage.prompt_tokens}")
    logger.info(f"Output tokens: {usage.completion_tokens}")
    logger.info(f"Total tokens: {usage.total_tokens}")

    # Check against limits
    if usage.prompt_tokens > ITPM_LIMIT * 0.8:
        logger.warning("Approaching ITPM limit")

    if usage.completion_tokens > OTPM_LIMIT * 0.8:
        logger.warning("Approaching OTPM limit")

    # Budget enforcement
    new_usage = current_usage + usage.total_tokens
    if new_usage > DAILY_TOKEN_BUDGET:
        raise Exception("Daily token budget exceeded")

    return new_usage
```

**Best practice:** Kombiner soft limits (warnings) med hard limits (enforcement) for å balansere reliability og cost control.

## Token Efficiency Metrics

### Nøkkel-metrikker for Optimalisering

1. **Token-to-response ratio**
   `average_tokens_per_request = total_tokens / request_count`

2. **Input/output ratio**
   `io_ratio = completion_tokens / prompt_tokens`
   Høy ratio = efficient prompt design

3. **Cost per request**
   `cost_per_request = (prompt_tokens * input_price + completion_tokens * output_price) / 1000`

4. **Tokens per user session**
   `session_tokens = sum(tokens) GROUP BY session_id`

5. **Prompt efficiency score**
   `efficiency = output_quality_score / total_tokens`

### Kusto Query for Efficiency Analysis

```kusto
AzureDiagnostics
| where Category == "RequestResponse"
| extend model = tostring(parse_json(properties_s)['model'])
| extend prompt_tokens = toint(parse_json(properties_s)['usage']['prompt_tokens'])
| extend completion_tokens = toint(parse_json(properties_s)['usage']['completion_tokens'])
| extend total_tokens = toint(parse_json(properties_s)['usage']['total_tokens'])
| summarize
    AvgPromptTokens = avg(prompt_tokens),
    AvgCompletionTokens = avg(completion_tokens),
    AvgTotalTokens = avg(total_tokens),
    IOEfficiency = avg(todecimal(completion_tokens) / todecimal(prompt_tokens)),
    RequestCount = count()
    by model, bin(TimeGenerated, 1d)
| project TimeGenerated, model, AvgPromptTokens, AvgCompletionTokens, IOEfficiency, RequestCount
```

## Chargeback Reporting

### Chargeback Model Components

**1. Data Collection:**
- Gateway logs (APIM) eller Application Insights
- Token usage per business unit/app
- Model pricing data (per 1K tokens)

**2. Cost Calculation:**
```python
# Eksempel pricing (GPT-4o, januar 2026)
INPUT_PRICE_PER_1K = 0.005   # USD
OUTPUT_PRICE_PER_1K = 0.015  # USD
NOK_EXCHANGE_RATE = 10.5

def calculate_cost(prompt_tokens, completion_tokens):
    input_cost = (prompt_tokens / 1000) * INPUT_PRICE_PER_1K
    output_cost = (completion_tokens / 1000) * OUTPUT_PRICE_PER_1K
    total_usd = input_cost + output_cost
    total_nok = total_usd * NOK_EXCHANGE_RATE
    return total_nok
```

**3. Attribution Logic:**
- Per user: Group by user_id
- Per app: Group by application_name
- Per business unit: Group by cost_center tag

**4. Report Generation:**

```kusto
// Monthly chargeback report
ApiManagementGatewayLogs
| where TimeGenerated >= startofmonth(now())
| where tolower(OperationId) in ('completions_create','chatcompletions_create')
| extend business_unit = tostring(parse_json(RequestHeaders)['X-Business-Unit'])
| extend model = tostring(parse_json(BackendResponseBody)['model'])
| extend prompt_tokens = toint(parse_json(parse_json(BackendResponseBody)['usage'])['prompt_tokens'])
| extend completion_tokens = toint(parse_json(parse_json(BackendResponseBody)['usage'])['completion_tokens'])
| summarize
    TotalPromptTokens = sum(prompt_tokens),
    TotalCompletionTokens = sum(completion_tokens),
    RequestCount = count()
    by business_unit, model
| extend InputCostUSD = (TotalPromptTokens / 1000.0) * 0.005
| extend OutputCostUSD = (TotalCompletionTokens / 1000.0) * 0.015
| extend TotalCostUSD = InputCostUSD + OutputCostUSD
| extend TotalCostNOK = TotalCostUSD * 10.5
| project business_unit, model, TotalPromptTokens, TotalCompletionTokens,
          RequestCount, TotalCostUSD, TotalCostNOK
| order by TotalCostNOK desc
```

### Showback vs Chargeback

| Aspekt | Showback | Chargeback |
|--------|----------|------------|
| **Formål** | Informasjon og bevisstgjøring | Faktisk fakturering |
| **Nøyaktighet** | Estimert (akseptabelt med sampling) | Høy presisjon påkrevd |
| **Data store** | Application Insights OK | Event Hubs + dedikert DB |
| **Frekvens** | Ukentlig/månedlig report | Real-time tracking |
| **Implementering** | Enklere | Mer kompleks |

**Anbefaling:** Start med showback for å bygge kostnadsbevissthet, deretter implementer chargeback når forretningskrav og infrastruktur er på plass.

## RAG-spesifikke Considerations

### Token Usage i RAG-pipelines

Azure OpenAI On Your Data (RAG) gjør **to** LLM-kall per brukerforespørsel:

**1. Intent Prompt** — Reformulering av query til search intents
**2. Generation Prompt** — Generering av svar basert på retrieved chunks

**Token breakdown:**

| Komponent | Beskrivelse | Token impact |
|-----------|-------------|--------------|
| Meta prompt | System instructions (inScope param avhengig) | 400-4000 tokens (modell-avhengig) |
| User question + history | Input fra bruker | Cap: 2000 tokens |
| Retrieved chunks | Dokumenter fra search (5-10 chunks @ 1024 tokens) | 5000-10000 tokens |
| Intent generation | Output fra første LLM-kall | ~25 tokens |
| Final response | Output fra andre LLM-kall | ~110 tokens |

**Eksempel (gpt-35-turbo-16k):**
- Generation prompt: 4297 tokens
- Intent prompt: 1366 tokens
- Response output: 111 tokens
- Intent output: 25 tokens
- **Total: ~5800 tokens per spørsmål**

**Optimaliseringstekniker:**
1. Reduser `retrieved_document_count` (default 5)
2. Juster `chunk_size` (default 1024)
3. Øk `strictness` (filtrer irrelevante chunks)
4. Bruk `inScope=True` for kortere meta prompt

### Monitoring RAG Token Usage

```kusto
// Dedicated query for RAG scenarios
AzureDiagnostics
| where Category == "RequestResponse"
| where properties_s contains "data_sources"  // RAG indicator
| extend prompt_tokens = toint(parse_json(properties_s)['usage']['prompt_tokens'])
| extend completion_tokens = toint(parse_json(properties_s)['usage']['completion_tokens'])
| extend total_tokens = toint(parse_json(properties_s)['usage']['total_tokens'])
| extend retrieved_docs = toint(parse_json(properties_s)['data_sources'][0]['parameters']['top_n_documents'])
| summarize
    AvgPromptTokens = avg(prompt_tokens),
    AvgCompletionTokens = avg(completion_tokens),
    AvgTotalTokens = avg(total_tokens),
    AvgRetrievedDocs = avg(retrieved_docs),
    RequestCount = count()
    by bin(TimeGenerated, 1h)
```

## Fine-tuned Models: Spesialkonsiderasjoner

### Tre Kostnadskomponenter

1. **Training cost** — Per token i training file
2. **Hosting cost** — Timepris mens deployed (uansett bruk)
3. **Inference cost** — Per 1000 tokens (input + output)

**Kritisk:** Fine-tuned modeller akkumulerer hosting cost **selv når de ikke brukes**. Etter 15 dager inaktivitet slettes deployment automatisk (modellen bevares, kan redeployes).

**Best practice:**
- Monitor deployment utilization
- Slett unused deployments promptly
- Bruk automation for deployment lifecycle

### Tracking Fine-tuning Costs

```kusto
AzureMetrics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where MetricName in ("FineTuningHours", "FineTuningTokens")
| summarize
    TotalTrainingTokens = sumif(Total, MetricName == "FineTuningTokens"),
    TotalHostingHours = sumif(Total, MetricName == "FineTuningHours")
    by Resource, bin(TimeGenerated, 1d)
| extend TrainingCostUSD = TotalTrainingTokens * 0.00008  // Eksempel pricing
| extend HostingCostUSD = TotalHostingHours * 2.0         // Eksempel: $2/hour
| extend TotalCostUSD = TrainingCostUSD + HostingCostUSD
```

## Provisioned Throughput Units (PTU): Tracking

### PTU vs. Consumption-based Billing

| Billing Model | Token Tracking Approach |
|---------------|-------------------------|
| **Pay-as-you-go** | Track individual tokens, calculate variable cost |
| **PTU** | Track utilization percentage against reserved capacity |

**PTU Metrics:**
- `PTUUtilization` — Percentage of reserved capacity used
- `ProcessedPromptTokens` — Input tokens processed
- Input TPM per PTU — Model-specific (f.eks. 8450 TPM for Llama-3.3-70B)

**Cost model:**
- Fixed monthly cost for PTU reservation
- Cost per token = (Monthly PTU cost) / (Total tokens processed)

### PTU Efficiency Monitoring

```kusto
AzureMetrics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where MetricName == "PTUUtilization"
| summarize AvgUtilization = avg(Average), MaxUtilization = max(Maximum)
    by Resource, bin(TimeGenerated, 1h)
| extend EfficiencyStatus = case(
    AvgUtilization < 50, "Underutilized",
    AvgUtilization >= 50 and AvgUtilization < 80, "Optimal",
    AvgUtilization >= 80, "Consider scaling"
)
```

**Anbefaling:** Kombiner PTU for baseline workload med pay-as-you-go for overflow traffic (via gateway pattern).

## Integrasjon med FinOps Practices

### FinOps Framework Alignment

**1. Inform:**
- Real-time dashboards med token usage
- Trend analysis og forecasting
- Anomaly detection på usage spikes

**2. Optimize:**
- Token efficiency metrics (se "Token Efficiency Metrics")
- Prompt optimization basert på cost/quality ratio
- Model selection guidance (GPT-4o vs. GPT-4o-mini)

**3. Operate:**
- Automated budget enforcement
- Chargeback/showback reporting
- Cost allocation til business units

### Azure Cost Management Integration

**1. Tag Strategy:**
```bash
# Standardized tagging
CostCenter: <code>
BusinessUnit: <name>
Application: <name>
Environment: Prod|Dev|Test
Owner: <email>
```

**2. Cost Analysis Views:**
- Filtrer per tag dimension
- Group by resource, subscription, eller custom tag
- Sammenlign faktisk vs. budsjett

**3. Budgets:**
- Opprett per resource group eller subscription
- Sett alert thresholds (50%, 80%, 100%, 120%)
- Action groups for automated response (webhook, Logic App)

## Best Practices

### 1. Data Store Selection

| Use Case | Recommended Store | Rationale |
|----------|-------------------|-----------|
| Showback (informasjon) | Application Insights | Enkel, innebygd, sampling OK |
| Chargeback (fakturering) | Event Hubs + Synapse/Fabric | Høy presisjon, no sampling |
| Real-time monitoring | Stream Analytics + Power BI | Low latency, streaming dashboards |
| Long-term audit | Azure Storage (cold tier) | Billig, compliance-friendly |

### 2. Attribution Hierarchy

**Prioriter:**
1. **User-level** — Mest granulær, best for interne chargeback
2. **Application-level** — God for multi-tenant SaaS
3. **Business unit-level** — Standard for enterprise showback
4. **Subscription-level** — Minst granulær, enklest å implementere

### 3. Monitoring Frequency

| Metric Type | Collection Frequency | Retention |
|-------------|---------------------|-----------|
| Real-time alerts | Per request | 7 dager |
| Operational dashboards | 1 minutt aggregation | 30 dager |
| Cost reporting | 1 time aggregation | 1 år |
| Audit logs | Per request (full fidelity) | 7 år (compliance) |

### 4. Gateway Pattern Decision Matrix

**Bruk gateway hvis:**
- ✅ Multiple clients eller multiple Azure OpenAI instances
- ✅ Chargeback requirement (nøyaktig attribution)
- ✅ Centralized policy enforcement (rate limiting, content filtering)
- ✅ Near real-time monitoring requirement

**Unngå gateway hvis:**
- ❌ Single client, single Azure OpenAI instance
- ❌ Latency er kritisk (gateway adds ~10-50ms)
- ❌ Simple showback er sufficient

### 5. Cost Optimization Triggers

**Alerts når:**
- Token usage øker >20% week-over-week (anomaly)
- Cost per user > baseline + 2 standard deviations
- PTU utilization < 50% (consider downscaling)
- Fine-tuned model har 0 requests i 7 dager (delete deployment)

## Relaterte Referanser

- **cost-optimization/token-optimization.md** — Teknikker for å redusere token consumption
- **cost-optimization/ptu-vs-payg.md** — Billing model selection
- **monitoring-observability/azure-monitor-integration.md** — Azure Monitor oppsett
- **monitoring-observability/alerting-strategies.md** — Alert configuration patterns
- **architecture/gateway-patterns.md** — API Management for AI workloads
- **mlops-genaiops/evaluation-metrics.md** — Quality vs. cost trade-offs

## Kilder (Microsoft Learn)

1. [Monitor Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/monitor-openai) — Official monitoring guide
2. [Implement advanced monitoring for Azure OpenAI in Foundry Models through a gateway](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-monitoring) *(Verified MCP 2026-04)* — Gateway patterns for usage tracking. Ny brukscase dokumentert: audit av model inputs/outputs for threat detection og data exfiltration detection. Merk: gateway monitoring kan bli single point of failure — vurder redundans.
3. [Plan to manage costs for Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/manage-costs) — Cost management strategies
4. [Token usage estimation for Azure OpenAI On Your Data](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data) — RAG-specific token calculations
5. [Understanding costs associated with PTU](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/provisioned-throughput-onboarding) — PTU billing model
6. [Application design for AI workloads](https://learn.microsoft.com/en-us/azure/well-architected/ai/application-design#consider-nonfunctional-requirements) — Cost and chargeback scenarios
7. [Architecture strategies for cost data](https://learn.microsoft.com/en-us/azure/well-architected/cost-optimization/collect-review-cost-data#generate-cost-reports) — Chargeback vs. showback

## For Cosmo

Når du diskuterer token usage tracking og attribution, vektlegg **gateway-pattern** som game-changer for chargeback-scenarioer. Mange organisasjoner undervurderer betydningen av nøyaktig attribution før de skalerer AI-løsninger til produksjon.

**Key talking points:**
1. Native Azure OpenAI telemetri har **masket IP** — ikke sufficient for per-app attribution
2. Gateway (APIM) gir **full observability** + centralized policy enforcement
3. Forskjellen mellom **showback** (informasjon) og **chargeback** (fakturering) krever ulik data fidelity
4. RAG-workloads har **2x token overhead** (intent + generation) — må planlegges inn i budsjett
5. Fine-tuned models har **hosting cost uavhengig av bruk** — krev proaktiv lifecycle management

Hvis løsningen skal brukes av **flere business units** eller krever **intern fakturering**, er gateway pattern ikke optional — det er kritisk arkitektur-komponent.

**Norsk offentlig sektor-vinkling:**
For offentlige virksomheter med krav til internprising (eks. NAV, Skatteetaten, fylkeskommuner med delte IT-tjenester), er nøyaktig cost attribution **ikke bare best practice — det er governance-krav**. Kombiner med Azure Cost Management tags for å oppfylle økonomiregelverket sitt krav til transparent ressursbruk.