open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity

docs(ms-ai-architect): KB-refresh critical cost 6/8 — token-counting verifisert mot fine-tuning-cost (RFT) + prompt-caching

Browse source 
Verifisert mot live Microsoft Learn (foundry/openai/how-to/fine-tuning-cost-management,
prompt-caching, use-your-data):
- RFT (Reinforcement Fine-Tuning) lagt til (filen dekket kun SFT/DPO): tidsbasert «pris = treningstid
  × timepris + grader-tokens», ~$100/time core training for o4-mini-2025-04-16, per-job-tak $5000
  (pauses + deploybart checkpoint ved tak, kan resume uten videre pristak). Cost-control-strategier med.
- Cache-TTL-tabell hadde FEIL akse («Foundry Classic vs AI Services») → korrigert til in-memory
  (5-10 min/maks 1t, alle GPT-4o+) vs extended (24h, GPT-5-serien+gpt-4.1, default for nyere).
- prompt_cache_key ~15 req/min overflow-caveat lagt til.
- On Your Data deprecation 2026-10-14 flagget (3 steder: token-est-seksjon, Foundry-seksjon, kilde #6).
- tiktoken model-set manglet o1-mini/o3-mini → lagt til.
- Header 2026-02→2026-06, verification-date 2026-02-04→2026-06. Priser urørt (kanonisk peker). validate 239/0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-06-23 22:53:35 +02:00
commit 692b3f75d8
1 changed files with 21 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

31
skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md
View file

@ -1,6 +1,6 @@
# Token Counting and Optimization Strategies # Token Counting and Optimization Strategies
**Last updated:** 2026-02 **Last updated:** 2026-06
**Status:** GA **Status:** GA
**Category:** Cost Optimization & FinOps for AI **Category:** Cost Optimization & FinOps for AI
@ -56,7 +56,7 @@ def num_tokens_from_messages(messages, model="gpt-4o"):
except KeyError: except KeyError:
encoding = tiktoken.get_encoding("o200k_base") encoding = tiktoken.get_encoding("o200k_base")
if model in {"gpt-4o", "gpt-4o-mini", "gpt-5", "gpt-4.1", "o1", "o3", "o4-mini"}: if model in {"gpt-4o", "gpt-4o-mini", "gpt-5", "gpt-4.1", "o1", "o1-mini", "o3", "o3-mini", "o4-mini"}:
tokens_per_message = 3 tokens_per_message = 3
tokens_per_name = 1 tokens_per_name = 1
@ -97,6 +97,8 @@ string TrimToTokenLimit(string text, int maxTokens, Tokenizer tokenizer)
### Token Usage Estimation (Azure OpenAI On Your Data) ### Token Usage Estimation (Azure OpenAI On Your Data)
> ⚠️ **On Your Data er deprecated og pensjoneres 2026-10-14.** Migrer til Foundry Agent Service + Foundry IQ for grounding. Token-estimering-metodikken under er fortsatt illustrativ for RAG-token-budsjettering.
```python ```python
import tiktoken import tiktoken
@ -127,14 +129,14 @@ token_output = TokenEstimator().estimate_tokens(input_text)
|-----------|-------|--------| |-----------|-------|--------|
| Minimum prompt length | 1 024 tokens | Cache hit kan først oppnås | | Minimum prompt length | 1 024 tokens | Cache hit kan først oppnås |
| Granularitet | 128 tokens | Etter første 1024 tokens, cache hit per 128 tokens | | Granularitet | 128 tokens | Etter første 1024 tokens, cache hit per 128 tokens |
| Cache TTL | 24 timer | Microsoft Foundry Classic | | Cache TTL (in-memory) | 5-10 min idle, max 1 time | Alle GPT-4o eller nyere |
| Cache TTL | 5-10 min idle, max 1 time | Azure AI Services | | Cache TTL (extended) | Opptil 24 timer (`prompt_cache_retention: "24h"`) | GPT-5-serien + gpt-4.1; default 24h for nyere modeller |
| Kostnad (Standard) | 50% rabatt på cached tokens | Varierer per modell | | Kostnad (Standard) | 50% rabatt på cached tokens | Varierer per modell |
| Kostnad (Provisioned) | Opptil 100% rabatt | Inkludert i PTU-pris | | Kostnad (Provisioned) | Opptil 100% rabatt | Inkludert i PTU-pris |
**Design-prinsipper:** **Design-prinsipper:**
1. **Plasser repetitivt innhold først** — system messages, instructions, reference docs 1. **Plasser repetitivt innhold først** — system messages, instructions, reference docs
2. **Bruk `prompt_cache_key`** for å påvirke routing og øke cache hit rate 2. **Bruk `prompt_cache_key`** for å påvirke routing og øke cache hit rate (NB: samme prefix + key over ~15 req/min → noen requests overflower til ekstra maskiner, og cache-effektiviteten faller)
3. **Unngå variasjon i første 1024 tokens** — én endring = cache miss 3. **Unngå variasjon i første 1024 tokens** — én endring = cache miss
```python ```python
@ -277,6 +279,8 @@ chunks = text_splitter.split_documents(pages)
Cost = # training tokens × # epochs × price per token Cost = # training tokens × # epochs × price per token
``` ```
> **RFT (Reinforcement Fine-Tuning)** bruker en *tidsbasert* kostnadsmodell (ikke token-basert) med per-job-tak på $5 000 — se «Fine-Tuning Costs» under.
**Token validation pre-training:** **Token validation pre-training:**
```python ```python
import json import json
@ -371,7 +375,7 @@ Er prompt >1024 tokens og repetitiv?
### Microsoft Foundry ### Microsoft Foundry
**Token Usage Estimation (On Your Data):** **Token Usage Estimation (On Your Data — deprecated, retires 2026-10-14):**
- Intent prompt: ~1 366 tokens (gjennomsnitt) - Intent prompt: ~1 366 tokens (gjennomsnitt)
- Generation prompt: ~4 297 tokens (gjennomsnitt) - Generation prompt: ~4 297 tokens (gjennomsnitt)
- Response: ~111 tokens (gjennomsnitt) - Response: ~111 tokens (gjennomsnitt)
@ -474,6 +478,13 @@ Image tokens (high-res) = (# tiles × 170) + 85
- Global Standard: $2 per 1M tokens (gpt-4.1, eksempel) - Global Standard: $2 per 1M tokens (gpt-4.1, eksempel)
- Developer (spot): 50% rabatt, kan bli paused/resumed - Developer (spot): 50% rabatt, kan bli paused/resumed
**Training (RFT — Reinforcement Fine-Tuning):**
- Tidsbasert, IKKE token-basert: `pris = treningstid (timer) × timepris + grader-inferencing-tokens (hvis model-grader brukes)`
- Timepris: ~$100/time core training for `o4-mini-2025-04-16` (eksempel; se pricing-side for gjeldende sats)
- **Per-job-tak: $5 000** — når taket nås pauses training og et deploybart checkpoint opprettes; du kan validere og resume (videre billing uten pristak)
- Ingen kostnad for kø, failed jobs eller jobber kansellert før training starter; ved kansellering belastes arbeid frem til siste checkpoint
- Kostnadskontroll: `reasoning_effort=Low`, begrens validering/`eval_samples`, velg minste grader-modell, juster `compute_multiplier`
**Hosting:** **Hosting:**
- $1.70/time per deployment (Standard/Global Standard) - $1.70/time per deployment (Standard/Global Standard)
- Påløper selv om modellen ikke brukes - Påløper selv om modellen ikke brukes
@ -570,18 +581,18 @@ def track_token_usage(prompt, completion, model="gpt-4o"):
## Kilder og verifisering ## Kilder og verifisering
**Microsoft Learn Documentation:** **Microsoft Learn Documentation:**
1. [Prompt caching - Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/prompt-caching) 1. [Prompt caching - Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/prompt-caching) — **Verified MCP 2026-06** (in-memory + extended 24h)
2. [Work with chat completions models - Token management](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/chatgpt#manage-conversations) 2. [Work with chat completions models - Token management](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/chatgpt#manage-conversations)
3. [Plan and manage costs for Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs) 3. [Plan and manage costs for Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs)
4. [Token counting in AI - Dynamics 365 Business Central](https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-system-app-token-counting) 4. [Token counting in AI - Dynamics 365 Business Central](https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-system-app-token-counting)
5. [Use Microsoft.ML.Tokenizers for text tokenization](https://learn.microsoft.com/en-us/dotnet/ai/how-to/use-tokenizers) 5. [Use Microsoft.ML.Tokenizers for text tokenization](https://learn.microsoft.com/en-us/dotnet/ai/how-to/use-tokenizers)
6. [Azure OpenAI On Your Data - Token usage estimation](https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data) 6. [Azure OpenAI On Your Data - Token usage estimation](https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data) — ⚠️ *deprecated, retires 2026-10-14*
7. [Cost management for fine-tuning](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning-cost-management) 7. [Cost management for fine-tuning](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning-cost-management) — **Verified MCP 2026-06** (SFT/DPO + RFT tidsbasert + $5000-tak)
**OpenAI Resources:** **OpenAI Resources:**
8. [OpenAI Cookbook - Token counting](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_format_inputs_to_ChatGPT_models.ipynb) 8. [OpenAI Cookbook - Token counting](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_format_inputs_to_ChatGPT_models.ipynb)
9. [tiktoken GitHub repository](https://github.com/openai/tiktoken) 9. [tiktoken GitHub repository](https://github.com/openai/tiktoken)
**Verification Date:** 2026-02-04 **Verification Date:** 2026-06
**MCP Calls:** 4 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1) **MCP Calls:** 4 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)
**Confidence Level:** High — all data sourced from official Microsoft Learn documentation and verified OpenAI tooling **Confidence Level:** High — all data sourced from official Microsoft Learn documentation and verified OpenAI tooling