From 692b3f75d80d024fb191203adc393558291a9ca2 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Tue, 23 Jun 2026 22:53:35 +0200 Subject: [PATCH] =?UTF-8?q?docs(ms-ai-architect):=20KB-refresh=20critical?= =?UTF-8?q?=20cost=206/8=20=E2=80=94=20token-counting=20verifisert=20mot?= =?UTF-8?q?=20fine-tuning-cost=20(RFT)=20+=20prompt-caching?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verifisert mot live Microsoft Learn (foundry/openai/how-to/fine-tuning-cost-management, prompt-caching, use-your-data): - RFT (Reinforcement Fine-Tuning) lagt til (filen dekket kun SFT/DPO): tidsbasert «pris = treningstid × timepris + grader-tokens», ~$100/time core training for o4-mini-2025-04-16, per-job-tak $5000 (pauses + deploybart checkpoint ved tak, kan resume uten videre pristak). Cost-control-strategier med. - Cache-TTL-tabell hadde FEIL akse («Foundry Classic vs AI Services») → korrigert til in-memory (5-10 min/maks 1t, alle GPT-4o+) vs extended (24h, GPT-5-serien+gpt-4.1, default for nyere). - prompt_cache_key ~15 req/min overflow-caveat lagt til. - On Your Data deprecation 2026-10-14 flagget (3 steder: token-est-seksjon, Foundry-seksjon, kilde #6). - tiktoken model-set manglet o1-mini/o3-mini → lagt til. - Header 2026-02→2026-06, verification-date 2026-02-04→2026-06. Priser urørt (kanonisk peker). validate 239/0. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../token-counting-optimization.md | 31 +++++++++++++------ 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md b/skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md index e08eca2..5dc246a 100644 --- a/skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md +++ b/skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md @@ -1,6 +1,6 @@ # Token Counting and Optimization Strategies -**Last updated:** 2026-02 +**Last updated:** 2026-06 **Status:** GA **Category:** Cost Optimization & FinOps for AI @@ -56,7 +56,7 @@ def num_tokens_from_messages(messages, model="gpt-4o"): except KeyError: encoding = tiktoken.get_encoding("o200k_base") - if model in {"gpt-4o", "gpt-4o-mini", "gpt-5", "gpt-4.1", "o1", "o3", "o4-mini"}: + if model in {"gpt-4o", "gpt-4o-mini", "gpt-5", "gpt-4.1", "o1", "o1-mini", "o3", "o3-mini", "o4-mini"}: tokens_per_message = 3 tokens_per_name = 1 @@ -97,6 +97,8 @@ string TrimToTokenLimit(string text, int maxTokens, Tokenizer tokenizer) ### Token Usage Estimation (Azure OpenAI On Your Data) +> ⚠️ **On Your Data er deprecated og pensjoneres 2026-10-14.** Migrer til Foundry Agent Service + Foundry IQ for grounding. Token-estimering-metodikken under er fortsatt illustrativ for RAG-token-budsjettering. + ```python import tiktoken @@ -127,14 +129,14 @@ token_output = TokenEstimator().estimate_tokens(input_text) |-----------|-------|--------| | Minimum prompt length | 1 024 tokens | Cache hit kan først oppnås | | Granularitet | 128 tokens | Etter første 1024 tokens, cache hit per 128 tokens | -| Cache TTL | 24 timer | Microsoft Foundry Classic | -| Cache TTL | 5-10 min idle, max 1 time | Azure AI Services | +| Cache TTL (in-memory) | 5-10 min idle, max 1 time | Alle GPT-4o eller nyere | +| Cache TTL (extended) | Opptil 24 timer (`prompt_cache_retention: "24h"`) | GPT-5-serien + gpt-4.1; default 24h for nyere modeller | | Kostnad (Standard) | 50% rabatt på cached tokens | Varierer per modell | | Kostnad (Provisioned) | Opptil 100% rabatt | Inkludert i PTU-pris | **Design-prinsipper:** 1. **Plasser repetitivt innhold først** — system messages, instructions, reference docs -2. **Bruk `prompt_cache_key`** for å påvirke routing og øke cache hit rate +2. **Bruk `prompt_cache_key`** for å påvirke routing og øke cache hit rate (NB: samme prefix + key over ~15 req/min → noen requests overflower til ekstra maskiner, og cache-effektiviteten faller) 3. **Unngå variasjon i første 1024 tokens** — én endring = cache miss ```python @@ -277,6 +279,8 @@ chunks = text_splitter.split_documents(pages) Cost = # training tokens × # epochs × price per token ``` +> **RFT (Reinforcement Fine-Tuning)** bruker en *tidsbasert* kostnadsmodell (ikke token-basert) med per-job-tak på $5 000 — se «Fine-Tuning Costs» under. + **Token validation pre-training:** ```python import json @@ -371,7 +375,7 @@ Er prompt >1024 tokens og repetitiv? ### Microsoft Foundry -**Token Usage Estimation (On Your Data):** +**Token Usage Estimation (On Your Data — deprecated, retires 2026-10-14):** - Intent prompt: ~1 366 tokens (gjennomsnitt) - Generation prompt: ~4 297 tokens (gjennomsnitt) - Response: ~111 tokens (gjennomsnitt) @@ -474,6 +478,13 @@ Image tokens (high-res) = (# tiles × 170) + 85 - Global Standard: $2 per 1M tokens (gpt-4.1, eksempel) - Developer (spot): 50% rabatt, kan bli paused/resumed +**Training (RFT — Reinforcement Fine-Tuning):** +- Tidsbasert, IKKE token-basert: `pris = treningstid (timer) × timepris + grader-inferencing-tokens (hvis model-grader brukes)` +- Timepris: ~$100/time core training for `o4-mini-2025-04-16` (eksempel; se pricing-side for gjeldende sats) +- **Per-job-tak: $5 000** — når taket nås pauses training og et deploybart checkpoint opprettes; du kan validere og resume (videre billing uten pristak) +- Ingen kostnad for kø, failed jobs eller jobber kansellert før training starter; ved kansellering belastes arbeid frem til siste checkpoint +- Kostnadskontroll: `reasoning_effort=Low`, begrens validering/`eval_samples`, velg minste grader-modell, juster `compute_multiplier` + **Hosting:** - $1.70/time per deployment (Standard/Global Standard) - Påløper selv om modellen ikke brukes @@ -570,18 +581,18 @@ def track_token_usage(prompt, completion, model="gpt-4o"): ## Kilder og verifisering **Microsoft Learn Documentation:** -1. [Prompt caching - Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/prompt-caching) +1. [Prompt caching - Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/prompt-caching) — **Verified MCP 2026-06** (in-memory + extended 24h) 2. [Work with chat completions models - Token management](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/chatgpt#manage-conversations) 3. [Plan and manage costs for Azure OpenAI](https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs) 4. [Token counting in AI - Dynamics 365 Business Central](https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/ai-system-app-token-counting) 5. [Use Microsoft.ML.Tokenizers for text tokenization](https://learn.microsoft.com/en-us/dotnet/ai/how-to/use-tokenizers) -6. [Azure OpenAI On Your Data - Token usage estimation](https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data) -7. [Cost management for fine-tuning](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning-cost-management) +6. [Azure OpenAI On Your Data - Token usage estimation](https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data) — ⚠️ *deprecated, retires 2026-10-14* +7. [Cost management for fine-tuning](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning-cost-management) — **Verified MCP 2026-06** (SFT/DPO + RFT tidsbasert + $5000-tak) **OpenAI Resources:** 8. [OpenAI Cookbook - Token counting](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_format_inputs_to_ChatGPT_models.ipynb) 9. [tiktoken GitHub repository](https://github.com/openai/tiktoken) -**Verification Date:** 2026-02-04 +**Verification Date:** 2026-06 **MCP Calls:** 4 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1) **Confidence Level:** High — all data sourced from official Microsoft Learn documentation and verified OpenAI tooling