From 0bc8dab22c964dc2d36c812568f7b45ad3cf4e02 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Wed, 24 Jun 2026 11:38:24 +0200 Subject: [PATCH] =?UTF-8?q?docs(ms-ai-architect):=20KB-refresh=20MEDIUM=20?= =?UTF-8?q?13/33=20=E2=80=94=20token-optimization:=20prompt-caching=20in-m?= =?UTF-8?q?emory=20vs=20extended=2024h=20(prompt=5Fcache=5Fretention)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../token-optimization-and-efficiency.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/skills/ms-ai-advisor/references/prompt-engineering/token-optimization-and-efficiency.md b/skills/ms-ai-advisor/references/prompt-engineering/token-optimization-and-efficiency.md index 99cd882..a096673 100644 --- a/skills/ms-ai-advisor/references/prompt-engineering/token-optimization-and-efficiency.md +++ b/skills/ms-ai-advisor/references/prompt-engineering/token-optimization-and-efficiency.md @@ -1,6 +1,6 @@ # Token Optimization and Cost Efficiency -**Last updated:** 2026-02 +**Last updated:** 2026-06-24 **Status:** GA **Category:** Prompt Engineering & LLM Optimization @@ -95,7 +95,7 @@ response = client.chat.completions.create( ### 2. Prompt Caching (automatisk funksjonalitet) -**Status:** GA for GPT-4o, GPT-4.1-serien, o-series (o1, o3-mini, o4-mini) +**Status:** In-memory caching GA for alle Azure OpenAI-modeller GPT-4o eller nyere (chat/completion/responses/realtime-operasjoner). Extended retention (24t) støttes for GPT-5-serien (gpt-5/5.1/5.1-codex/5.2/5.3-codex/5.4 m.fl.) + gpt-4.1. Prompt caching reduserer kostnader ved å cache de første 1024+ tokens i en prompt. Når identisk innhold gjentas, betaler du redusert pris for cached tokens. @@ -106,9 +106,11 @@ Prompt caching reduserer kostnader ved å cache de første 1024+ tokens i en pro | **Minimum prompt-lengde** | 1024 tokens | | **Cache-treff** | Første 1024 tokens må være identiske | | **Granularitet** | Cache-treff per 128 tokens etter første 1024 | -| **Cache-levetid** | 5-10 min inaktivitet (max 1 time for Microsoft Foundry; 24t for Foundry Models) | +| **Cache-levetid** | *In-memory* (default t.o.m. gpt-5.4): ryddes typisk innen 5–10 min inaktivitet, alltid innen 1 time. *Extended retention* (opptil 24t) via `prompt_cache_retention: "24h"` — default for nyere modeller | | **Deling** | Ikke delt mellom Azure subscriptions | +> **Data-residens:** In-memory caching er kompatibel med alle data-residency-regioner. Extended retention lagrer data midlertidig på GPU-maskiner og holder data in-region kun ved Regional Standard / Regional Provisioned deployment-typer. + #### Design for cache-hits ```python