docs(architect): weekly KB update — 106 files refreshed (2026-04)
Updates across all 5 skills: ms-ai-advisor, ms-ai-engineering, ms-ai-governance, ms-ai-security, ms-ai-infrastructure. Key changes: - Language Services (Custom Text Classification, Text Analytics, QnA): retirement warning 2029-03-31, migration guides to Foundry/GPT-4o - Agentic Retrieval: 50M free reasoning tokens/month (Public Preview) - Computer Use: Claude Sonnet 4.5 (preview) + OpenAI CUA models - Agent Registry: Risks column (M365 E7), user-shared/org-published types - Declarative agents: schema v1.5 → v1.6, Store validation requirements - MLflow 3: 13 built-in LLM judges, production monitoring, Genie Code - AG-UI HITL: ApprovalRequiredAIFunction (C#) + @tool(approval_mode) (Python) - Entra ID Ignite 2025: Agent ID Admin/Developer RBAC roles, Conditional Access - Security Copilot: 400 SCU/month per 1000 M365 E5 licenses, auto-provisioned - Fast Transcription API: phrase lists, 14-language multi-lingual transcription - Azure Monitor Workbooks: Bicep support, RBAC specifics - Power Platform Copilot: data residency (Norway/Europe → EU DB, Bing → USA) - RAG security-rbac: 4-approach table (GA + 3 preview access control methods) - IaC MLOps: Well-Architected OE:05 principles, Bicep/Terraform patterns - Translator: image file batch translation Preview (JPEG/PNG/BMP/WebP) All 106 files: Last updated 2026-04 | Verified: MCP 2026-04 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
0eb30fa853
commit
6645e93205
104 changed files with 1986 additions and 520 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Real-Time Reasoning and Performance Optimization
|
||||
|
||||
**Last updated:** 2026-02
|
||||
**Last updated:** 2026-04 | Verified: MCP 2026-04
|
||||
**Status:** GA (Realtime API: Public Preview)
|
||||
**Category:** Prompt Engineering & LLM Optimization
|
||||
|
||||
|
|
@ -307,6 +307,25 @@ Deployment C: Chatbot (variabel prompt, medium output)
|
|||
- **Non-streaming:** End-to-end Request Time
|
||||
- **Streaming:** Time to Response (TTFT), Average Token Generation Rate
|
||||
|
||||
### Azure Speech Service (TTS Latency)
|
||||
|
||||
**Teknikker for å redusere speech synthesis latency (Verified MCP 2026-04):**
|
||||
|
||||
| Teknikk | Effekt |
|
||||
|---------|--------|
|
||||
| **Streaming (AudioDataStream)** | Start avspilling ved første audio-chunk; ikke vent på komplett audio |
|
||||
| **Pre-connect** | Åpne WebSocket-forbindelsen proaktivt mens bruker snakker; kall `SpeakTextAsync` når svar er klart |
|
||||
| **Gjenbruk SpeechSynthesizer** | Unngå ny TCP/SSL/HTTP-handshake per request; bruk object pool |
|
||||
| **Komprimert lyd** | MP3 (48kbps) vs PCM (384kbps) — 87% lavere nettverkspayload for mobil/ustabile nettverk |
|
||||
| **Text streaming (WebSocket v2)** | Send GPT-output til TTS chunk for chunk via `wss://{region}.tts.speech.microsoft.com/cognitiveservices/websocket/v2`. Ideelt for real-time AI-dialoger. |
|
||||
|
||||
**Latency-metrikker fra Speech SDK:**
|
||||
- `first byte client latency` — fra syntese starter til første audio-chunk mottas (inkl. nettverks-RTT)
|
||||
- `finish client latency` — fra syntese starter til all lyd er mottatt
|
||||
- `first byte service latency` — behandlingstid på Azure TTS-siden
|
||||
|
||||
**Anbefaling:** For sanntids AI-dialoger (GPT + TTS), kombiner Realtime API (audio in/out) med Speech SDK text streaming for hybrid norsk/engelsk-løsninger.
|
||||
|
||||
### Copilot Studio
|
||||
|
||||
**Relevans:** Copilot Studio kan integrere Azure OpenAI custom models via Power Platform connectors.
|
||||
|
|
@ -477,9 +496,9 @@ Deployment C: Chatbot (variabel prompt, medium output)
|
|||
[https://learn.microsoft.com/en-us/azure/ai-foundry/openai/realtime-audio-quickstart](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/realtime-audio-quickstart)
|
||||
Hentet: januar 2026. Kode-eksempler for Python, JavaScript, deployment steps.
|
||||
|
||||
4. **Lower speech synthesis latency using Speech SDK**
|
||||
4. **Lower speech synthesis latency using Speech SDK** (Re-verified MCP 2026-04)
|
||||
[https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-lower-speech-synthesis-latency](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-lower-speech-synthesis-latency)
|
||||
Hentet: januar 2026. Dekker text streaming for TTS (komplementær til Realtime API).
|
||||
Hentet: januar 2026, re-verified april 2026. Dekker: first byte latency vs finish latency, streaming via AudioDataStream, pre-connect og SpeechSynthesizer-gjenbruk (object pool), komprimert lyd (MP3 48kbps vs PCM 384kbps), text streaming via WebSocket v2 (wss endpoint) for real-time GPT-output vocalization.
|
||||
|
||||
**Verification steps:**
|
||||
|
||||
|
|
@ -487,6 +506,7 @@ Deployment C: Chatbot (variabel prompt, medium output)
|
|||
2. ✅ **Realtime API models:** Bekreftet at `gpt-4o-mini-realtime-preview` og `gpt-4o-realtime-preview` er tilgjengelige i East US 2 / Sweden Central.
|
||||
3. ✅ **VAD modes:** Bekreftet at `server_vad`, `semantic_vad`, og `none` er supported turn detection types.
|
||||
4. ✅ **Latency metrics:** Bekreftet at Time to Response (TTFT) og Average Token Generation Rate er recommended metrics for streaming.
|
||||
5. ✅ **Speech latency:** first byte client latency og AudioDataStream-streaming bekreftet. Text streaming via WebSocket v2 bekreftet for C#, Python.
|
||||
5. ⚠️ **Pricing:** Audio token pricing ikke eksplisitt i dokumentasjon per januar 2026. Brukt representative estimates basert på historisk OpenAI pricing structure.
|
||||
|
||||
**Confidence level:** Høy (✅) for tekniske detaljer, Middels (⚠️) for pricing og production-readiness av Realtime API (public preview).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue