feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command

Add /ultraresearch-local for structured research combining local codebase
analysis with external knowledge via parallel agent swarms. Produces research
briefs with triangulation, confidence ratings, and source quality assessment.

New command: /ultraresearch-local with modes --quick, --local, --external, --fg.
New agents: research-orchestrator (opus), docs-researcher, community-researcher,
security-researcher, contrarian-researcher, gemini-bridge (all sonnet).
New template: research-brief-template.md.

Integration: --research flag in /ultraplan-local accepts pre-built research
briefs (up to 3), enriches the interview and exploration phases. Planning
orchestrator cross-references brief findings during synthesis.

Design principle: Context Engineering — right information to right agent at
right time. Research briefs are structured artifacts in the pipeline:
ultraresearch → brief → ultraplan --research → plan → ultraexecute.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-08 08:58:35 +02:00
commit baa2d0220b
488 changed files with 213221 additions and 0 deletions

View file

@ -0,0 +1,552 @@
# Real-Time Streaming and Live Monitoring Dashboards
**Last updated:** 2026-02
**Status:** GA
**Category:** Monitoring & Observability
---
## Introduksjon
Real-time streaming og live monitoring er kritiske kapabiliteter for operasjonell overvåking av AI-applikasjoner i produksjon. Mens tradisjonell logging og metrics aggregeres over tid (typisk 1-5 minutter), tilbyr real-time løsninger innsikt med under ett sekunds latency, noe som er essensielt for debugging, incident response og operasjonell overvåking av AI-tjenester.
Microsoft-stakken tilbyr to primære løsninger: **Application Insights Live Metrics** for sanntidsovervåking av applikasjonsytelse med 1-sekunds latency, og **Fabric Real-Time Intelligence** med Real-Time Dashboards for streaming analytics på tvers av multiple datakilder. Live Metrics er optimalisert for utviklere som trenger umiddelbar tilbakemelding under deployment og debugging, mens Real-Time Intelligence er bygget for operasjonelle scenarier som krever kontinuerlig overvåking av store datavolumer.
Begge løsningene har ulike arkitekturer: Live Metrics bruker en dedikert push-basert control channel som streamer data direkte fra applikasjonen til portalen uten persistering, mens Real-Time Dashboards benytter Kusto Query Language (KQL) mot eventhouses eller Azure Data Explorer med refresh-intervaller ned til 10 sekunder.
---
## Kjernekomponenter
### Application Insights Live Metrics
| Komponent | Beskrivelse | Latency |
|-----------|-------------|---------|
| **Live Metrics Stream** | Push-basert streaming av telemetri fra app til portal | < 1 sekund |
| **Custom Filters** | Real-time filtering på URL, duration, telemetry type | Real-time |
| **Server Instance Filtering** | Isoler spesifikke server-instanser for debugging | Real-time |
| **Performance Counters** | Windows performance counters (CPU, memory, requests) | < 1 sekund |
| **Exception Stack Traces** | Full stack traces for exceptions når de skjer | Real-time |
| **Control Channel** | Secure channel for filter-signaler (krever Entra ID auth) | N/A |
**Støttede plattformer:**
```csharp
// ASP.NET Core - OpenTelemetry (anbefalt, enabled by default)
builder.Services.AddOpenTelemetry().UseAzureMonitor(options => {
options.EnableLiveMetrics = true; // Default: true
});
// ASP.NET Core - Classic API (manual config)
using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse;
var quickPulseProcessor = null;
config.DefaultTelemetrySink.TelemetryProcessorChainBuilder
.Use((next) => {
quickPulseProcessor = new QuickPulseTelemetryProcessor(next);
return quickPulseProcessor;
})
.Build();
var quickPulseModule = new QuickPulseTelemetryModule();
quickPulseModule.Initialize(config);
quickPulseModule.RegisterTelemetryProcessor(quickPulseProcessor);
```
**Nøkkelegenskaper:**
- **On-demand streaming:** Data sendes KUN når Live Metrics-panen er åpen (spart kostnader)
- **Ingen persistering:** Data vises kun i real-time, ikke lagret for historisk analyse
- **Sampling:** Stack traces og exceptions samples, men alle metrics og counters sendes
- **Gratis:** Ingen ekstra kostnad for Live Metrics data (kun standard ingestion)
### Fabric Real-Time Intelligence & Real-Time Dashboards
| Komponent | Beskrivelse | Refresh Rate |
|-----------|-------------|--------------|
| **Real-Time Dashboard** | No-code dashboard med KQL queries | 10 sekunder - manuell |
| **Eventhouse** | Time-series optimalisert database for streaming data | Subsecond ingestion |
| **Eventstream** | No-code streaming pipelines med transformasjoner | Near real-time |
| **Data Activator** | Event detection med subsecond latency, trigger actions | < 1 sekund |
| **Copilot for Dashboards** | AI-generert dashboard fra natural language prompts | N/A |
**Støttede datakilder:**
| Datakilde | Use Case | Latency |
|-----------|----------|---------|
| Eventhouse | Fabric-native streaming events | Subsecond |
| Azure Data Explorer | Log og telemetry analytics | < 5 sekunder |
| Application Insights | App performance metrics | 1-5 minutter* |
| Log Analytics | Azure resource logs | 1-5 minutter* |
*Application Insights og Log Analytics har inherent ingestion latency (1-5 min), men kan queries via Real-Time Dashboard.
**KQL Query Eksempel (streaming dashboard):**
```kusto
// Real-time monitoring av AI request latency
AIRequests
| where timestamp > ago(5m)
| summarize
AvgDuration = avg(duration),
P95Duration = percentile(duration, 95),
RequestCount = count()
by bin(timestamp, 10s), operation_Name
| render timechart
```
**Auto-refresh konfigurasjon:**
- Minimum refresh interval: **10 sekunder**
- Anbefalt for high-volume scenarier: **30-60 sekunder** (reduserer compute cost)
- Manual refresh: Alltid tilgjengelig for ad-hoc analyse
---
## Arkitekturmønstre
### Mønster 1: Live Metrics for CI/CD Validation
**Scenario:** Validere deployment i sanntid under release pipeline.
**Arkitektur:**
```
Developer → Deploy til Azure → App starter → Live Metrics åpnes i portal
↓
Monitor exceptions, latency, dependencies
↓
Validate fix → Close Live Metrics (stop streaming)
```
**Fordeler:**
- Umiddelbar feedback (< 1 sekund latency)
- Ingen setup utover Application Insights instrumentation
- Gratis (no-cost streaming)
- Filter på spesifikk server instance under rolling deployment
**Ulemper:**
- Krever manuell observasjon (ingen alerting)
- Data persisteres ikke (kun for live debugging)
- Control channel må sikres med Entra ID for production (API keys deprecated Sept 2025)
**Når bruke:**
- Debugging av nye deployments
- Load testing validation
- Exception tracking under release
- Server-specific performance issues
### Mønster 2: Real-Time Dashboard for Operations
**Scenario:** Kontinuerlig overvåking av AI-tjenester i produksjon med alerting.
**Arkitektur:**
```
AI App → Application Insights → Eventhouse (via Eventstream)
↓
Real-Time Dashboard (10s refresh)
↓
Data Activator → Teams/Email Alert
```
**Fordeler:**
- Persistent storage i Eventhouse (long-term analytics)
- Proactive alerting via Data Activator
- Multi-source dashboards (combine App Insights + Azure Monitor)
- No-code dashboard creation med Copilot
**Ulemper:**
- Higher latency enn Live Metrics (10s min refresh)
- Krever Fabric capacity (kostnader)
- Mer kompleks setup (Eventstream pipeline)
**Når bruke:**
- Production operations med SLA requirements
- Multi-service monitoring (microservices)
- Compliance requirements (data retention)
- Business stakeholder visibility (shareable dashboards)
### Mønster 3: Hybrid - Live Metrics + Real-Time Dashboard
**Scenario:** Kombinere ad-hoc debugging (Live Metrics) med persistent monitoring (Real-Time Dashboard).
**Arkitektur:**
```
AI App → Application Insights ─┬→ Live Metrics (on-demand)
└→ Eventhouse → Real-Time Dashboard
↓
Data Activator Alerts
```
**Fordeler:**
- Best of both worlds: Live debugging + persistent monitoring
- Cost-efficient (Live Metrics kun når åpen, dashboard alltid tilgjengelig)
- Enkel escalation fra dashboard til Live Metrics for deep-dive
**Ulemper:**
- Dobbel ingestion (App Insights + Eventhouse) hvis begge brukes samtidig
- Mer kompleks setup
**Når bruke:**
- Enterprise AI-løsninger med både dev og ops teams
- Critical workloads som krever både proactive alerting og reactive debugging
---
## Beslutningsveiledning
### Når bruke Live Metrics
| Kriterium | Anbefaling |
|-----------|------------|
| **Use Case** | Debugging, deployment validation, load testing |
| **Latency krav** | < 1 sekund |
| **Data retention** | Ikke nødvendig (kun live view) |
| **Alerting** | Ikke påkrevd (manuell observasjon OK) |
| **Cost sensitivity** | Høy (gratis streaming) |
| **Team** | Developers, DevOps |
### Når bruke Real-Time Dashboard
| Kriterium | Anbefaling |
|-----------|------------|
| **Use Case** | Operations monitoring, SLA tracking, compliance |
| **Latency krav** | 10 sekunder - 1 minutt OK |
| **Data retention** | Påkrevd (historisk analyse) |
| **Alerting** | Kritisk (proactive incident response) |
| **Multi-source** | Ja (combine App Insights + Azure Monitor + custom events) |
| **Team** | Operations, business stakeholders |
### Vanlige feil
❌ **Bruke Live Metrics for production alerting**
Live Metrics har ingen alerting-kapabilitet. Bruk Real-Time Dashboard med Data Activator.
❌ **Åpne Live Metrics kontinuerlig i produksjon**
Live Metrics streamer data kun når panen er åpen, men slutter ikke automatisk. Lukk etter debugging for å stoppe streaming overhead på app.
❌ **Forvente persistering i Live Metrics**
Data i Live Metrics discarderes når du lukker panen. Bruk Logs eller Metrics Explorer for historiske queries.
❌ **Sette Real-Time Dashboard refresh til 10s for alle scenarier**
Høyere refresh rate = høyere compute cost. Bruk 30-60s for most production dashboards, 10s kun for critical metrics.
❌ **Bruke usecured control channel i production**
API keys for Live Metrics retired Sept 2025. Migrer til Entra ID authentication.
### Røde flagg
🚩 **Live Metrics viser ingen data**
- Sjekk firewall: Live Metrics bruker **separate endpoints** (`live.applicationinsights.azure.com`) enn vanlig telemetri
- Verifiser TLS 1.2 support (Live Metrics krever TLS 1.2+)
- Bekreftet at OpenTelemetry Distro er nyeste versjon (live metrics enabled by default)
🚩 **Real-Time Dashboard viser gammel data (> 1 min latency)**
- Application Insights har inherent ingestion latency (1-5 min). For true real-time, stream direkt til Eventhouse via Eventstream.
🚩 **Data Activator trigger for ofte (false positives)**
- Bruk anomaly detection functions i KQL (series_decompose_anomalies) for å detektere avvik fra baseline istedenfor statiske thresholds.
---
## Integrasjon med Microsoft-stakken
### Azure OpenAI + Live Metrics
```csharp
// Track Azure OpenAI calls i Live Metrics
var activity = new Activity("AzureOpenAI.ChatCompletion");
activity.Start();
try {
var response = await openAIClient.GetChatCompletionsAsync(deploymentName, options);
telemetryClient.TrackDependency(
"AzureOpenAI",
"ChatCompletion",
deploymentName,
activity.StartTimeUtc,
activity.Duration,
success: true);
} catch (Exception ex) {
telemetryClient.TrackException(ex);
throw;
} finally {
activity.Stop();
}
```
Live Metrics vil vise:
- Dependency latency i real-time
- Exception stack traces hvis Azure OpenAI API feiler
- CPU/memory impact av token processing
### Copilot Studio + Real-Time Dashboard
**Scenario:** Monitor Copilot Studio agent performance med Real-Time Dashboard.
1. **Enable Application Insights** for Copilot Studio (via Power Platform admin)
2. **Create Eventstream** som subscriber til App Insights metrics
3. **Build Real-Time Dashboard** med KQL queries:
```kusto
// Copilot Studio conversation success rate (10s buckets)
customEvents
| where name == "ConversationCompleted"
| extend success = tostring(customDimensions.Success)
| summarize
SuccessRate = countif(success == "true") * 100.0 / count()
by bin(timestamp, 10s)
| render timechart
```
4. **Setup Data Activator** til å trigger alert hvis SuccessRate < 95% i 1 minutt.
### Azure AI Foundry + Eventhouse
Azure AI Foundry Observability dashboard støtter **native integration med Application Insights**, som kan streams til Eventhouse for real-time dashboards:
1. **Enable Application Insights** for AI Foundry project
2. **Use Eventstream** til å route App Insights logs til Eventhouse
3. **Create Real-Time Dashboard** med queries for:
- Token usage per minute
- Model latency (P50, P95, P99)
- Content safety violations
- Groundedness scores
---
## Offentlig sektor (Norge)
### GDPR og datasuverenitet
**Live Metrics:**
- **Ingen persistering:** Data i Live Metrics lagres ikke, kun streames til browser. GDPR Article 6(1)(f) "legitimate interests" for debugging.
- **PII i custom filters:** Bruk IKKE custom filters med potensielt sensitive data (customer ID, email) før Entra ID authentication er aktivert (påkrevd fra Sept 2025).
**Real-Time Dashboard:**
- **Eventhouse data residency:** Velg Fabric capacity i Norway East/West for datasuverenitet.
- **Data retention policies:** Eventhouse støtter granular retention policies per table (påkrevd for Forvaltningsloven § 10 journalføring).
### Schrems II og dataoverføring
- **Live Metrics endpoints:** `live.applicationinsights.azure.com` er hosted i Azure public cloud. For Schrems II compliance, bruk Azure Government (ingen Live Metrics) eller on-prem Azure Stack HCI med Azure Arc-enabled Application Insights.
- **Real-Time Dashboard:** Fabric eventhouses kan deployes til Norway regions med Microsoft EU Data Boundary compliance.
### AI Act Article 72 - Logging
Real-Time Dashboards dekker **AI Act Article 72** krav for "automatic recording of events (logs)":
- **High-risk AI systems** (Article 6): Real-Time Dashboard + Eventhouse retention ≥ 6 måneder.
- **Audit trail:** KQL queries mot Eventhouse gir immutable audit log for AI decisions.
- **Incident response:** Data Activator alerts sikrer rask respons på AI failures (Article 9 risk management).
### Forvaltningsloven § 11 - Begrunnelse
Real-Time Dashboards kan kombineres med AI Foundry tracing til å bygge "begrunnelse for vedtak":
```kusto
// Retrieve AI decision trace for specific case
AITraces
| where timestamp between (datetime(2026-02-05T10:00) .. datetime(2026-02-05T10:05))
| where customDimensions.caseId == "CASE-12345"
| project timestamp, operation_Name, promptTokens, completionTokens, groundednessScore
| order by timestamp asc
```
Dette gir sporbarhet for AI-baserte vedtak (påkrevd av Forvaltningsloven § 11).
---
## Kostnad og lisensiering
### Application Insights Live Metrics
| Komponent | Kostnad | Merknad |
|-----------|---------|---------|
| **Live Metrics streaming** | **Gratis** | Ingen ekstra ingestion cost |
| **Underlying telemetry** | Standard App Insights pricing | $2.88/GB (pay-as-you-go) |
| **Entra ID authentication** | Inkludert i Entra ID P1/P2 | Påkrevd fra Sept 2025 |
**Optimaliseringstips:**
- Lukk Live Metrics etter debugging (stopper streaming overhead på app)
- Bruk sampling for underlying telemetry (påvirker ikke Live Metrics, men reduserer ingestion cost)
### Fabric Real-Time Intelligence
| Komponent | Kostnad (estimat) | Merknad |
|-----------|-------------------|---------|
| **Real-Time Dashboard** | Inkludert i Fabric capacity | Ingen ekstra kostnad |
| **Eventhouse storage** | ~$0.10/GB/måned | Time-series compressed |
| **Eventstream compute** | Inkludert i Fabric capacity | Avhenger av throughput |
| **Data Activator** | Separate SKU (preview pricing TBA) | Per reflex/trigger |
| **Minimum Fabric capacity** | F2 SKU (~$262/måned) | Kan skales opp/ned |
**Kostnadsmodell for eventhouse:**
- **Ingestion:** Ingen ekstra kostnad (dekket av capacity)
- **Storage:** Compressed time-series (~10:1 compression ratio for telemetry)
- **Query compute:** CU usage avhenger av dashboard refresh rate og query complexity
**Optimaliseringstips:**
- Bruk **table update policies** for pre-aggregation (reduserer query compute)
- Set dashboard refresh til **30-60s** for non-critical metrics (reduserer CU usage)
- Enable **caching** for ofte-brukte queries (cache retention 5 min - 1 time)
- Bruk **materialized views** for expensive aggregations (calculate once, query mange)
### Total Cost of Ownership (TCO) Eksempel
**Scenario:** AI-tjeneste med 1M requests/dag, 5 GB telemetry/dag.
**Option 1: Live Metrics only**
- App Insights ingestion: 5 GB/dag × 30 dager × $2.88/GB = **$432/måned**
- Live Metrics: **$0/måned**
- **Total: $432/måned**
**Option 2: Real-Time Dashboard + Eventhouse**
- App Insights ingestion: **$432/måned**
- Eventhouse storage: 150 GB × $0.10 = **$15/måned**
- Fabric F2 capacity: **$262/måned**
- **Total: $709/måned**
**Trade-off:**
- **+65% cost** for Real-Time Dashboard, MEN får persistent storage, alerting, multi-source dashboards.
- For enterprise workloads med SLA requirements: Real-Time Dashboard ROI gjennom redusert downtime.
---
## For arkitekten (Cosmo)
### Spørsmål å stille
1. **Hva er primary use case for real-time monitoring?**
- Debugging/deployment validation → Live Metrics
- Production operations med SLA → Real-Time Dashboard
- Begge → Hybrid approach
2. **Hva er akseptabel latency for monitoring?**
- < 1 sekund → Live Metrics
- 10 sekunder - 1 minutt → Real-Time Dashboard
- Varierer per metric → Kombiner begge
3. **Er data retention påkrevd (compliance, audit)?**
- Nei → Live Metrics sufficient
- Ja → Real-Time Dashboard med Eventhouse
4. **Har dere eksisterende Fabric capacity?**
- Ja → Legg til Real-Time Dashboard (no extra infra cost)
- Nei → Vurder cost/benefit mot managed App Insights only
5. **Trenger dere alerting basert på real-time metrics?**
- Ja → Data Activator (krever Real-Time Dashboard)
- Nei → Live Metrics eller standard Azure Monitor alerts (1-5 min latency)
6. **Hvor mange datakilder skal monitors?**
- Kun én app → Live Metrics
- Multiple apps/services → Real-Time Dashboard (unified view)
7. **Hvem er primary audience for dashboards?**
- Developers → Live Metrics (ad-hoc debugging)
- Operations/business → Real-Time Dashboard (shareable, no-code)
8. **Er Schrems II compliance påkrevd?**
- Ja → Fabric eventhouse i Norway regions
- Nei → Standard Application Insights OK
### Fallgruver
⚠️ **Over-reliance på Live Metrics for production**
Live Metrics er designet for debugging, ikke production monitoring. Mangler alerting og persistering.
⚠️ **Underestimere Fabric capacity krav**
Real-Time Dashboard krever minimum F2 SKU. Start med F2, skaler opp hvis CU throttling.
⚠️ **Ignorere API key deprecation (Sept 2025)**
Migrer til Entra ID authentication for Live Metrics control channel NÅ, ikke vent til deadline.
⚠️ **Sette for aggressive refresh rates**
10s refresh på alle dashboards gir høy CU cost. Bruk 30-60s for de fleste metrics.
⚠️ **Blande real-time streaming med batch ETL**
Real-Time Dashboard er IKKE erstatning for data warehouse. Bruk for operational monitoring, ikke business analytics.
### Anbefalinger per modenhetsnivå
**Nivå 1 - Proof of Concept:**
- Start med **Live Metrics** (gratis, zero-config)
- Enable for ASP.NET Core/Java/Python apps (enabled by default med OpenTelemetry)
- Bruk under deployment validation
**Nivå 2 - Pilot (produksjon med begrenset scope):**
- Introduser **Real-Time Dashboard** for critical services
- Deploy Eventhouse i Norway region (GDPR compliance)
- Setup Data Activator for 2-3 critical alerts (error rate, latency)
- Start med F2 capacity, monitor CU usage
**Nivå 3 - Production (enterprise-scale):**
- Hybrid approach: Live Metrics for developers + Real-Time Dashboard for ops
- Multi-source dashboards (App Insights + Azure Monitor + custom events)
- Materialized views for expensive aggregations
- Entra ID authentication for Live Metrics control channel
- KQL alert queries med anomaly detection (series_decompose_anomalies)
**Nivå 4 - Optimalisert:**
- Custom Eventstream pipelines med pre-aggregation
- Dedicated Fabric capacity (F8+) for high-throughput
- Automated dashboard generation med Copilot
- Integration med Power BI for business stakeholder reporting
- Cross-region replication av Eventhouse (disaster recovery)
---
## Kilder og verifisering
### Microsoft Learn (Verified via MCP)
1. **Live Metrics: Monitor and diagnose with 1-second latency**
https://learn.microsoft.com/en-us/azure/azure-monitor/app/live-stream
**Confidence:** Verified (Feb 2026) - Authoritative doc for Live Metrics
2. **What is Real-Time Dashboard?**
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/real-time-dashboards-overview
**Confidence:** Verified (Feb 2026) - Fabric Real-Time Intelligence GA features
3. **Configure Azure Monitor OpenTelemetry**
https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-configuration#live-metrics
**Confidence:** Verified (Feb 2026) - OpenTelemetry Live Metrics config
4. **Troubleshoot Live Metrics issues**
https://learn.microsoft.com/en-us/troubleshoot/azure/azure-monitor/app-insights/troubleshoot-live-metrics
**Confidence:** Verified (Feb 2026) - Firewall, TLS requirements
5. **Monitor .NET and Node.js applications with Application Insights (Classic API)**
https://learn.microsoft.com/en-us/azure/azure-monitor/app/classic-api#collecting-telemetry-data
**Confidence:** Verified (Feb 2026) - Manual Live Metrics setup (legacy)
6. **Build real-time monitoring and observable systems for media**
https://learn.microsoft.com/en-us/azure/architecture/example-scenario/monitoring/monitoring-observable-systems-media
**Confidence:** Verified (Feb 2026) - Real-time architecture patterns
7. **Observability in generative AI**
https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/observability
**Confidence:** Verified (Feb 2026) - AI Foundry monitoring integration
8. **Implement advanced monitoring for Azure OpenAI through a gateway**
https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-monitoring#near-real-time-monitoring
**Confidence:** Verified (Feb 2026) - Near real-time vs batch monitoring trade-offs
### Confidence per seksjon
| Seksjon | Confidence | Kilde |
|---------|-----------|-------|
| Application Insights Live Metrics | Verified | MCP fetch: live-stream doc |
| Fabric Real-Time Dashboard | Verified | MCP fetch: real-time-dashboards-overview |
| Arkitekturmønstre | Baseline | Synthesized fra multiple MCP sources |
| Code samples | Verified | MCP code search: C# Live Metrics setup |
| Kostnad og lisensiering | Baseline | Pricing calculated fra public Azure pricing (Feb 2026) |
| Offentlig sektor compliance | Baseline | Applied GDPR/AI Act principles til verified features |
**MCP calls:** 6 (3 × search, 2 × fetch, 1 × code search)
**Unique sources:** 8 Microsoft Learn URLs
**Last verified:** 2026-02-05