open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 4 Activity

feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command

Browse source 
Add /ultraresearch-local for structured research combining local codebase
analysis with external knowledge via parallel agent swarms. Produces research
briefs with triangulation, confidence ratings, and source quality assessment.

New command: /ultraresearch-local with modes --quick, --local, --external, --fg.
New agents: research-orchestrator (opus), docs-researcher, community-researcher,
security-researcher, contrarian-researcher, gemini-bridge (all sonnet).
New template: research-brief-template.md.

Integration: --research flag in /ultraplan-local accepts pre-built research
briefs (up to 3), enriches the interview and exploration phases. Planning
orchestrator cross-references brief findings during synthesis.

Design principle: Context Engineering — right information to right agent at
right time. Research briefs are structured artifacts in the pipeline:
ultraresearch → brief → ultraplan --research → plan → ultraexecute.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-08 08:58:35 +02:00
commit baa2d0220b
488 changed files with 213221 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

470
skills/ms-ai-infrastructure/references/hybrid-edge/network-constrained-ai-deployment.md Normal file
View file

@ -0,0 +1,470 @@
# Network-Constrained AI Deployment
**Last updated:** 2026-02
**Status:** GA
**Category:** Hybrid Cloud & Edge AI
---
## Introduksjon
Nettverksbegrensede miljoer — med lav bandwidth, hoey latens, eller intermitterende tilkobling — krever spesialtilpassede AI-deployments. Standard sky-baserte AI-arkitekturer som sender data frem og tilbake til cloud endpoints feiler i slike miljoer, enten pa grunn av uakseptabel latens eller fordi tilkoblingen simpelthen ikke er palitelig nok.
For norsk offentlig sektor er dette relevant i mange scenarier: rurale omrader med begrenset mobildekning, maritime miljoer med satellittkommunikasjon, tunneler og underjordiske anlegg, feltoperasjoner i krisesituasjoner, og industrielle miljoer med nettverksisolasjon av sikkerhetsgrunner. AI-losninger for slike miljoer ma optimaliseres for minimal nettverksbruk.
Microsoft tilbyr flere teknologier for nettverksbegrensede deployments: modellkvantisering og -komprimering med Olive/ONNX Runtime for mindre modeller, Azure IoT Edge med utvidet offline-stoette for edge-prosessering, delta-synkronisering for effektiv dataoverfoering, og bandwidth-bevisst batching for a maksimere utnyttelsen av tilgjengelig tilkobling.
---
## Kjernekomponenter
| Komponent | Formal | Teknologi |
|-----------|--------|-----------|
| ONNX Runtime | Optimalisert lokal inferens | Cross-platform |
| Olive | Modellkomprimering og kvantisering | Python |
| Azure IoT Edge | Edge-prosessering med buffring | Container runtime |
| Delta Sync | Inkrementell datasynkronisering | Protokoll |
| MQTT | Lettvekts meldingsprotokoll | IoT-standard |
| gRPC | Effektiv binart API-protokoll | Google RPC |
| Protocol Buffers | Kompakt serialisering | Google |
---
## Model Size Reduction
### Kvantiseringsstrategier for nettverksbegrensede miljoer
| Teknikk | Stoerrelses-reduksjon | Kvalitetstap | Nedlastningstid (1 Mbps) |
|---------|---------------------|-------------|--------------------------|
| FP32 (original) | Baseline (7 GB for 3.8B) | Ingen | 15+ timer |
| FP16 | 2x (3.5 GB) | Minimalt | 7+ timer |
| INT8 | 4x (1.75 GB) | < 1% | 3.5 timer |
| INT4 | 8x (875 MB) | 1-3% | 1.7 timer |
| INT4 + Pruning | 10-12x (600 MB) | 2-5% | 1.2 timer |
| Distillation | 20-50x (140-350 MB) | 5-15% | 15-40 min |
### Olive-basert komprimering
```python
# Olive pipeline for maksimal modellkomprimering
import json
olive_config = {
"input_model": {
"type": "HfModel",
"model_path": "microsoft/Phi-3-mini-4k-instruct"
},
"passes": {
# Steg 1: Konverter til ONNX
"conversion": {
"type": "OnnxConversion",
"target_opset": 17
},
# Steg 2: Grafoptimalisering
"optimization": {
"type": "OrtTransformersOptimization",
"model_type": "gpt2",
"opt_level": 2,
"only_onnxruntime": True
},
# Steg 3: Kvantisering til INT4
"quantization": {
"type": "OnnxMatMul4Quantizer",
"block_size": 32,
"is_symmetric": True,
"accuracy_level": 4
},
# Steg 4: Strukturell pruning (fjern unodvendige vekter)
"pruning": {
"type": "SlicGPT",
"sparsity": 0.25, # 25% reduksjon
"calibration_data_config": {
"name": "c4",
"split": "validation",
"num_samples": 128
}
}
},
"engine": {
"evaluator": {
"metrics": [
{
"name": "model_size",
"type": "custom_metric",
"priority": 1
},
{
"name": "latency",
"type": "latency",
"priority": 2
}
]
}
}
}
# Kjor Olive-pipeline
# olive run --config olive_compress.json
```
### Kunnskapsdesstillasjon for ultra-sma modeller
```python
# Destiller fra Phi-3 Medium (14B) til en 1B custom-modell
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
class ModelDistillation:
def __init__(self, teacher_model: str, student_config: dict):
self.teacher = AutoModelForCausalLM.from_pretrained(teacher_model)
self.student = self._create_student(student_config)
self.tokenizer = AutoTokenizer.from_pretrained(teacher_model)
def distill(self, dataset, epochs: int = 5, temperature: float = 2.0):
"""Destiller laerer-modell til elev-modell"""
optimizer = torch.optim.AdamW(self.student.parameters(), lr=1e-4)
for epoch in range(epochs):
for batch in dataset:
inputs = self.tokenizer(batch["text"], return_tensors="pt",
padding=True, truncation=True)
# Laerer-prediksjoner (soft targets)
with torch.no_grad():
teacher_logits = self.teacher(**inputs).logits
# Elev-prediksjoner
student_logits = self.student(**inputs).logits
# Destillasjonsloss: KL-divergens med temperatur
loss = self._distillation_loss(
student_logits, teacher_logits, temperature
)
loss.backward()
optimizer.step()
optimizer.zero_grad()
def _distillation_loss(self, student_logits, teacher_logits, temperature):
"""KL-divergens mellom student og teacher"""
import torch.nn.functional as F
soft_student = F.log_softmax(student_logits / temperature, dim=-1)
soft_teacher = F.softmax(teacher_logits / temperature, dim=-1)
return F.kl_div(soft_student, soft_teacher, reduction='batchmean') * (temperature ** 2)
```
---
## Partial Model Loading
### Modell-segmentering for inkrementell nedlasting
```python
# Delvis modellnedlasting for nettverksbegrensede miljoer
import os
import hashlib
from typing import Optional
class IncrementalModelLoader:
def __init__(self, model_dir: str, remote_url: str):
self.model_dir = model_dir
self.remote_url = remote_url
self.manifest_path = os.path.join(model_dir, "manifest.json")
def check_and_download(self, bandwidth_kbps: float) -> dict:
"""Sjekk modellstatus og last ned manglende deler"""
manifest = self._get_remote_manifest()
local_state = self._get_local_state()
missing_segments = []
total_download_bytes = 0
for segment in manifest["segments"]:
local_hash = local_state.get(segment["name"])
if local_hash != segment["hash"]:
missing_segments.append(segment)
total_download_bytes += segment["size"]
# Estimer nedlastningstid
download_time_seconds = (total_download_bytes * 8) / (bandwidth_kbps * 1000)
return {
"model_version": manifest["version"],
"total_segments": len(manifest["segments"]),
"missing_segments": len(missing_segments),
"download_size_mb": total_download_bytes / 1024 / 1024,
"estimated_time_minutes": download_time_seconds / 60,
"can_use_partial": manifest.get("supports_partial_inference", False),
"minimum_segments_for_inference": manifest.get("min_segments", 1)
}
def download_prioritized(self, bandwidth_kbps: float,
time_budget_minutes: float) -> str:
"""Last ned modellsegmenter prioritert innenfor tidsbudsjett"""
check = self.check_and_download(bandwidth_kbps)
if check["estimated_time_minutes"] <= time_budget_minutes:
# Full nedlasting mulig
return self._download_all_segments()
else:
# Prioritert delvis nedlasting
# Last ned kritiske segmenter foerst (embedding, attention heads)
return self._download_critical_first(time_budget_minutes, bandwidth_kbps)
def _download_critical_first(self, time_budget: float, bw: float) -> str:
"""Last ned de viktigste modelldelene foerst"""
priority_order = [
"embeddings", # Nodvendig for all inferens
"attention_layers", # Kjernekapabilitet
"ffn_layers", # Detaljert prosessering
"output_head" # Siste lag
]
# Download i prioritert rekkefoolge innenfor tidsbudsjett
downloaded = []
remaining_seconds = time_budget * 60
for priority in priority_order:
segment_size = self._get_segment_size(priority)
download_time = (segment_size * 8) / (bw * 1000)
if download_time <= remaining_seconds:
self._download_segment(priority)
downloaded.append(priority)
remaining_seconds -= download_time
else:
break
return f"Lastet ned {len(downloaded)}/{len(priority_order)} segmenter"
```
---
## Bandwidth-Aware Batching
### Adaptiv batchstoerrelser basert pa tilgjengelig bandwidth
```python
# Bandwidth-bevisst batch-synkronisering
import asyncio
import time
from dataclasses import dataclass
@dataclass
class BandwidthProfile:
estimated_kbps: float
latency_ms: float
reliability: float # 0-1, andel vellykkede overforinger
class AdaptiveBatchSync:
def __init__(self):
self.bandwidth_history: list[BandwidthProfile] = []
self.compression_enabled = True
async def measure_bandwidth(self) -> BandwidthProfile:
"""Mal tilgjengelig bandwidth med minimal data"""
test_data = b"x" * 1024 # 1 KB testpakke
start = time.time()
try:
# Send testpakke og mal round-trip
success = await self._send_probe(test_data)
elapsed = time.time() - start
profile = BandwidthProfile(
estimated_kbps=(len(test_data) * 8) / (elapsed * 1000),
latency_ms=elapsed * 1000,
reliability=1.0 if success else 0.0
)
except Exception:
profile = BandwidthProfile(
estimated_kbps=0, latency_ms=float('inf'), reliability=0.0
)
self.bandwidth_history.append(profile)
return profile
def calculate_optimal_batch(self, pending_items: int,
avg_item_size_kb: float) -> dict:
"""Beregn optimal batchstoorrelse basert pa nettverksforhold"""
if not self.bandwidth_history:
return {"batch_size": 1, "reason": "Ingen maalinger"}
recent = self.bandwidth_history[-5:] # Siste 5 maalinger
avg_bw = sum(p.estimated_kbps for p in recent) / len(recent)
avg_reliability = sum(p.reliability for p in recent) / len(recent)
if avg_reliability < 0.3:
# Svart upaalitelig — minimale batche
return {"batch_size": 1, "compress": True, "priority_only": True}
if avg_bw < 10: # < 10 kbps
# Ekstremt lav bandwidth
batch_size = min(5, pending_items)
return {
"batch_size": batch_size,
"compress": True,
"format": "protobuf",
"priority_only": True,
"estimated_time_s": (batch_size * avg_item_size_kb) / avg_bw
}
elif avg_bw < 100: # 10-100 kbps
# Lav bandwidth
batch_size = min(20, pending_items)
return {
"batch_size": batch_size,
"compress": True,
"format": "protobuf",
"priority_only": False
}
elif avg_bw < 1000: # 100 kbps - 1 Mbps
# Medium bandwidth
batch_size = min(100, pending_items)
return {
"batch_size": batch_size,
"compress": True,
"format": "json_gzip"
}
else: # > 1 Mbps
# God bandwidth
return {
"batch_size": min(500, pending_items),
"compress": False,
"format": "json"
}
```
---
## Latency Compensation Patterns
### Strategier for latenskompensering
| Monster | Beskrivelse | Implementering |
|---------|-------------|----------------|
| Optimistisk UI | Vis resultat umiddelbart, korriger senere | Lokal prediksjon + sky-validering |
| Prefetching | Forhaandslast sannsynlige data | Prediktiv caching |
| Stale-while-revalidate | Vis cachet data mens ny hentes | Cache-lag med TTL |
| Lokal buffer | Buffer resultater lokalt | SQLite + event queue |
| Priority queue | Prioriter kritiske data | Vektet synk-koe |
| Komprimering | Reduser datamengde | gzip, protobuf, CBOR |
### Implementering av latenskompensering
```python
# Latenskompenserende AI-proxy
import asyncio
import gzip
import json
from collections import OrderedDict
class LatencyCompensatingProxy:
def __init__(self, local_model, cache_size: int = 1000):
self.local_model = local_model
self.cache = OrderedDict()
self.cache_size = cache_size
self.pending_validations = asyncio.Queue()
async def predict(self, input_data: dict) -> dict:
"""Prediksjon med latenskompensering"""
cache_key = self._hash_input(input_data)
# Sjekk cache forst
if cache_key in self.cache:
cached = self.cache[cache_key]
self.cache.move_to_end(cache_key)
return {**cached, "source": "cache"}
# Lokal prediksjon (umiddelbar)
local_result = self.local_model.predict(input_data)
# Cache resultatet
self._cache_result(cache_key, local_result)
# Koe for sky-validering i bakgrunnen
await self.pending_validations.put({
"cache_key": cache_key,
"input": input_data,
"local_result": local_result
})
return {**local_result, "source": "local", "validated": False}
async def background_validator(self):
"""Bakgrunnsvalidering mot sky-modell"""
while True:
item = await self.pending_validations.get()
try:
cloud_result = await self._cloud_predict(item["input"])
# Oppdater cache med validert resultat
self._cache_result(item["cache_key"], {
**cloud_result,
"validated": True
})
# Varsle om avvik
if self._significant_difference(
item["local_result"], cloud_result
):
await self._notify_correction(item, cloud_result)
except Exception:
pass # Sky utilgjengelig — behold lokal prediksjon
def _significant_difference(self, local: dict, cloud: dict) -> bool:
"""Sjekk om sky-resultat avviker vesentlig fra lokalt"""
if local.get("label") != cloud.get("label"):
return True
if abs(local.get("confidence", 0) - cloud.get("confidence", 0)) > 0.2:
return True
return False
```
---
## Norsk offentlig sektor
### Nettverksbegrensede scenarier i Norge
| Scenario | Typisk bandwidth | Latens | Tilgjengelighet |
|----------|-----------------|--------|-----------------|
| Rural mobildekning | 1-10 Mbps | 50-200 ms | 70-90% |
| Maritim (kyst) | 0.5-5 Mbps | 200-600 ms | 60-80% |
| Tunnel/underjordisk | 0 Mbps (isolert) | N/A | 0% |
| Svalbart | 1-50 Mbps | 500+ ms | 80% |
| Beredskap (krise) | 0.1-1 Mbps | Variable | 30-70% |
| Felt (skog/fjell) | 0-5 Mbps | 100-500 ms | 40-80% |
### Anbefalinger
- Dimensjoner alltid for verste-tilfelle tilkobling
- Bruk INT4-kvantiserte modeller som standard for edge-deployment
- Implementer bandwidth-maling for a tilpasse sync-strategi dynamisk
- Protobuf/CBOR for serialisering i stedet for JSON i lav-bandwidth-scenarier
- Prioriter anomalier og kritiske resultater i sync-koeen
---
## Beslutningsrammeverk
| Scenario | Anbefaling | Begrunnelse |
|----------|------------|-------------|
| < 100 kbps | Full lokal inferens + minimal sync | Ikke nok bandwidth for sky-AI |
| 100 kbps - 1 Mbps | Lokal inferens + delta-sync | Synkroniser resultater, ikke radata |
| 1-10 Mbps | Hybrid med progressiv enhancement | Sky-validering av lokale resultater |
| > 10 Mbps | Standard sky-AI med lokal fallback | Normal drift med offline-buffer |
| Intermitterende | Event sourcing + prioritert batch-sync | Palitelig leveranse over tid |
| Satelitt (hoey latens) | Full lokal med periodisk bulk-sync | Latens for hoey for interaktiv sky-AI |
---
## For Cosmo
- **Modellstoerrelse er den viktigste faktoren for nettverksbegrensede deployments** — bruk INT4-kvantisering som standard, og vurder destillasjon for ultralette modeller under 500 MB
- **Bandwidth-bevisst batching er pabudt** — maal tilgjengelig bandwidth kontinuerlig og tilpass batchstoerrelser og kompresjonsformat dynamisk
- **Protobuf/CBOR sparer 60-80% bandwidth** sammenlignet med JSON — bruk binaere serialiseringsformater for all edge-til-sky-kommunikasjon i lav-bandwidth-miljoer
- **Inkrementell modellnedlasting er kritisk** for modelloppdatering over lav bandwidth — last ned kun endrede lag/segmenter, og stoeett delvis modellbruk under nedlasting
- **For norsk offentlig sektor: Design for 100 kbps som worst case** — mange felt-scenarier i rural Norge har begrenset 4G-dekning, og maritime/beredskapsscenarier kan ha enda lavere bandwidth