open/ms-ai-architect
1
0
Fork 0
Code Issues Pull requests Releases Activity

feat(ms-ai-architect): G5 gull-friskhets-mikropass LUKKET — 2 gull-feil rettet (genaiops#2, model-selection#8), v3 REHABILITERT — slår v2 på fersk gull (P 89.7/R 92.1 vs 86.8/86.8); adopsjonsbeslutning reversert [skip-docs]

Browse source
This commit is contained in:
Kjell Tore Guttormsen 2026-06-30 11:21:45 +02:00
commit bc3fc85b20
6 changed files with 598 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

14
scripts/kb-eval/data/gold-correctness-set.json
View file

@ -5,7 +5,11 @@
"sample_frame": "fase0-sample-frame.json",
"raw_returns": "fase0-returns/batch-*.json",
"note_evidence_quote": "Per-claim evidence_quote omitted here to bound size; full verbatim quotes live in the session transcript. notes + evidence_url capture each verdict basis.",
"claim_count": 373
"claim_count": 373,
"reconciliation_log": [
"2026-06-30 Spor 2b: 12 judge-vs-gold-uenigheter adjudert mot live; 4 gull-feil rettet + 1 note-fiks. Logg: docs/ref-kb-gold-reconciliation-2026-06.md.",
"2026-06-30 G5 friskhets-mikropass: 5 omstridte claims (gold=outdated, v3=grounded) re-adjudert mot live MS Learn. 2 stale gull rettet outdated->correct (genaiops-llm-specific-practices.md#2 '1600+' live 1900 tett nedre grense; model-selection-price-performance.md#8 Model Router GA nov 2025). 3 opprettholdt som outdated (judge-feil bekreftet: multi-model-strategy-costs.md#2, token-usage-tracking-attribution.md#3, ai-foundry-disaster-recovery-planning.md#9). Logg: docs/ref-kb-correctness-program-2026-06.md §8 G5."
]
},
"claims": [
{
@ -1705,11 +1709,11 @@
"stratum": "volatile",
"claim": "Model Catalog 1600+ foundation models",
"claim_type": "sku",
"verdict": "outdated",
"verdict": "correct",
"evidence_url": "https://learn.microsoft.com/azure/foundry/concepts/foundry-models-overview",
"lastmod_changed": false,
"file_last_updated": "2026-06-19",
"notes": "Kilde over 1900. 1600+ utdatert lavere tall."
"notes": "RECONCILED 2026-06-30 (G5 friskhets-mikropass): outdated->correct. Live (foundry-models-overview): 'There are over 1,900 models'. '1600+' er en SANN nedre grense (1900 >= 1600), understatement kun ~1,19x - innenfor den ratifiserte nedre-grense-policyen (grov = >~2x og beslutningsendrende). Gull var her for strengt: motsa egen policy. v3 grounded (R1) var korrekt. Fil kan friskes til '1900+' (Spor 0, valgfritt) men er IKKE feil som den star."
},
{
"id": "ms-ai-engineering/mlops-genaiops/genaiops-llm-specific-practices.md#3",
@ -4008,11 +4012,11 @@
"stratum": "volatile",
"claim": "Model Router er GA-funksjonalitet i Microsoft Foundry",
"claim_type": "status",
"verdict": "outdated",
"verdict": "correct",
"evidence_url": "https://learn.microsoft.com/azure/foundry/foundry-models/how-to/model-choice-guide",
"lastmod_changed": false,
"file_last_updated": "2026-06-01",
"notes": "Kilde lenker 'model router for Foundry (preview)'. Fortsatt preview, ikke GA."
"notes": "RECONCILED 2026-06-30 (G5 friskhets-mikropass): outdated->correct. Model Router gikk GA nov 2025: whats-new-model-router har 'Model router GA version', kanonisk concept-side (foundry/openai/concepts/model-router) titulerer uten '(preview)' og lister v2025-11-18 som 'Active (latest)'. Gull var stale - basert pa en utdatert '(preview)'-lenkelabel pa siteringssiden (model-choice-guide); resterende '(preview)' gjelder enkelt-modeller (grok/DeepSeek/Claude-fotnoter), ikke ruter-tjenesten. v3 grounded (R7, fulgte til kanonisk side) var korrekt."
},
{
"id": "ms-ai-security/cost-optimization/model-selection-price-performance.md#9",