ms-ai-architect/scripts/kb-eval/data
Kjell Tore Guttormsen 79e34312ea feat(ms-ai-architect): S1 judge bake-off RESULT — GATE FAIL (recall 68.4% < 80%) [skip-docs]
Blind fan-out: 15 batcher Opus 4.8 xhigh, 255 P-påstander dekket (0 missing/dupes/extra),
hver judget mot live MS Learn via microsoft_docs_fetch (judgen så aldri gull-verdict).

Resultat mot forhåndsregistrert gate (recall >=0.80, presisjon >=0.70):
- judge: presisjon 81.3% (PASS), recall 68.4% (FAIL, Wilson [52.5-80.9%]), F1 0.743, 26/38 fanget
- staleness baseline: recall 0/38 (judge slår den desisivt)
- recall-drag konsentrert: sku 37.5%, taxonomy 66.7%; version/status/tpm allerede 80-86%
- 10/12 bommer var grounded-men-feil (brittle dekomponering, isolert til sku/taxonomy)

Per pre-registrert gate: STOPP, bygg IKKE S3. Fork (operatør) registrert i plan-doc:
(a) honorer FAIL, (b) scoped judge, (c) målrettet iterasjon + re-kjør.
2026-06-26 20:45:09 +02:00
..
fase0-returns feat(ms-ai-architect): Fase 0 steg 2 — verifiserings-fan-out (373 påstander) + gull-sett m/ deterministisk lastmod-join + TDD [skip-docs] 2026-06-26 15:16:10 +02:00
base-rate-report.json feat(ms-ai-architect): Fase 0 steg 3 — base-rate-rapport + GATE: BYGG Fase 3 (scoped/hybrid) [skip-docs] 2026-06-26 17:15:54 +02:00
base-rate-report.md feat(ms-ai-architect): Fase 0 steg 3 — base-rate-rapport + GATE: BYGG Fase 3 (scoped/hybrid) [skip-docs] 2026-06-26 17:15:54 +02:00
eval-baseline.json feat(ms-ai-architect): Sesjon 15 — B2 K10 søsken-scope-ikke-overlapp 2026-06-20 11:59:59 +02:00
fase0-sample-frame.json feat(ms-ai-architect): Fase 0 steg 1 — deterministisk sample-frame (45 volatil + 10 kontroll) + TDD [skip-docs] 2026-06-26 14:18:53 +02:00
gold-correctness-set.json feat(ms-ai-architect): Fase 0 steg 2 — verifiserings-fan-out (373 påstander) + gull-sett m/ deterministisk lastmod-join + TDD [skip-docs] 2026-06-26 15:16:10 +02:00
judge-bakeoff-claims.json feat(ms-ai-architect): S1 judge bake-off harness + forhåndsregistrert gate (TDD) [skip-docs] 2026-06-26 20:10:58 +02:00
judge-bakeoff-report.json feat(ms-ai-architect): S1 judge bake-off RESULT — GATE FAIL (recall 68.4% < 80%) [skip-docs] 2026-06-26 20:45:09 +02:00
judge-bakeoff-report.md feat(ms-ai-architect): S1 judge bake-off RESULT — GATE FAIL (recall 68.4% < 80%) [skip-docs] 2026-06-26 20:45:09 +02:00
judge-bakeoff-results.json feat(ms-ai-architect): S1 judge bake-off RESULT — GATE FAIL (recall 68.4% < 80%) [skip-docs] 2026-06-26 20:45:09 +02:00
judge-results.json feat(ms-ai-architect): Spor D steg A — eng+infra K10-fix → korpus-invariant ≥90 oppfylt 2026-06-23 17:56:43 +02:00
k1-trigger-prompts.json fix(ms-ai-architect): Sesjon 11 — K1 trigger-presisjon, blindet judge PASS alle 5 2026-06-20 10:00:47 +02:00