open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity
ms-ai-architect/scripts/kb-eval/data/r11-footer-class-2026-08-11.json
Kjell Tore Guttormsen 41f769a166 docs(ms-ai-architect): footer-klassen handklassifisert — 58 medlemmer, en fjerde unnslippsform, og «alle avvik samme vei» falt 
§9.16. Leveransen er en maaling og en scope-anbefaling for aapent spoersmaal
#17; ingen korpusfil er roert.

«>=48» var riktig som gulv. Handklassifisert: 58 ekte medlemmer i 58 filer.
Bøtter: 30 konsistent / 11 inkonsistent / 1 tvetydig / 16 ikke sjekkbar.
Hvert anker verifisert ordrett og unikt (0 avvik).

FJERDE UNNSLIPPSFORM: tellingen baaret av handlingsordet («3 soek»,
«4 search queries», «2 deep fetch»), ikke av «kall». Mitt eget nett var blindt
for den — domain-specific-prompt-optimization.md:589 ble kun reddet av
nabolinja, og model-versioning-registry-management.md var usynlig i sin helhet
for baade MCP- og verktoeynavn-nettet.

COMPLETENESS: i stedet for flere ordgjetninger ble hele etikettvokabularet i
proveniensblokkene enumerert — 68 distinkte etiketter / 141 linjer, alle
plasserbare i kall-telling, kilde-telling eller annet. Ingen femte form.
Den kjente residualen (alle nett krever et SIFFER) ble testet, ikke antatt:
21 tallord-treff, alle broedtekst, null medlemmer.

RETNINGSARGUMENTET RE-MAALT: 10 av 11 inkonsistente gaar oppgitt < sum, 1 gaar
motsatt (chain-of-thought-prompting.md, oppgitt 4 over enumerert 3). «Alle
avvik gaar samme vei» er falsifisert som absolutt og maa slutte aa siteres
slik; tendensen (91 %) overlever. «Null motsatt» var et artefakt av at
populasjonen var maalt til under halv stoerrelse.

FORMEN DEKKER IKKE KLASSEN: 46 av 58 er linjeform, 12 er blokkform der «slett
linja» er udefinert — og én blokk blander naboklassen (Unique sources) inn i
samme punktliste som kall-linjene.

ANBEFALING #17: én klasse-entry for de 42 uten aapen beslutning, 16 egne
entries for dem som baerer én. Beslutning #14 («én entry per fil») ble tatt da
klassen ble antatt aa vaere 14 filer; 42 nesten-identiske entries er nettopp
mekanismen §9.15 viste at lot en feilmaaling gjemme seg bak sine egne kopier.

Datasett: scripts/kb-eval/data/r11-footer-class-2026-08-11.json
Suite 1047/1047.
2026-08-11 21:52:19 +02:00

575 lines
No EOL
17 KiB
JSON
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"_meta": {
"measured": "2026-08-11",
"population": "skills/**/references/**/*.md (389 files)",
"class": "a stated count of MCP tool calls made while researching THIS document",
"buckets": {
"konsistent": "total stated + breakdown stated + they agree under the only reasonable reading",
"inkonsistent": "total stated + breakdown stated + they disagree under every reasonable reading",
"tvetydig": "total stated + breakdown stated, verdict flips between two defensible readings",
"ikke_sjekkbar": "no internal cross-check exists (bare total, breakdown without total, or a component with no number)"
},
"flags": "orthogonal to bucket: provenance_mix, label_value_mismatch, prose_repeat, direction_stated_gt_enumerated"
},
"members": [
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/adaptive-cards-copilot-responses.md",
"l": 517,
"a": "**MCP calls:** 3 docs_search, 2 docs_fetch, 1 code_sample_search",
"b": "ikke_sjekkbar",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/copilot-api-rate-limiting-resilience.md",
"l": 496,
"a": "**MCP Calls:** 6 (3 searches, 2 fetches, 1 code sample search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/declarative-agents-grounding-strategies.md",
"l": 461,
"a": "**MCP-kall:** 7 (3 search, 3 fetch, 1 code sample search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/enterprise-governance-copilot-deployment.md",
"l": 920,
"a": "- 3 microsoft_docs_search calls",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
919,
922
]
},
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/microsoft-graph-api-copilot-integration.md",
"l": 544,
"a": "**MCP calls:** 7 (3x docs_search, 2x docs_fetch, 1x code_sample_search, 1x ToolSearch)",
"b": "tvetydig",
"flags": [
"provenance_mix"
],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/sharepoint-copilot-agents.md",
"l": 356,
"a": "**MCP-calls:** 5 (3 search + 2 fetch).",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/copilot-extensibility/teams-copilot-message-extensions.md",
"l": 469,
"a": "**MCP-kall utført:** 6 (3 search, 2 fetch, 1 code sample search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/prompt-engineering/chain-of-thought-prompting.md",
"l": 500,
"a": "**Totalt:** 4 MCP-kall, 3 unike Microsoft Learn-kilder.",
"b": "inkonsistent",
"flags": [
"direction_stated_gt_enumerated"
],
"form": "block",
"span": [
494,
500
],
"stated": 4,
"enumerated": 3,
"delta": 1
},
{
"f": "skills/ms-ai-advisor/references/prompt-engineering/domain-specific-prompt-optimization.md",
"l": 589,
"a": "- **MCP-søk** — 3 søk mot microsoft-learn (2026-02-04)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
587,
591
]
},
{
"f": "skills/ms-ai-advisor/references/prompt-engineering/multi-turn-conversation-management.md",
"l": 683,
"a": "**MCP calls:** 5 (search + fetch)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-advisor/references/prompt-engineering/prompt-testing-and-evaluation.md",
"l": 1080,
"a": "**MCP Calls:** 3 (microsoft_docs_search × 2, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 3,
"enumerated": 5,
"delta": -2
},
{
"f": "skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md",
"l": 563,
"a": "- **MCP calls:** 3 (microsoft_docs_search) + 2 (microsoft_docs_fetch) + 1 (microsoft_code_sample_search) = 6",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/agent-orchestration/agent-memory-and-context-management.md",
"l": 535,
"a": "**MCP calls**: 6 (3x microsoft_docs_search, 2x microsoft_docs_fetch, 1x microsoft_code_sample_search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/agent-orchestration/foundry-workflows-visual-orchestration.md",
"l": 651,
"a": "**MCP calls**: 4 (2x docs_search, 2x docs_fetch)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/agent-orchestration/multi-agent-orchestration-patterns.md",
"l": 720,
"a": "**Total MCP calls:** 6 (3 microsoft_docs_search + 2 microsoft_docs_fetch + 1 microsoft_code_sample_search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-api-best-practices.md",
"l": 762,
"a": "**MCP call summary:** 7 microsoft_docs_search + 4 microsoft_docs_fetch + 1 microsoft_code_sample_search = 12 total MCP calls",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-cost-optimization.md",
"l": 396,
"a": "**Total MCP calls:** 6",
"b": "ikke_sjekkbar",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-governance-compliance.md",
"l": 753,
"a": "**Total antall MCP-kall:** 8 (4 docs_search + 4 docs_fetch)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-networking-security.md",
"l": 620,
"a": "**MCP calls:** 7 (microsoft_docs_search, microsoft_docs_fetch, microsoft_code_sample_search)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/azure-ai-vision-image-analysis.md",
"l": 392,
"a": "**MCP-kall totalt:** 4 (3 docs_search + 1 code_sample_search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/document-intelligence-prebuilt-models.md",
"l": 565,
"a": "**Totalt MCP-kall:** 5 (3× search, 2× fetch, 1× code samples)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 5,
"enumerated": 6,
"delta": -1
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/speech-services-text-to-speech.md",
"l": 538,
"a": "**Totalt antall MCP-kall:** 7 (4 × docs_search, 3 × docs_fetch, 1 × code_sample_search)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 7,
"enumerated": 8,
"delta": -1
},
{
"f": "skills/ms-ai-engineering/references/azure-ai-services/translator-document-translation.md",
"l": 402,
"a": "**Total MCP calls:** 4 (docs_search) + 3 (docs_fetch) = **7**",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/data-drift-monitoring-detection.md",
"l": 378,
"a": "**MCP Calls:** 5 (3 × microsoft_docs_search, 1 × microsoft_docs_fetch, 1 × microsoft_code_sample_search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/genaiops-llm-specific-practices.md",
"l": 383,
"a": "**Totalt:** 18 kilder, 8 MCP-kall.",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/inferencing-optimization-caching.md",
"l": 1005,
"a": "**Total MCP-kall:** 7 (docs search) + 3 (docs fetch) + 2 (code samples) = **12**",
"b": "konsistent",
"flags": [
"prose_repeat"
],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md",
"l": 933,
"a": "- **microsoft_docs_search calls:** 4",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
932,
935
]
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
"l": 744,
"a": "**MCP Calls:** 8 (microsoft-learn docs search + fetch, code samples)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
"l": 738,
"a": "**Totalt antall MCP-kall:** 6 (3x search, 2x fetch, 1x code samples)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md",
"l": 1067,
"a": "**MCP-kall utført:** 8 (microsoft_docs_search × 5, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
"l": 580,
"a": "- **Total searches:** 3 (Azure ML registry, AI Foundry, MLOps lifecycle)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
579,
582
]
},
{
"f": "skills/ms-ai-engineering/references/mlops-genaiops/responsible-ai-mlops-integration.md",
"l": 733,
"a": "**MCP-calls brukt:** 6 (microsoft_docs_search x 3, microsoft_docs_fetch x 2, microsoft_code_sample_search x 1)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/rag-architecture/rag-cost-optimization.md",
"l": 558,
"a": "**MCP calls:** 3 (search) + 2 (fetch) = 5 total",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-engineering/references/rag-architecture/rag-document-preprocessing.md",
"l": 791,
"a": "**Totalt antall MCP-kilder:** 3 docs_search calls + 2 docs_fetch calls = **5 MCP-kall**",
"b": "konsistent",
"flags": [
"label_value_mismatch"
],
"form": "line"
},
{
"f": "skills/ms-ai-governance/references/monitoring-observability/real-time-streaming-monitoring.md",
"l": 560,
"a": "**MCP calls:** 6 (3 × search, 2 × fetch, 1 × code search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-governance/references/monitoring-observability/response-quality-metrics-rag.md",
"l": 637,
"a": "**MCP research calls:** 3 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 3,
"enumerated": 6,
"delta": -3
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/ai-act-annex-iii-checklist.md",
"l": 523,
"a": "- `microsoft_docs_search`: 2 queries (EU AI Act compliance, Purview AI governance)",
"b": "ikke_sjekkbar",
"flags": [
"provenance_mix"
],
"form": "block",
"span": [
521,
525
]
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/ai-act-compliance-guide.md",
"l": 733,
"a": "- `microsoft_docs_search`: 3 queries (EU AI Act compliance, governance, risk classification)",
"b": "ikke_sjekkbar",
"flags": [
"provenance_mix"
],
"form": "block",
"span": [
732,
735
]
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/ai-impact-assessment-framework.md",
"l": 653,
"a": "**Antall dokumenter søkt:** 4 (search queries) + 2 (deep fetch)",
"b": "ikke_sjekkbar",
"flags": [
"label_value_mismatch"
],
"form": "line"
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/ai-risk-taxonomy-classification.md",
"l": 460,
"a": "- **microsoft_docs_search:** 3 calls (AI risk classification, AI Act levels, Azure framework)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
458,
463
]
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/algorithmic-accountability-auditability.md",
"l": 562,
"a": "### MCP Calls: 6",
"b": "konsistent",
"flags": [],
"form": "block",
"span": [
562,
565
]
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md",
"l": 599,
"a": "**Total MCP calls:** 6 (3 searches + 2 fetches + 1 code sample search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/human-in-the-loop-oversight.md",
"l": 832,
"a": "**MCP Calls:** 6 (3 searches + 2 fetches + 1 code sample search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/model-monitoring-drift-detection.md",
"l": 774,
"a": "### Total MCP Calls: 4",
"b": "inkonsistent",
"flags": [],
"form": "block",
"span": [
774,
777
],
"stated": 4,
"enumerated": 6,
"delta": -2
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/responsible-ai-framework-overview.md",
"l": 374,
"a": "**MCP-søk utført:** 3 søk (microsoft-learn)",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
374,
375
]
},
{
"f": "skills/ms-ai-governance/references/responsible-ai/responsible-ai-policy-development.md",
"l": 561,
"a": "**MCP Calls:** 4 (microsoft_docs_search x3, microsoft_docs_fetch x2)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 4,
"enumerated": 5,
"delta": -1
},
{
"f": "skills/ms-ai-security/references/ai-security-engineering/ai-security-scoring-framework.md",
"l": 514,
"a": "**MCP calls:** 5 (3 søk + 2 fetch)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
"l": 536,
"a": "**MCP-kall:** 6 (3x microsoft_docs_search, 2x microsoft_docs_fetch, 1x microsoft_code_sample_search)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/ai-security-engineering/norwegian-content-safety.md",
"l": 537,
"a": "**MCP-kall:** 6 (microsoft_docs_search x6)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/ai-security-engineering/output-validation-grounding-verification.md",
"l": 694,
"a": "**MCP-kall utført:** 4 (2x docs_search, 1x code_sample_search, 2x docs_fetch)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 4,
"enumerated": 5,
"delta": -1
},
{
"f": "skills/ms-ai-security/references/ai-security-engineering/prompt-injection-defense-patterns.md",
"l": 477,
"a": "- 3 MCP microsoft-learn docs_search calls",
"b": "ikke_sjekkbar",
"flags": [],
"form": "block",
"span": [
476,
478
]
},
{
"f": "skills/ms-ai-security/references/cost-optimization/azure-ai-foundry-cost-governance.md",
"l": 903,
"a": "**Total MCP Calls:** 4 (3x microsoft_docs_search, 1x microsoft_docs_fetch, 1x microsoft_code_sample_search)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 4,
"enumerated": 5,
"delta": -1
},
{
"f": "skills/ms-ai-security/references/cost-optimization/budget-forecasting-ai-projects.md",
"l": 529,
"a": "**Total MCP calls:** 3 (docs_search) + 2 (docs_fetch) + 1 (code_sample_search) = 6",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/cost-optimization/inference-endpoint-cost-optimization.md",
"l": 616,
"a": "**Totalt MCP-kall:** 3 (microsoft_docs_search) + 2 (microsoft_docs_fetch) + 1 (microsoft_code_sample_search) = 6",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/cost-optimization/model-selection-price-performance.md",
"l": 570,
"a": "**MCP-kall brukt:** 6 (4x docs_search, 2x docs_fetch)",
"b": "konsistent",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/cost-optimization/reserved-capacity-planning.md",
"l": 557,
"a": "**MCP Calls:** 3",
"b": "ikke_sjekkbar",
"flags": [],
"form": "line"
},
{
"f": "skills/ms-ai-security/references/cost-optimization/small-language-models-economics.md",
"l": 644,
"a": "**Total MCP-kall:** 4 (3x search, 2x fetch, 1x code samples)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 4,
"enumerated": 6,
"delta": -2
},
{
"f": "skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md",
"l": 611,
"a": "**MCP Calls:** 4 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)",
"b": "inkonsistent",
"flags": [],
"form": "line",
"stated": 4,
"enumerated": 6,
"delta": -2
}
]
}
Reference in a new issue View git blame Copy permalink