feat(ms-ai-architect): R7.3 bølge 1 — payload-06..15 dømt+ingestet (221 claims, 10 flagged, 48 flagg; ledger 99→109) [skip-docs]
This commit is contained in:
parent
80bfd02726
commit
c4554708ca
1 changed files with 687 additions and 1 deletions
|
|
@ -4,7 +4,7 @@
|
|||
"derived_from": "scripts/kb-update/data/full-pass-worklist.json (243 due, all never-verified)",
|
||||
"cadence": "R7–R10 (5 økter × ~49, 8–10 samtidige)",
|
||||
"batch": "R7.1",
|
||||
"count": 99,
|
||||
"count": 109,
|
||||
"generated": "2026-07-18"
|
||||
},
|
||||
"files": [
|
||||
|
|
@ -3555,6 +3555,65 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 17,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/infrastructure-as-code-mlops.md#1",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/powershell/dsc/overview",
|
||||
"evidence_quote": "Microsoft's Desired State Configuration (DSC) is a declarative configuration platform. With DSC, the state of a machine is described using a format that should be clear to understand even if the reader isn't a subject matter expert. Unlike imperative tools, with DSC the definition of an application environment is separate from programming logic that enforces that definition.",
|
||||
"reason": "Deklarativ/imperativ-rammen og plasseringen av Bicep, ARM-templates og Terraform er grunnet (what-is-infrastructure-as-code + WAF OE:05-definisjonene), men den stated, lastbærende delen «PowerShell DSC = imperativt verktøy» motsies: Microsoft beskriver DSC som en deklarativ konfigurasjonsplattform og kontrasterer den eksplisitt mot imperative verktøy (også: «Configurations declare the state of target devices, rather than writing instructions for how to place devices in that state»).",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md",
|
||||
"line": 39,
|
||||
"claim": "IaC-verktøy deles i to hovedtyper: deklarative verktøy (Bicep — Microsofts DSL for Azure som kompilerer til ARM templates | ARM templates (JSON) — Azure Resource Managers native format | Terraform — multi-cloud med Azure provider) og imperative verktøy (Azure CLI-scripts med `az`-kommandoer | PowerShell DSC for VM-konfigurasjon).",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/infrastructure-as-code-mlops.md#9",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/how-to-manage-workspace-terraform",
|
||||
"evidence_quote": "The following table contains a list of resource providers required by Azure Machine Learning: Microsoft.MachineLearningServices | Microsoft.Storage | Microsoft.ContainerRegistry | Microsoft.KeyVault | Microsoft.Notebooks | Microsoft.ContainerService",
|
||||
"reason": "Jeg hentet den kanoniske opplistende tabellen (bekreftet identisk pa bade Terraform- og ARM-mal-artikkelen), og Microsoft.Insights - som claimen pastar er pakrevd - star IKKE i den; Microsoft.Network er dessuten kun betinget (managed VNet), mens Microsoft.Notebooks og Microsoft.ContainerService mangler i claimen.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md",
|
||||
"line": 458,
|
||||
"claim": "Resource providers som må registreres for Azure ML IaC: Microsoft.MachineLearningServices | Microsoft.Storage | Microsoft.KeyVault | Microsoft.ContainerRegistry | Microsoft.Insights | Microsoft.Network.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/infrastructure-as-code-mlops.md#15",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/container-registry/container-registry-skus",
|
||||
"evidence_quote": "Azure Container Registry offers three Pricing Plan options: Basic, Standard, and Premium.",
|
||||
"reason": "Geo-replikeringsdelen holder (Premium adds features such as geo-replication), men den stated, lastbærende SKU-oppregningen holder ikke: claimen sier ACR tilbys i Basic og Premium, mens den kanoniske SKU-siden slar fast tre planer inkludert Standard - som «satisfies the needs of many production scenarios» og derfor er beslutningsendrende for kostnad.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md",
|
||||
"line": 632,
|
||||
"claim": "Azure Container Registry tilbys i SKU-ene Basic og Premium, der Premium brukes for geo-replikering i produksjon.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/infrastructure-as-code-mlops.md#16",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R1",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/azure-vmware/sql-server-hybrid-benefit",
|
||||
"evidence_quote": "Save up to 85% over standard pay-as-you-go rate using Windows Server and SQL Server licenses with Azure Hybrid Benefit.",
|
||||
"reason": "To av tre deler er bekreftet - lisenstypeverdien Windows_Server (ARM: licenseType: Windows_Server; CLI: --set licenseType=Windows_Server) og kravet om eksisterende Windows Server-lisenser med Software Assurance - men den øvre grensen «opptil 40 %» finnes ikke pa noen learn.microsoft.com-side; Microsofts publiserte tak for Azure Hybrid Benefit er «up to 85%», sa den oppgitte ceilingen er ikke Microsofts dokumenterte verdi (R1, upper bound).",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/infrastructure-as-code-mlops.md",
|
||||
"line": 644,
|
||||
"claim": "Azure Hybrid Benefit aktiveres i Terraform via license_type = \"Windows_Server\" og krever eksisterende Windows Server-lisenser; kan gi opptil 40 % lavere VM-kostnad.",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/llm-evaluation-production.md",
|
||||
"batch": "R7.1",
|
||||
|
|
@ -3626,6 +3685,610 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-fundamentals-overview.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 20,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-fundamentals-overview.md#10",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/reliability/regions-list",
|
||||
"evidence_quote": "Access to this region is restricted to support specific customer scenarios, such as disaster recovery within a specific geographic area. To request access to a restricted region for your Azure subscription, see Azure region access request process.",
|
||||
"reason": "Norway East er en full region med tilgjengelighetssoner, men den kanoniske regionslisten merker Norway West som begrenset region — tilgang må søkes om og er forbeholdt scenarioer som katastrofegjenoppretting — så den andre bærende delen (Norway West er tilgjengelig for datalagring) motsies av kilden.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-fundamentals-overview.md",
|
||||
"line": 264,
|
||||
"claim": "Data må lagres i EU/Norge, og Azure-regionene Norway East og Norway West er tilgjengelige for dette.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-fundamentals-overview.md#11",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-serverless-compute?view=azureml-api-2",
|
||||
"evidence_quote": "You can use it to run all types of jobs by using Azure Machine Learning studio, the Python SDK, and Azure CLI.",
|
||||
"reason": "Begge bærende deler svikter: siden er merket v2 (current) uten preview- eller nyhetsmerking og serverless er standard når compute utelates (If no compute target is specified for command, sweep, and AutoML jobs, the compute defaults to serverless compute), og den anbefales for alle jobbtyper inkludert distribuert trening — ikke spesifikt for mindre workloads.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-fundamentals-overview.md",
|
||||
"line": 305,
|
||||
"claim": "Serverless compute i Azure Machine Learning omtales som en ny funksjon, anbefalt for mindre workloads.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-fundamentals-overview.md#17",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R3",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/mlops-maturity-model",
|
||||
"evidence_quote": "The MLOps maturity model encompasses five levels of technical capability.",
|
||||
"reason": "Modellen har fem nivåer med navnene No MLOps, DevOps but no MLOps, Automated training, Automated model deployment og Full MLOps automated operations; verken antallet (claimen sier fire) eller navnene (Manual, Partial automation, Full CI/CD, Full MLOps with monitoring) finnes på siden — rammen claimen beskriver er erstattet.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-fundamentals-overview.md",
|
||||
"line": 351,
|
||||
"claim": "MLOps maturity model beskrives med trinnene Manual | Partial automation | Full CI/CD | Full MLOps with monitoring.",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 31,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-security-access-control.md#13",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/how-to-setup-customer-managed-keys?view=azureml-api-2",
|
||||
"evidence_quote": "Set Key type to RSA. We recommend selecting at least 3072 for the RSA key size.",
|
||||
"reason": "Nøkkeltypen RSA er instruert, men 3072 bit er en anbefaling (We recommend selecting at least 3072), ikke et krav; claimens lastbærende del må være minimum 3072 bit gjør en anbefaling om til en obligatorisk grense.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||
"line": 304,
|
||||
"claim": "CMK-nøkkelen for Azure ML workspace må være en RSA-nøkkel på minimum 3072 bit.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-security-access-control.md#16",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/concept-data-encryption?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Den siterte siden sier kun at Azure Machine Learning uses Transport Layer Security (TLS) uten å oppgi versjon; ingen Learn-side fastslår TLS 1.2 for alle de fire kommunikasjonsveiene (kun Kubernetes online endpoints er eksplisitt dokumentert med TLS 1.2), og ingen side oppgir en motstridende versjon.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||
"line": 340,
|
||||
"claim": "All kommunikasjon i Azure ML bruker TLS 1.2 (workspace til storage account, workspace til compute, Studio til workspace API og inference-klienter til online endpoints).",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-security-access-control.md#22",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/compliance/offerings/cloud-services-in-audit-scope",
|
||||
"evidence_quote": "",
|
||||
"reason": "Learn oppgir ikke tjenestenivå-omfanget: siden om tjenester i revisjonsomfang henviser til Appendiks A og B i et PDF-dokument på Service Trust Portal, så verken ISO 27001, ISO 27018 eller SOC 2 Type II bekreftes eller avkreftes for Azure Machine Learning på learn.microsoft.com.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||
"line": 632,
|
||||
"claim": "Azure Machine Learning er sertifisert mot ISO 27001 | ISO 27018 | SOC 2 Type II.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-security-access-control.md#23",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/reference-machine-learning-cloud-parity?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen Learn-side enumererer regiontilgjengelighet for Azure Machine Learning i offentlig sky (cloud parity-siden dekker Azure Government og 21Vianet); den autoritative listen ligger på azure.microsoft.com Products available by region, utenfor learn.microsoft.com.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||
"line": 634,
|
||||
"claim": "Azure Machine Learning er tilgjengelig i norsk region (Oslo/Norway East).",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-security-access-control.md#24",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/key-vault/general/overview",
|
||||
"evidence_quote": "",
|
||||
"reason": "Key Vault har kun tierne standard og premium, og begge oppgis med customer-managed keys som bruksområde, men verken Azure ML-dokumentasjonen eller Key Vault-sidene stiller et tier-krav for CMK, og lisenspåstanden om managed identities, RBAC og Private Link står ikke på noen Learn-side.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||
"line": 662,
|
||||
"claim": "Customer-managed keys i Azure ML krever Azure Key Vault i tier standard eller premium; managed identities, RBAC og Private Link krever ingen egen lisens ut over Azure-abonnementet/infrastrukturkostnad.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 27,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#5",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/devops/artifacts/start-using-azure-artifacts",
|
||||
"evidence_quote": "Azure Artifacts supports multiple package types, including NuGet, npm, Python, Maven, Cargo, and Universal Packages.",
|
||||
"reason": "NuGet og pip (Python) stemmer, men conda finnes ikke i den kanoniske opplistingen av pakketyper — heller ikke i Feature availability-tabellen (NuGet, dotnet, npm, Maven, Gradle, Python, Cargo, Universal Packages), saa conda-feeden er ikke tilbudt.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 153,
|
||||
"claim": "Azure Artifacts tilbyr pakkefeeds for NuGet | pip | conda til ML-biblioteker og delte komponenter.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#6",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/devops/release-notes/features-timeline-released#azure-devops-services",
|
||||
"evidence_quote": "June 30 2025 | Azure DevOps MCP Server public preview",
|
||||
"reason": "Naturlig-spraak-delen stemmer (what-is-azure-devops viser promptene Summarize the current sprint status, List all work items that are blocked og Show the success rate for all pipelines), men den baerende dateringen er feil: MCP-serveren gikk i public preview 30. juni 2025 (sprint 258) og ble GA i sprint 264 i 2025 — ikke en 2026-funksjon.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 155,
|
||||
"claim": "Azure DevOps MCP Server gir naturlig-språk-spørringer for prosjektstyring (f.eks. «Summarize sprint status», «List blocked work items», «Show pipeline success rates») og er en 2026-funksjon.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#10",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/how-to-devops-machine-learning",
|
||||
"evidence_quote": "- task: AzureMLJobWaitTask@1",
|
||||
"reason": "Den kanoniske siden for Azure Pipelines mot Azure Machine Learning dokumenterer kun AzureMLJobWaitTask@1 fra Machine Learning-utvidelsen, og modellutrulling skjer via pipeline-YAML og az ml CLI (deploy-online-endpoint-pipeline.yml paa den siterte siden); ingen learn-side nevner en oppgave AzureMLModelDeploy@1.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 204,
|
||||
"claim": "Azure Pipelines-oppgaven for modellutrulling til Azure Machine Learning er AzureMLModelDeploy@1.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#12",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/devops/artifacts/start-using-azure-artifacts",
|
||||
"evidence_quote": "Azure Artifacts supports multiple package types, including NuGet, npm, Python, Maven, Cargo, and Universal Packages.",
|
||||
"reason": "Private Python-feeds stemmer, men conda-pakkehosting finnes ikke i pakketype-opplistingen, og Docker image registry (Azure Container Registry) samt dependency security scanning er ikke Azure Artifacts-kapabiliteter — kapabilitetslisten paa what-is-azure-devops er upstream sources, versjonering, tilgangskontroll, build-integrasjon og kodesoek.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 262,
|
||||
"claim": "Azure Artifacts har kapabilitetene private Python-pakkefeeds | conda-pakkehosting | Docker image registry (Azure Container Registry) | dependency security scanning.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#17",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/machine-learning-operations-v2",
|
||||
"evidence_quote": "| Model tester | R | R, KVR | R | R | R | R | LAR | MR |",
|
||||
"reason": "Owner-delen stemmer (kun platform technical support og CI/CD processes har O i produksjonstabellen), men RBAC-tabellene gir data scientists ADS (Machine Learning Data Scientist) — ikke Contributor — paa dev-workspacet, og model testers har Reader i preproduksjon og ingen workspace-rolle i produksjonsmiljoeene (som ifoelge siden inkluderer staging og test), aldri Contributor.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 425,
|
||||
"claim": "Anbefalt RBAC for Azure ML-workspaces: dev-workspace - data scientists har Contributor og data analysts har Reader; staging-workspace - model testers har Contributor og data scientists har Reader; produksjons-workspace - kun CI/CD-prosesser og platform support har Owner.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#19",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R3",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/foundry-classic/concepts/prompt-flow",
|
||||
"evidence_quote": "Prompt flow in Microsoft Foundry and Azure Machine Learning will be retired on April 20, 2027. Prompt flow is no longer recommended for new development.",
|
||||
"reason": "Team-samarbeid rundt flows er bekreftet (Debug, share, and iterate your flows with ease through team collaboration), men rammen er superseded: dokumentasjonen plasserer prompt flow i Microsoft Foundry portal (classic) og Azure Machine Learning studio — produktnavnet Azure AI Studio finnes ikke lenger — og prompt flow er dessuten under utfasing med frist 2027-04-20.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 494,
|
||||
"claim": "Delte prompt flows for team-samarbeid ligger i Azure AI Studio.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#21",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/devops/organizations/security/access-levels",
|
||||
"evidence_quote": "Stakeholders use drag-and-drop to create and change work items, but they can only change the State field on cards.",
|
||||
"reason": "Fem gratis brukere med Basic og ubegrenset antall gratis stakeholders stemmer, men den baerende delen read-only-tilgang er feil: den kanoniske access-levels-siden gir Stakeholder tilgang til View My Work Items med add and modify work items, samt oppretting og endring av kort paa boards.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 541,
|
||||
"claim": "Azure DevOps gratis tier gir opptil 5 brukere med Basic access og ubegrenset antall stakeholders med read-only-tilgang.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#24",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/devops/organizations/projects/migrate-public-project",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen learn.microsoft.com-side oppgir ubegrensede minutter; den naermeste formuleringen sier bare free runner minutes for public repositories uten tallfesting — GitHub-kvoter dokumenteres paa docs.github.com, saa verdien kan verken bekreftes eller motbevises.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 554,
|
||||
"claim": "GitHub Actions gir ubegrenset antall minutter for offentlige repositories.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/mlops-teams-collaboration-tools.md#25",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/power-platform/alm/devops-github-actions",
|
||||
"evidence_quote": "",
|
||||
"reason": "En learn-side bekrefter 2 000 action minutes per maaned gratis, men ingen learn-side oppgir 500 MB artefaktlagring eller knytter tallene til private repositories — den baerende lagringsverdien kan ikke verifiseres paa learn.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||
"line": 557,
|
||||
"claim": "GitHub Actions gratis tier for private repositories gir 2000 minutter per måned og 500 MB lagring for artefakter.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 20,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-deployment-strategies-azure.md#8",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/monitor-azure-machine-learning-reference?view=azureml-api-2",
|
||||
"evidence_quote": "**Requests Per Minute**The number of requests sent to online endpoint within a minute | `RequestsPerMinute`",
|
||||
"reason": "RequestLatency og CpuUtilizationPercentage finnes i den kanoniske metrikklisten, men RequestsPerSecond gjør det ikke — metrikken heter `RequestsPerMinute`; ett bærende (og ordrett kopierbart) metrikknavn er dermed feil.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md",
|
||||
"line": 538,
|
||||
"claim": "Azure Monitor eksponerer metrikkene RequestLatency | RequestsPerSecond | CpuUtilizationPercentage for Azure ML online endpoint deployments.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-deployment-strategies-azure.md#13",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-online-endpoints?view=azureml-api-2",
|
||||
"evidence_quote": "If your bandwidth usage exceeds the limit, your request is delayed. ... Two response trailers are returned if the bandwidth limit is enforced: `ms-azureml-bandwidth-request-delay-ms` ... `ms-azureml-bandwidth-response-delay-ms`",
|
||||
"reason": "5 MBps-delen stemmer, men den andre bærende delen er feil: kilden sier at overskridelse av båndbreddegrensen forsinker forespørselen (med delay-trailere), mens HTTP 429 er dokumentert for «Too many pending requests» og rate-limit på requests per second — ikke for båndbredde.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md",
|
||||
"line": 799,
|
||||
"claim": "Default bandwidth quota for et Azure ML online endpoint er 5 MBps per endpoint; overskridelse gir throttling med HTTP 429-feil.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-deployment-strategies-azure.md#14",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/concept-plan-manage-cost?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen Learn-side oppgir at Azure ML-workspacet er gratis eller at man kun betaler for compute/storage; kostnadssiden lister avhengige ressurser (ACR, Blob Storage, Key Vault, Azure Monitor, load balancer, VNet, båndbredde) og henviser prising til den JS-rendrede azure.microsoft.com-siden. (Kun delpåstanden om ingen tilleggsavgift er dekket: «you pay for the compute and networking charges. There's no added surcharge».)",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md",
|
||||
"line": 817,
|
||||
"claim": "Azure Machine Learning workspace er gratis (man betaler kun for underliggende compute/storage), og alle deployment-funksjoner som blue-green, mirroring og A/B er inkludert.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-drift-performance-degradation.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 21,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-drift-performance-degradation.md#18",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/concept-data-collection?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Verken den siterte konseptsiden, concept-data-collection eller concept-plan-manage-cost sier noe om at Data Collector er kostnadsfri eller inkludert i endpoint-kostnaden; prisinformasjon er ikke oppgitt på noen hentet Learn-side.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-drift-performance-degradation.md",
|
||||
"line": 457,
|
||||
"claim": "Azure ML Data Collector medfører ingen ekstra kostnad – den er inkludert i endpoint-kostnaden.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-drift-performance-degradation.md#19",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/concept-plan-manage-cost?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Kostnadssiden beskriver hvilke ressurser som påløper kostnader sammen med workspacet, men ingen hentet Learn-side slår fast at workspace/Event Grid er uten lisenskostnad eller at Azure Monitor er inkludert i subscriptionen slik claimet formulerer det.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-drift-performance-degradation.md",
|
||||
"line": 470,
|
||||
"claim": "Azure ML Workspace har ingen lisenskostnad (pay-per-use for compute/storage), Event Grid har ingen lisenskostnad (pay-per-event), og Azure Monitor er inkludert i Azure-subscriptions.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-drift-performance-degradation.md#21",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R7",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/databricks/mlflow3/genai/eval-monitor/concepts/judges/",
|
||||
"evidence_quote": "| `RelevanceToQuery` | ... | `RetrievalRelevance` | ... | `Safety` | ... | `RetrievalGroundedness` | ... | `Correctness` | ... | `RetrievalSufficiency` | ... | `Guidelines` |",
|
||||
"reason": "Selve kapabiliteten (samme scorere i utvikling og produksjon) er grunnet, men de navngitte scorerne er lastbærende SDK-identifikatorer: den kanoniske oppregningen av innebygde judges inneholder RetrievalGroundedness og RelevanceToQuery - ingen scorer heter Groundedness eller Relevance i MLflow 3 (de navnene tilhører Azure ML sitt prompt flow GenAI-signal, ikke MLflow-scorerne).",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-drift-performance-degradation.md",
|
||||
"line": 684,
|
||||
"claim": "MLflow 3 production monitoring gjenbruker utviklings-scorerne Groundedness og Relevance på produksjonstraces.",
|
||||
"disposition": "outdated"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 22,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#1",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R3",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/concepts/built-in-evaluators",
|
||||
"evidence_quote": "Similarity | AI-assisted textual similarity measurement.",
|
||||
"reason": "Gjeldende Foundry-enumerasjon navngir metrikken «Similarity» (SDK-klasse SimilarityEvaluator, nøkkel «similarity»); «GPT similarity» er den utgåtte Prompt Flow-/gpt_-prefiks-benevnelsen, så claimets ramme er en omdøpt/erstattet metrikk.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 37,
|
||||
"claim": "AI-assisterte kvalitetsmetrikker i Microsoft Foundry omfatter Groundedness | Relevance | Coherence | Fluency | GPT similarity, krever en judge-modell (GPT-3.5+/GPT-4), og kun GPT similarity krever ground truth.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#3",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/risk-safety-evaluators",
|
||||
"evidence_quote": "Unlike LLM-as-judge evaluators such as coherence and fluency, these evaluators run against Microsoft's hosted safety models.",
|
||||
"reason": "De seks risikonavnene og «krever ikke ground truth» (required inputs: query, response) holder, men den bærende delen «kjøres av en Foundry-hostet GPT-4» motsies: siden sier tjenesten «employs a set of language models» / «hosted safety models» og kontrasterer dem eksplisitt mot GPT-baserte LLM-as-judge-evaluatorer.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 39,
|
||||
"claim": "Risk & Safety-metrikker i Microsoft Foundry omfatter Self-harm | Hateful content | Violence | Sexual content | Protected material | Indirect attack, krever ikke ground truth, og kjøres av en Foundry-hostet GPT-4.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#4",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/how-to/evaluate-generative-ai-app?view=foundry-classic",
|
||||
"evidence_quote": "**Traces** | Evaluates agent interactions already captured in Application Insights. Select the agent and time range, and the portal retrieves the matching traces for evaluation.",
|
||||
"reason": "Steg 1-tabellen «Select evaluation target» lister fire mål — Agent, Model, Dataset OG Traces — så den eksakte påstanden om «tre evalueringsmål» er superseded.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 43,
|
||||
"claim": "Microsoft Foundry støtter tre evalueringsmål: Model | Agent | Dataset.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#5",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators",
|
||||
"evidence_quote": "| Relevance | `query`, `response` | `deployment_name` |",
|
||||
"reason": "Flere deler stemmer (Groundedness med context, Similarity med ground truth, NLP-metrikker med response+ground truth, safety med query+response), men den bærende delen «Relevance krever query + response + context» motsies — RAG-evaluatortabellen krever kun query og response for Relevance.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 51,
|
||||
"claim": "Data mapping-krav for Foundry-evaluering: Groundedness og Relevance krever query + response + context; Coherence og Fluency krever query + response; GPT similarity krever query + response + ground truth; F1/BLEU/ROUGE/METEOR krever response + ground truth; safety-metrikker krever query + response.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#6",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/foundry-classic/how-to/develop/evaluate-sdk",
|
||||
"evidence_quote": "",
|
||||
"reason": "gpt-4o er dokumentert som gyldig judge-deployment, men ingen hentet learn.microsoft.com-side oppgir api_version-verdien 2024-06-01 — model_config-eksemplene bruker gjennomgående miljøvariabler (api_version=os.environ.get(\"AZURE_API_VERSION\")), så verken bekreftelse eller motbevis finnes.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 80,
|
||||
"claim": "Azure AI Evaluation SDK konfigurerer judge-modellen mot Azure OpenAI med api_version 2024-06-01 og azure_deployment gpt-4o.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#8",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R3",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/concepts/scorers",
|
||||
"evidence_quote": "Built-in judges also include **Guidelines judges**, built-in judges that check whether responses pass or fail custom natural-language rules, such as style or factuality guidelines.",
|
||||
"reason": "Siden strukturerer scorers i FIRE tilnærminger (Built-in judges, Custom judges, Code-based scorers, Third-party scorers); Guidelines judges og multi-turn judges er undertyper av built-in judges, og Third-party scorers mangler helt i claimet — claimets fem-type-ramme er erstattet.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 129,
|
||||
"claim": "MLflow 3 har fem scorer-typer: Built-in judges | Guidelines judges | Custom LLM judges | Code-based scorers | Multi-turn judges.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#9",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/concepts/judges/",
|
||||
"evidence_quote": "RelevanceToQuery, RetrievalRelevance, Safety, RetrievalGroundedness, Correctness, RetrievalSufficiency, Guidelines, ExpectationsGuidelines, ToolCallCorrectness, ToolCallEfficiency",
|
||||
"reason": "«Available judges»-tabellen på den kanoniske Built-in LLM judges-siden lister disse ti; verken «Fluency» eller «Equivalence» finnes der — fraværet i den enumererende siden er bevis mot eksistens-påstanden.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 133,
|
||||
"claim": "MLflow 3 built-in judges omfatter Correctness | RetrievalGroundedness | Safety | RelevanceToQuery | Fluency | Equivalence.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#18",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-regions-limits-virtual-network",
|
||||
"evidence_quote": "These regions support the following safety evaluators: Hate and unfairness, Sexual, Violent, Self-harm, Indirect attack, Code vulnerabilities, and Ungrounded attributes.",
|
||||
"reason": "Den kanoniske regionlisten for risk- og safety-evaluatorer er East US 2, North Central US, France Central, Sweden Central, Switzerland West og Australia East — UK South står ikke der, og claimets «kun»-liste utelater tre faktiske regioner.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 384,
|
||||
"claim": "AI-assisterte safety-metrikker er kun hostet i regionene East US 2 | France Central | UK South | Sweden Central.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#19",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/confidential-computing/overview-azure-products",
|
||||
"evidence_quote": "",
|
||||
"reason": "Den siterte observability-siden nevner ikke confidential computing, og confidential-computing-sidene omtaler VM-/GPU-SKU-er (NCCadsH100v5 m.fl.) uten å si noe om GA-status for LLM judges — påstanden kan verken bekreftes eller motbevises mot learn.microsoft.com.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 405,
|
||||
"claim": "Azure confidential computing er ikke GA for LLM judges.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#20",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/foundry/concepts/observability",
|
||||
"evidence_quote": "",
|
||||
"reason": "Den siterte siden sier bare at observability-funksjoner «are billed based on consumption as listed in our Azure pricing page» og peker videre til den JS-rendrede prissiden; ingen learn-side slår fast at plattformen er uten lisenskostnad eller at man kun betaler compute/LLM-tokens.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 428,
|
||||
"claim": "Microsoft Foundry har ingen lisenskostnad for selve plattformen — pay-as-you-go der man kun betaler for compute/LLM-tokens.",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#21",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/databricks/admin/account-settings/account",
|
||||
"evidence_quote": "Azure Databricks is available with two pricing options, Standard and Premium, which offer features for different types of workloads.",
|
||||
"reason": "Azure Databricks har prisnivåene Standard, Premium (og Trial ved opprettelse) — noe «Enterprise-tier» finnes ikke i den kanoniske enumerasjonen, så tier-påstanden er feil.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 430,
|
||||
"claim": "MLflow 3 er inkludert i Databricks-abonnement på Premium- eller Enterprise-tier.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-evaluation-frameworks.md#22",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/python/api/overview/azure/ai-evaluation-readme?view=azure-python",
|
||||
"evidence_quote": "",
|
||||
"reason": "Readme-siden lenker til kildekode og PyPI-pakken, men ingen hentet learn.microsoft.com-side oppgir lisenstype (MIT) eller at SDK-en er gratis å bruke.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-evaluation-frameworks.md",
|
||||
"line": 431,
|
||||
"claim": "Azure AI Evaluation SDK er open source under MIT-lisens og gratis å bruke.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 23,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-versioning-registry-management.md#1",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/how-to-manage-registries?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Hverken registry-siden, MLflow-modellregistersiden eller cloud-parity-tabellen merker modellversjonering/registry-håndtering med GA- eller preview-status; fravær av preview-banner er ingen positiv GA-påstand.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"line": 4,
|
||||
"claim": "Model versioning og registry management i Azure Machine Learning er merket som GA (generelt tilgjengelig).",
|
||||
"disposition": "unsourced"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-versioning-registry-management.md#7",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/how-to-share-models-pipelines-across-workspaces-with-registries?view=azureml-api-2",
|
||||
"evidence_quote": "The registry resource's system-assigned managed identity has `AcrPull` permission on the Azure Container Registry (ACR) instance associated with that registry. When a workspace compute needs to pull an environment image, the AzureML Registry creates and returns an ACR token with an appropriate scope map allowing the image to be pulled by the workspace compute. Neither the workspace nor the compute managed identity has direct access to the registry's ACR.",
|
||||
"reason": "ACR-token-delen stemmer, men den andre bærende delen er motsagt: AcrPull ligger på registryets egen system-tildelte identitet, og siden sier eksplisitt at verken workspace- eller compute-identiteten har direkte ACR-tilgang - workspace-compute får altså ikke AcrPull-rollen.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"line": 82,
|
||||
"claim": "Tilgangskontroll mot Azure ML Registry er ACR token-basert, og workspace-compute får `AcrPull`-rollen via registryets managed identity.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-versioning-registry-management.md#16",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-models-mlflow?view=azureml-api-2",
|
||||
"evidence_quote": "Organizational registries aren't supported for model management with MLflow.",
|
||||
"reason": "Den andre bærende delen er motsagt: organisasjonsregistre støttes ikke for modellhåndtering via MLflow, så azureml://registries/<navn> er en Azure ML asset-URI (CLI/SDK), ikke en MLflow registry-URI; Learn sier dessuten at MLflow-registry-URI-en har samme format og verdi som workspace-ets tracking-URI, ikke formen azureml://<workspace>.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"line": 263,
|
||||
"claim": "MLflow Registry URI i Azure ML har to former: `azureml://<workspace>` for workspace-registry | `azureml://registries/<registry-name>` for Azure ML Registry.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-versioning-registry-management.md#19",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/reliability/regions-list",
|
||||
"evidence_quote": "Access to this region is restricted to support specific customer scenarios, such as disaster recovery within a specific geographic area. To request access to a restricted region for your Azure subscription, see Azure region access request process.",
|
||||
"reason": "Norway West er i Europa-tabellen merket med nettopp dette restricted-ikonet (kun Norway East er en åpen region), så Norway West kan ikke uten videre velges som registry-region for data residency; ingen Learn-side lister dessuten Norway East/West som støttede Azure ML Registry-regioner - den ene bærende delen faller.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"line": 344,
|
||||
"claim": "Azure ML Registry kan konfigureres i regionene Norway East og Norway West for data residency.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-versioning-registry-management.md#20",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R8",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/concept-plan-manage-cost?view=azureml-api-2",
|
||||
"evidence_quote": "Bandwidth charges reflect usage; the more data transferred, the greater the charge.",
|
||||
"reason": "Andre bærende del er motsagt: kostnadssiden lister Azure Container Registry, Key Vault, Azure Monitor, load balancer (rundt $0.33/dag per compute instance), virtuelt nettverk/private endpoints og båndbredde som kostnader i tillegg til compute og storage - kun compute og storage koster stemmer ikke.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"line": 428,
|
||||
"claim": "Azure Machine Learning er inkludert i Azure-abonnementet uten ekstra lisenskostnad; kun compute og storage koster.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/model-versioning-registry-management.md#21",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/concept-mlflow?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Learn omtaler MLflow som an open-source framework / the largest open source AI engineering platform, men ingen learn.microsoft.com-side oppgir Apache 2.0-lisensen eller at MLflow er gratis - lisensverdien kan ikke bekreftes eller avkreftes.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
|
||||
"line": 429,
|
||||
"claim": "MLflow er open source under Apache 2.0-lisens og gratis.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/monitoring-observability-ml-systems.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 25,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/monitoring-observability-ml-systems.md#24",
|
||||
"judge_verdict": "not_grounded",
|
||||
"rule": "R2",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/azure-monitor/logs/manage-access",
|
||||
"evidence_quote": "Built-in roles ... Privileged Monitoring Data Reader ... Log Analytics Data Reader ... Log Analytics Reader ... Log Analytics Contributor ... The /read permission is usually granted from a role that includes */read or * permissions, such as the built-in Reader and Contributor roles.",
|
||||
"reason": "Den kanoniske siden for tilgangsstyring på Log Analytics workspace lister Reader/Contributor, Log Analytics Reader/Contributor/Data Reader og Privileged Monitoring Data Reader, men Monitoring Metrics Publisher er fraværende - den rollen gir kun Microsoft.Insights/Metrics/Write (publisering av metrikker) og er ikke en tilgangsrolle for Log Analytics-workspace, så én bærende del av oppramsingen er feil.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/monitoring-observability-ml-systems.md",
|
||||
"line": 423,
|
||||
"claim": "RBAC-roller for tilgangskontroll på Log Analytics workspace: Reader | Contributor | Log Analytics Reader | Monitoring Metrics Publisher.",
|
||||
"disposition": "outdated"
|
||||
},
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/monitoring-observability-ml-systems.md#25",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Ingen learn.microsoft.com-side omtaler en Model Monitoring v3 eller en Q2 2026-roadmap; konseptsiden og how-to-sidene beskriver kun dagens signaler, og Learn publiserer ikke produktroadmaps, så påstanden kan verken bekreftes eller motbevises.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/monitoring-observability-ml-systems.md",
|
||||
"line": 663,
|
||||
"claim": "Azure ML Model Monitoring v3 er på roadmap for Q2 2026.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/prompt-flow-production-deployment.md",
|
||||
"batch": "R7.1",
|
||||
|
|
@ -3636,6 +4299,29 @@
|
|||
"verified_by": "judge-v3.1",
|
||||
"flags": []
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/responsible-ai-mlops-integration.md",
|
||||
"batch": "R7.3",
|
||||
"judged_at": "2026-07-25",
|
||||
"per_file_verdict": "flagged",
|
||||
"claim_count": 15,
|
||||
"verified": null,
|
||||
"verified_by": null,
|
||||
"flags": [
|
||||
{
|
||||
"id": "ms-ai-engineering/mlops-genaiops/responsible-ai-mlops-integration.md#11",
|
||||
"judge_verdict": "source_silent",
|
||||
"rule": "",
|
||||
"evidence_url": "https://learn.microsoft.com/azure/machine-learning/concept-plan-manage-cost?view=azureml-api-2",
|
||||
"evidence_quote": "",
|
||||
"reason": "Verken den siterte siden eller kostnadssiden (som lister ACR, Blob Storage, Key Vault, Azure Monitor og compute som kostnadsdrivere) sier noe om at Model Registry er uten ekstrakostnad; påstanden kan verken bekreftes eller motbevises på learn.microsoft.com.",
|
||||
"file": "skills/ms-ai-engineering/references/mlops-genaiops/responsible-ai-mlops-integration.md",
|
||||
"line": 595,
|
||||
"claim": "Model Registry er inkludert i Azure Machine Learning-workspace uten ekstrakostnad.",
|
||||
"disposition": "unsourced"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"file": "skills/ms-ai-engineering/references/rag-architecture/azure-ai-search-setup.md",
|
||||
"batch": "R7.1",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue