ms-ai-architect/scripts/kb-eval/data/r74-payloads.json

[
  {
    "file": "skills/ms-ai-governance/references/monitoring-observability/response-quality-metrics-rag.md",
    "claim_count": 20,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/monitoring-observability/response-quality-metrics-rag.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/monitoring-observability/response-quality-metrics-rag.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#1\",\n    \"claim\": \"Response quality metrics og evaluering for RAG-systemer i Microsoft Foundry / Azure AI Evaluation har status GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#2\",\n    \"claim\": \"De AI-assisterte RAG-spesifikke evaluatorene i Azure AI Evaluation er: Groundedness | Groundedness Pro | Relevance | Response Completeness | Retrieval | Coherence | Fluency.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#3\",\n    \"claim\": \"Groundedness Pro utfører en streng consistency-sjekk med Azure AI Content Safety, tar Query, Context og Response som input, og returnerer boolsk True/False med reason – ikke en 1-5 Likert-score.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#4\",\n    \"claim\": \"Document Retrieval-evaluatoren krever ground truth (query relevance labels) og gir metrikkene Fidelity | NDCG | XDCG | Max Relevance | Holes.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#5\",\n    \"claim\": \"De NLP-baserte similarity-metrikkene i Azure AI Evaluation er: F1 Score | BLEU | GLEU | ROUGE | METEOR, og alle tar Response og Ground truth som input.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/observability#what-are-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#6\",\n    \"claim\": \"Azure AI-evaluatorer støtter både reasoning models (o-series) og non-reasoning models (GPT-4o, GPT-4.1) som LLM-dommer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/observability#what-are-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#7\",\n    \"claim\": \"GroundednessEvaluator i azure.ai.evaluation har parameteren is_reasoning_model (settes til True for o-series reasoning models) i tillegg til model_config og threshold.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/python/api/azure-ai-evaluation/azure.ai.evaluation.groundednessevaluator\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#8\",\n    \"claim\": \"gpt-4.1-mini er en GPT-4.1-variant og ikke en o-series reasoning model, og anbefales som kostnadseffektiv dommermodell for standard evaluering.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#9\",\n    \"claim\": \"DocumentRetrievalEvaluator konfigureres med ground_truth_label_min=0, ground_truth_label_max=4 og ndcg_threshold, det vil si query relevance labels på skalaen 0-4.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#10\",\n    \"claim\": \"AI-assisterte evaluatorer bruker Likert-skala 1-5 med en threshold for pass/fail, og default threshold er 3 (score ≥3 = pass, <3 = fail).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#11\",\n    \"claim\": \"Foundry-evalueringer opprettes med client.evals.create der testing_criteria bruker type azure_ai_evaluator og evaluator_name-verdier som builtin.groundedness | builtin.relevance.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/evaluate-generative-ai-app\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#12\",\n    \"claim\": \"Prompt Flow utfases 20. april 2027 i Microsoft Foundry og Azure Machine Learning og anbefales ikke for ny utvikling; QAEvaluator i Azure AI Evaluation SDK er ikke berørt av utfasingen.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/flow-bulk-test-evaluation\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#13\",\n    \"claim\": \"QAEvaluator er en composite evaluator som kjører groundedness | relevance | coherence | fluency | similarity | F1 samtidig.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/python/api/azure-ai-evaluation/azure.ai.evaluation.qaevaluator\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#14\",\n    \"claim\": \"Azure Databricks støtter MLflow 3 GenAI evaluation via mlflow.genai.evaluate med scorers som RetrievalGroundedness og RetrievalRelevance.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#15\",\n    \"claim\": \"Azure OpenAI kan brukes i Norge-regioner (Norway East) for data residency ved evaluering.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#16\",\n    \"claim\": \"Groundedness Pro bruker Azure AI Content Safety-tjenesten, og data kan teoretisk sendes til US-regioner.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#17\",\n    \"claim\": \"For evaluering finnes to forbruksmodeller for Azure OpenAI: PAYG (Pay-as-you-go) for pre-production-testing og ad-hoc-evalueringer, og PTU (Provisioned Throughput) med fast månedlig kostnad og garantert kapasitet for kontinuerlig produksjonsevaluering.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#18\",\n    \"claim\": \"Microsoft Foundry har en gratis tier for evaluerings-UI-et; man betaler kun for underliggende compute (GPT-kall).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#19\",\n    \"claim\": \"Azure AI Content Safety (brukt av Groundedness Pro) er inkludert i Azure-abonnementet og faktureres pay-per-transaction.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/response-quality-metrics-rag.md#20\",\n    \"claim\": \"Azure AI Evaluation SDK krever ingen spesielle lisenser – det er et open source Python-bibliotek.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/observability\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/monitoring-observability/response-quality-metrics-rag.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/monitoring-observability/sla-monitoring-ai-services.md",
    "claim_count": 23,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/monitoring-observability/sla-monitoring-ai-services.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/monitoring-observability/sla-monitoring-ai-services.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#1\",\n    \"claim\": \"Azure OpenAI tilbyr SLA for både tilgjengelighet (Availability SLA) og latens (Latency SLA), der Latency SLA gjelder Provisioned-Managed deployments.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/faq#what-are-the-slas-service-level-agreements-in-azure-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#2\",\n    \"claim\": \"Availability SLA for Azure OpenAI garanterer 99.9% oppetid og gjelder alle Azure OpenAI-ressurser.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/faq#what-are-the-slas-service-level-agreements-in-azure-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#3\",\n    \"claim\": \"Latency SLA gjelder kun Provisioned-Managed deployments i Azure OpenAI, og garantert nivå varierer per modell.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/faq#what-are-the-slas-service-level-agreements-in-azure-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#4\",\n    \"claim\": \"Azure OpenAI har ingen standard Throughput SLA; throughput-garantier (requests/second eller tokens/second) må avtales separat som custom.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/faq#what-are-the-slas-service-level-agreements-in-azure-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#5\",\n    \"claim\": \"Azure Monitor Platform Metrics samler automatisk inn metrikkene AvailabilityRate | ModelAvailabilityRate for Azure OpenAI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/monitor-openai-reference\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#6\",\n    \"claim\": \"Metric Alerts i Azure Monitor støtter dynamic thresholds og multi-resource-regler.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#7\",\n    \"claim\": \"Simple Log Search Alerts er i preview og evaluerer hver logg-rad individuelt (nær-sanntid), raskere enn tradisjonelle log alerts.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#8\",\n    \"claim\": \"Azure Monitor-metrikkene for SLA-tracking av Azure OpenAI er AvailabilityRate | ModelAvailabilityRate | ModelRequests | StatusCode (dimensjon) | TimeToFirstToken | ContextTokens | GeneratedTokens.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/monitor-openai-reference\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#9\",\n    \"claim\": \"AvailabilityRate beregnes som (Total Calls - Server Errors) / Total Calls der server errors er HTTP-statuskoder >= 500, og aggregeres som Average over 5 minutter.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-cognitiveservices-accounts-metrics\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#10\",\n    \"claim\": \"ModelAvailabilityRate måler tilgjengelighet per modell-deployment og aggregeres som Min/Max/Average.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-cognitiveservices-accounts-metrics\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#11\",\n    \"claim\": \"StatusCode er en dimensjon i Azure OpenAI-metrikkene, med HTTP-statuskoder som 200 | 429 | 500 | 503.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/monitor-openai-reference\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#12\",\n    \"claim\": \"TimeToFirstToken måler latens til første token ved streaming, knyttes til Latency SLA for PTU-deployments, og aggregeres som P95/P99.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/monitor-openai-reference\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#13\",\n    \"claim\": \"AvailabilityRate skal ikke brukes for Azure OpenAI-tjenester; ModelAvailabilityRate skal brukes i stedet.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-cognitiveservices-accounts-metrics\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#14\",\n    \"claim\": \"Log Analytics-workspace støtter retention fra 30 til 730 dager.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#15\",\n    \"claim\": \"Standard retention for Azure Monitor platform metrics er 93 dager.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#16\",\n    \"claim\": \"Azure OpenAIs 99.9% SLA gjelder for Norway East-regionen, og West Europe kan brukes som failover-region ved lengre utfall.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/faq#what-are-the-slas-service-level-agreements-in-azure-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#17\",\n    \"claim\": \"Power BI Pro eller Premium kreves kun dersom SLA-rapporter skal deles bredt i organisasjonen, og er ikke obligatorisk for SLA-monitorering.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#18\",\n    \"claim\": \"Azure Monitor har alert-typene metric | log search | simple log search (preview, per-rad evaluering) | activity log | smart detection | Prometheus.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#19\",\n    \"claim\": \"Alerts i Azure Monitor lagres i 30 dager.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#20\",\n    \"claim\": \"Stateless metric alerts med evalueringsfrekvens under 5 minutter trigges 1-6 minutter etter, mens frekvens på 5 minutter eller mer trigges 15-30 minutter etter.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#21\",\n    \"claim\": \"Stateful log search alerts settes til resolved når betingelsen ikke er oppfylt i 2-3 frekvensperioder (avhengig av frekvens).\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#22\",\n    \"claim\": \"Query-based metric alerts for Prometheus/OpenTelemetry er i public preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/sla-monitoring-ai-services.md#23\",\n    \"claim\": \"Azure AI Search krever 2 replicas for å oppnå 99.9% SLA.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search#service-level-agreement\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/monitoring-observability/sla-monitoring-ai-services.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/monitoring-observability/token-usage-tracking-attribution.md",
    "claim_count": 22,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/monitoring-observability/token-usage-tracking-attribution.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/monitoring-observability/token-usage-tracking-attribution.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#1\",\n    \"claim\": \"Token-telling i Azure OpenAI varierer per modell og er basert på modell-spesifikk tokenizer, med GPT-2-tokenizeren (tiktoken-encoding 'gpt2') som baseline.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/manage-costs\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#2\",\n    \"claim\": \"Azure OpenAI samler automatisk disse token-baserte platform metrics i Azure Monitor: TokenTransaction (total token count, input + output) | InputTokens | OutputTokens | ProcessedPromptTokens (tokens faktisk prosessert, kan avvike ved caching).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#3\",\n    \"claim\": \"Azure OpenAI tilbyr out-of-box dashboards med kategorien «Tokens-Based Usage», som viser token consumption over tid | breakdown per modell | comparison mot quota limits.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#4\",\n    \"claim\": \"Token-sporing via API Management bruker loggtabellen ApiManagementGatewayLogs, der Azure OpenAI-kall identifiseres med OperationId-verdiene completions_create | chatcompletions_create, og token-data hentes fra BackendResponseBody-feltene usage.prompt_tokens | usage.completion_tokens | usage.total_tokens.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-monitoring\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#5\",\n    \"claim\": \"Azure OpenAI-ressurser kan tagges for kostnadsattribuering med Azure CLI-kommandoen az openai account update --name <navn> --resource-group <rg> --tags.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/manage-costs\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#6\",\n    \"claim\": \"Azure OpenAI har separate token-baserte rate limits for input tokens per minutt (ITPM) og output tokens per minutt (OTPM).\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/concepts/manage-costs\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#7\",\n    \"claim\": \"Azure OpenAI On Your Data (RAG) gjør to LLM-kall per brukerforespørsel: intent prompt (reformulering av query til search intents) | generation prompt (generering av svar basert på retrieved chunks).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#8\",\n    \"claim\": \"I Azure OpenAI On Your Data utgjør meta prompt (system instructions, avhengig av inScope-parameteren) 400-4000 tokens, modell-avhengig.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#9\",\n    \"claim\": \"I Azure OpenAI On Your Data er brukerspørsmål pluss samtalehistorikk begrenset til maks 2000 tokens.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#10\",\n    \"claim\": \"I Azure OpenAI On Your Data utgjør retrieved chunks 5-10 chunks à 1024 tokens, det vil si 5000-10000 tokens.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#11\",\n    \"claim\": \"Eksempel for gpt-35-turbo-16k i Azure OpenAI On Your Data: generation prompt 4297 tokens, intent prompt 1366 tokens, response output 111 tokens, intent output 25 tokens — totalt ca. 5800 tokens per spørsmål.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#12\",\n    \"claim\": \"Token-bruk i Azure OpenAI On Your Data justeres via parameterne retrieved_document_count (default 5) | chunk_size (default 1024) | strictness | inScope.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data#token-usage-estimation-for-azure-openai-on-your-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#13\",\n    \"claim\": \"Fine-tuned modeller i Azure OpenAI har tre kostnadskomponenter: training cost (per token i training-filen) | hosting cost (timepris mens modellen er deployet, uansett bruk) | inference cost (per 1000 tokens, input + output).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#14\",\n    \"claim\": \"En fine-tuned Azure OpenAI-deployment slettes automatisk etter 15 dager med inaktivitet; modellen bevares og kan redeployes.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#15\",\n    \"claim\": \"Azure Monitor eksponerer metrikkene FineTuningHours og FineTuningTokens for Azure OpenAI under ResourceProvider MICROSOFT.COGNITIVESERVICES.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#16\",\n    \"claim\": \"Azure OpenAI har to faktureringsmodeller for token-sporing: pay-as-you-go (consumption-based, spor individuelle tokens) | PTU (Provisioned Throughput Units, spor utnyttelsesprosent mot reservert kapasitet).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/provisioned-throughput-billing\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#17\",\n    \"claim\": \"Metrikken AzureOpenAIProvisionedManagedUtilizationV2 viser prosentandelen av reservert PTU-kapasitet som er i bruk.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#18\",\n    \"claim\": \"Input TPM per PTU er modell-spesifikk, for eksempel 8450 TPM for Llama-3.3-70B.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/provisioned-throughput-billing\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#19\",\n    \"claim\": \"Azure Well-Architected Framework anbefaler som minimum budget alert-terskler på 90 % (ideelt forbruk), 100 % (mål) og 110 % (mindre ideelt), samt 110 % for forecast alerts.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/cost-optimization/collect-review-cost-data#generate-cost-reports\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#20\",\n    \"claim\": \"Azure Cost Management beholder kostnadsdata i 13 måneder.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/cost-optimization/collect-review-cost-data#generate-cost-reports\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#21\",\n    \"claim\": \"Cost Management-eksport til Azure Data Lake Storage Gen2 støtter FOCUS-malen (FinOps Open Cost and Usage Specification) — et leverandøragnostisk, standardisert skjema.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/cost-optimization/collect-review-cost-data#generate-cost-reports\"\n  },\n  {\n    \"id\": \"ms-ai-governance/monitoring-observability/token-usage-tracking-attribution.md#22\",\n    \"claim\": \"Azure Cost Management skiller mellom actual costs (som fakturert, med 24-72 timers ingestion-forsinkelse) og amortized costs (commitment-baserte kjøp fordelt jevnt over perioden).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/cost-optimization/collect-review-cost-data#generate-cost-reports\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/monitoring-observability/token-usage-tracking-attribution.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/citizen-communication-ai-decisions.md",
    "claim_count": 15,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/citizen-communication-ai-decisions.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/citizen-communication-ai-decisions.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#1\",\n    \"claim\": \"Model Interpretability-komponenten i Responsible AI Dashboard (Azure Machine Learning) genererer menneskeforståelige forklaringer av modellprediksjoner på tre nivåer: global forklaring | lokal forklaring | kohort-forklaring\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#2\",\n    \"claim\": \"Responsible AI Dashboard inneholder en Counterfactual What-If-komponent som viser hvordan modeller reagerer på endringer i input-faktorer\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#3\",\n    \"claim\": \"Responsible AI Scorecard er et konfigurerbart PDF-rapportverktøy som kan tilpasses for både tekniske og ikke-tekniske interessenter\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#4\",\n    \"claim\": \"Microsoft Foundry Evaluation Tools omfatter safety metrics | hallucination detection | bias assessment for å vurdere modellkvalitet før produksjonssetting\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#5\",\n    \"claim\": \"Azure AI Language PII Detection kan automatisk detektere og fjerne personopplysninger (telefonnummer, e-postadresser) fra treningsdata og logg-data\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#6\",\n    \"claim\": \"Microsoft Learn grupperer PII-funksjonalitet i tre feature-typer: Document-based PII | Conversation PII | Text PII\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#7\",\n    \"claim\": \"Document-based PII (tidligere omtalt «Native Document PII») detekterer og redigerer PII i native dokumentformater `.pdf` | `.docx` | `.txt` og bevarer dokumentstruktur\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#8\",\n    \"claim\": \"Document-based PII er en asynkron, lagringsbasert pipeline som gir maskinlesbar JSON-metadata\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#9\",\n    \"claim\": \"Conversation PII er en tur-/transkriptbasert modell for tale-transskripsjoner og uformelle møtereferater\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#10\",\n    \"claim\": \"Text PII er synkron tekstanalyse med forhåndsdefinerte kategorier: navn | adresser | personnummer | finansielle kontonumre\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#11\",\n    \"claim\": \"Text PII har GA-API-versjon `2026-05-01` og preview-versjon `2026-05-15-preview`\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#12\",\n    \"claim\": \"PII-funksjonaliteten er tilgjengelig via Microsoft Foundry-portalen (ny) eller Foundry (classic), og ressursen heter «Azure Language in Foundry Tools»\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#13\",\n    \"claim\": \"PII-deteksjonen støtter norsk tekst gjennom en flerspråklig NLP-modell\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#14\",\n    \"claim\": \"PII-deteksjonen har en egen dedikert kategori `NOIdentityNumber` for norsk fødselsnummer\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/citizen-communication-ai-decisions.md#15\",\n    \"claim\": \"Azure OpenAI med Zero Data Retention lagrer ikke prompts og completions, og tjenesten gjenbruker dem ikke\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/citizen-communication-ai-decisions.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/copyright-ai-training-data-norway.md",
    "claim_count": 15,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/copyright-ai-training-data-norway.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/copyright-ai-training-data-norway.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#1\",\n    \"claim\": \"Microsoft tilbyr Customer Copyright Commitment (CCC) som juridisk garanti i Product Terms fra 1. desember 2023, der Microsoft forplikter seg til å forsvare kunder mot visse tredjepartskrav om opphavsrettsbrudd relatert til Output Content.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#2\",\n    \"claim\": \"CCC-dekningen omfatter: Azure OpenAI Service (de universelle kravene gjelder) | konfigurerbare GAI-tjenester, det vil si Microsoft Copilot Studio og GitHub Copilot (dekkes av egne tjeneste-spesifikke mitigations, ikke de universelle kravene).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#3\",\n    \"claim\": \"GitHub Offerings har per 3. april 2026 ingen tilleggskrav (required mitigations) for CCC-dekning.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#4\",\n    \"claim\": \"Copilot Studio har egne «bring your own model»-krav for CCC-dekning, effektive fra 1. juni 2025.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#5\",\n    \"claim\": \"De universelle required mitigations for CCC-dekning består av to krav: Metaprompt | Testing and Evaluation Report.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#6\",\n    \"claim\": \"Metaprompt-kravet i CCC (kundens løsning må inkludere en metaprompt som instruerer modellen til å forhindre opphavsrettsbrudd i output) er effektivt fra 1. desember 2023.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#7\",\n    \"claim\": \"Microsoft Learns «Safety system message templates» inneholder en kategori «Protected Material – Text» med eksempel på anbefalt metaprompt mot opphavsrettsbrudd.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/safety-system-message-templates\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#8\",\n    \"claim\": \"Kravet om Testing and Evaluation Report i CCC (evalueringer som guided red teaming, systematisk måling eller ekvivalent tilnærming for å oppdage output av tredjepartsinnhold) er effektivt fra 1. desember 2023.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#9\",\n    \"claim\": \"For nye tjenester, funksjoner, modeller eller bruksområder publiseres nye CCC-krav og trer i kraft ved eller etter lansering.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#10\",\n    \"claim\": \"Ellers har kunder seks måneder fra publisering til å implementere nye CCC-mitigations for å opprettholde dekning.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/customer-copyright-commitment?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#11\",\n    \"claim\": \"Microsofts datagarantier for Azure OpenAI (Models sold by Azure) omfatter seks punkter: kundens prompts, completions, embeddings og treningsdata er ikke tilgjengelig for andre kunder | kundens data er ikke tilgjengelig for OpenAI eller andre leverandører av Models sold by Azure | kundens data brukes ikke av leverandørene til å forbedre deres modeller eller tjenester | kundens data brukes ikke til å trene generative AI-grunnmodeller uten kundens tillatelse eller instruksjon | kundens data, prompts og completions brukes ikke til å forbedre Microsofts eller tredjeparters produkter eller tjenester uten kundens eksplisitte tillatelse | fine-tunede modeller er eksklusivt tilgjengelig for kundens bruk.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#12\",\n    \"claim\": \"GPT-3-serien er trent på offentlig tilgjengelig fri tekst bestående av: 60 % filtrert Common Crawl | WebText-datasett | internett-bøker | Wikipedia.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#13\",\n    \"claim\": \"GPT-4 er trent på offentlig tilgjengelig data (internett) og data lisensiert av OpenAI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#14\",\n    \"claim\": \"Modellene i Azure OpenAI er fine-tunet med RLHF (reinforcement learning with human feedback).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/copyright-ai-training-data-norway.md#15\",\n    \"claim\": \"Microsoft tilbyr data residency i flere europeiske regioner, inkludert Norway East, for fine-tuning-operasjoner.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/copyright-ai-training-data-norway.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-ai-governance-structure.md",
    "claim_count": 12,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-ai-governance-structure.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-ai-governance-structure.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#1\",\n    \"claim\": \"I Microsofts delte ansvarsmodell for AI-styring dekker Microsoft for SaaS (Microsoft Copilot): full application stack | modell-lifecycle | plugin governance | sikkerhetssystemer, mens kunden dekker: policies | brukeropplæring | output-validering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#2\",\n    \"claim\": \"I den delte ansvarsmodellen dekker Microsoft for PaaS (Azure AI): plattformsikkerhet | underliggende tjenester, mens kunden dekker: modelldesign | tuning | integrasjon.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#3\",\n    \"claim\": \"I den delte ansvarsmodellen dekker Microsoft for IaaS: infrastruktur, mens kunden dekker: modelldesign | implementasjon | drift.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#4\",\n    \"claim\": \"Kundeansvar som gjelder uavhengig av tjenestemodell består av fem punkter: etablere AI governance og tilsynsmekanismer | brukerpolicies, review-prosesser og opplæring | identitets-, enhets- og tilgangsstyring | data governance-rammeverk (klassifisering, beskyttelse, livssyklus) | mapping til compliance-krav (GDPR, EU AI Act etc.).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#5\",\n    \"claim\": \"Azure Policy og Microsoft Purview gir automatisert policy enforcement for AI-styring: reduserer menneskelig feil | sikrer konsistent policyapplisering på tvers av alle AI-deployments | sanntidsmonitorering og umiddelbar respons på policybrudd, mens manuell enforcement brukes for komplekse scenarioer som krever menneskelig vurdering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#6\",\n    \"claim\": \"Microsofts AI Center of Excellence (AI CoE) har åtte ansvarsområder: definere AI-strategi | utvikle AI-kompetanse | lede pilotprosjekter | definere og håndheve AI-standarder | opprette intake- og prioriteringsworkflows | utvikle gjenbrukbare assets | måle og rapportere resultater | administrere AI-tjenester (valgfritt).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#7\",\n    \"claim\": \"Rollene i et AI Center of Excellence omfatter: representanter fra juridisk, sikkerhet, produkt og engineering | eksekutiv sponsing med klar myndighet til å håndheve policies | konsultativ støtte fremfor gatekeeper-rolle.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#8\",\n    \"claim\": \"Microsofts Responsible AI-rammeverk er basert på seks prinsipper: Fairness | Reliability and Safety | Privacy and Security | Inclusiveness | Transparency | Accountability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/responsible-ai-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#9\",\n    \"claim\": \"Miljøpreparering for governance av AI-agenter består av: governed application landing zones for PaaS/IaaS | konfigurasjon av identitet, tilgang og data governance for SaaS (Copilot Studio).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/governance-security-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#10\",\n    \"claim\": \"Data governance for AI-agenter omfatter: isolering av konfidensiell data | restriksjoner på datatilgang og permissions | standardisering av kunnskaps- og tool-integrasjoner | mandatory transparency om AI-involvering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/governance-security-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#11\",\n    \"claim\": \"Auditing og transparency for AI-agenter omfatter: planlagte audits av deployede AI-agenter | monitoring for model drift, bias og risikoprofiler | incident response protocols | public transparency reports.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/governance-security-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-ai-governance-structure.md#12\",\n    \"claim\": \"Prosedyrer for continuous risk monitoring av AI: kvartalsvis risikovurdering for høyrisiko AI-workloads | årlige vurderinger for lavrisiko systemer | strukturert måleplan med både kvantitative metrics og kvalitative indikatorer | standardiserte rapporter til stakeholders | uavhengige reviews (eksterne auditører eller interne reviewere).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-ai-governance-structure.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md",
    "claim_count": 7,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#1\",\n    \"claim\": \"AI-genererte svar må være tilgjengelige for skjermleserne Narrator | JAWS | NVDA.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/windows/configuration/accessibility/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#2\",\n    \"claim\": \"Anbefalte lesevennlige fonter for dysleksi i AI-grensesnitt er Fluent Sitka Small | Fluent Calibri.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/windows/configuration/accessibility/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#3\",\n    \"claim\": \"Azure OpenAI er navnet på Microsofts Azure-tjeneste for språkmodeller som en virksomhet kan velge å bruke.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/ai/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#4\",\n    \"claim\": \"GPT-4 er en tilgjengelig modell som kan score høyt på MMLU-benchmarken.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/ai/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#5\",\n    \"claim\": \"Microsoft Human-AI Experiences Design Library (HAX Toolkit) er tilgjengelig og kan brukes som utgangspunkt ved design av AI-grensesnitt.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/ai/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#6\",\n    \"claim\": \"Azure Well-Architected Framework har egen veiledning for AI-arbeidslaster bestående av design methodology | application design | responsible AI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/ai/design-methodology\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md#7\",\n    \"claim\": \"Power Platform Well-Architected inneholder pilaren Experience Optimization med anbefalinger for en brukersentrert designstrategi (user-centered design).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/well-architected/experience-optimization/user-centered-design\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-1-user-centric-design.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-2-interoperability.md",
    "claim_count": 15,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-2-interoperability.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-2-interoperability.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#1\",\n    \"claim\": \"Kjerneteknologiene i Azure Integration Services er Azure API Management | Azure Logic Apps | Azure Service Bus | Azure Event Grid | Azure Functions | Azure Data Factory.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/integration/integration-start-here\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#2\",\n    \"claim\": \"Azure Integration Services dekker de fire grunnleggende integrasjonsmønstrene orchestration | messaging | events | API-er.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/integration/integration-start-here\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#3\",\n    \"claim\": \"Azure API Management kan publisere modeller/tjenester som managed API-er med en developer portal.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#4\",\n    \"claim\": \"Azure API Management støtter rate limiting | caching | authentication | transformation.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#5\",\n    \"claim\": \"Azure API Management støtter OAuth 2.0 med Microsoft Entra ID for sikker autentisering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#6\",\n    \"claim\": \"Azure API Management støtter datatransformasjon fra XML til JSON, samt versjonshåndtering og backward compatibility.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#7\",\n    \"claim\": \"Azure API Center gir sentralisert tracking | discovery | reuse | governance av API-er.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#8\",\n    \"claim\": \"Azure Logic Apps har over 400 konnektorer for både skytjenester og on-premises-systemer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#9\",\n    \"claim\": \"Azure Service Bus støtter AMQP (Advanced Message Queuing Protocol) for enterprise messaging.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#10\",\n    \"claim\": \"Azure Service Bus tilbyr to meldingsmodeller: Queue-modell (én-til-én) | Topic/Subscription-modell (pub/sub).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#11\",\n    \"claim\": \"Azure Event Grid kan koble tjenester til handlere som Azure Functions | Logic Apps | custom handlers.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#12\",\n    \"claim\": \"Azure Event Grid har innebygd retry-logikk for event-basert utvikling.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#13\",\n    \"claim\": \"Azure Data Factory støtter ETL/ELT for å transformere data til felles format.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/integration/integration-start-here\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#14\",\n    \"claim\": \"Azure Data Factory integrerer med Azure AI Search | Azure Machine Learning | Synapse Analytics.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/integration/integration-start-here\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-2-interoperability.md#15\",\n    \"claim\": \"Azure Service Bus gir garantert levering | load leveling | transactional messaging.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-2-interoperability.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-4-trust-security.md",
    "claim_count": 22,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-4-trust-security.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-4-trust-security.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#1\",\n    \"claim\": \"Microsoft Foundry tilbyr managed endpoints med Azure Private Link.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#2\",\n    \"claim\": \"Azure OpenAI støtter customer-managed keys (CMK) for datakryptering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#3\",\n    \"claim\": \"Azure Machine Learning støtter nettverksisolasjon med VNet/Subnet.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#4\",\n    \"claim\": \"Purview leverer data governance og lineage tracking.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#5\",\n    \"claim\": \"Azure OpenAI har innebygd content filtering som omfatter prompt shields | groundedness detection.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/app-modernization-guidance/innovate/innovate-and-automate-using-ai-services#build-responsible,-secure-ai-systems\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#6\",\n    \"claim\": \"Azure OpenAI tilbyr Provisioned Throughput Units (PTU) som modell for garantert kapasitet.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#7\",\n    \"claim\": \"Microsoft Foundry tilbyr managed endpoints med auto-scaling.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#8\",\n    \"claim\": \"Azure Front Door gir global load balancing og DDoS-beskyttelse.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#9\",\n    \"claim\": \"Azure OpenAI støtter diagnostikk-logging til Log Analytics.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#10\",\n    \"claim\": \"Microsoft Foundry tilbyr model monitoring med data drift detection.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#11\",\n    \"claim\": \"Zero Trust bygger på tre kjerneprinsipper: Never trust, always verify | least privilege access (Just-In-Time, Just-Enough-Access) | assume breach.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#12\",\n    \"claim\": \"Microsoft har publisert en ny adopsjonsramme, Zero Trust adoption framework, med business-outcome-fokusert implementering.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#13\",\n    \"claim\": \"Microsofts nettverksveiledning for Zero Trust omfatter kryptering av all nettverkstrafikk | mikrosegmentering med NSG og Azure Firewall | avvikling av legacy VPN til fordel for identitetsbaserte tilnærminger.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#14\",\n    \"claim\": \"Zero Trust-ressursbeskyttelse mot destruktive angrep omfatter resource locks | immutable backups | geo-replication.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#15\",\n    \"claim\": \"Microsofts Zero Trust-arkitektur dekker seks pilarer: Identities | Devices | Applications | Data | Infrastructure | Networks.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#16\",\n    \"claim\": \"Azure OpenAI støtter Managed Identity-autentisering | VNET-injection | Private Endpoints.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#17\",\n    \"claim\": \"Azure AI Search støtter RBAC på index-nivå | network isolation | CMK-kryptering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#18\",\n    \"claim\": \"Azure Machine Learning støtter VNET-isolerte workspaces | Managed Identity for compute | private endpoints for model serving.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#19\",\n    \"claim\": \"Copilot Studio støtter Data Loss Prevention (DLP) | Conditional Access | audit logging.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#20\",\n    \"claim\": \"Conditional Access-politikker kan baseres på brukerrisiko (Entra ID Protection) | enhetsstatus (Intune compliance) | lokasjon (geografisk/IP-basert) | applikasjonssensitivitet.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/security/fundamentals/zero-trust\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#21\",\n    \"claim\": \"Customer-managed keys (CMK) via Azure Key Vault støttes for Azure Storage | Azure OpenAI (fine-tuned modeller) | Azure AI Search (indexer).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/zero-trust/deploy/networks\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digdir-principle-4-trust-security.md#22\",\n    \"claim\": \"TLS 1.3 kan aktiveres for all data i transit i Azure.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/zero-trust/deploy/networks\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/digdir-principle-4-trust-security.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-accessibility-action-plan.md",
    "claim_count": 14,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-accessibility-action-plan.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-accessibility-action-plan.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#1\",\n    \"claim\": \"Microsoft Learn-modulen «Use AI tools to create an inclusive learning environment» er tilgjengelig for målgruppene K-12 lærere | bedriftsbrukere | utdanningsinstitusjonell ledelse.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/training/modules/use-ai-tools-to-create-inclusive-learning-environment/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#2\",\n    \"claim\": \"Læringsmålene i Microsoft Learn-modulen «Use AI tools to create an inclusive learning environment» er: gjenkjenne AI-rollen i å støtte samarbeidslæring | vurdere tekst-til-tale-teknologi og hvem som har nytte av den | forstå hvordan AI i Microsoft Teams forbedrer tilgjengelighet for brukere med hørselshemming eller ADHD.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/training/modules/use-ai-tools-to-create-inclusive-learning-environment/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#3\",\n    \"claim\": \"Microsoft Learn-modulen «Use AI tools to create an inclusive learning environment» dekker adaptiv læring | AI-drevet tilbakemelding | personalisert innholdslevering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/training/modules/use-ai-tools-to-create-inclusive-learning-environment/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#4\",\n    \"claim\": \"Microsofts Responsible AI Standard bygger på seks prinsipper: Fairness | Reliability and Safety | Privacy and Security | Inclusiveness | Transparency | Accountability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#5\",\n    \"claim\": \"Microsofts verktøy for tilgjengelig AI omfatter Azure AI Speech (tekst-til-tale og tale-til-tekst) | Azure AI Translator (flerspråklig støtte) | Immersive Reader (forenklet lesing) | Azure AI Vision (bildegjenkjenning) | Copilot Studio (chatbots med innebygde tilgjengelighetsfunksjoner).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/training/modules/create-accessible-solutions-using-ai-innovations/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#6\",\n    \"claim\": \"Accessibility Insights støtter automatisk tilgjengelighetstesting av web | Windows | Android.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#7\",\n    \"claim\": \"Azure AI Speech tilbyr tekst-til-tale | tale-til-tekst | talegjenkjenning.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#8\",\n    \"claim\": \"Azure AI Translator støtter 100+ språk, inkludert nynorsk og bokmål.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#9\",\n    \"claim\": \"Azure AI Vision tilbyr bildeanalyse | OCR | ansiktsgjenkjenning.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#10\",\n    \"claim\": \"Immersive Reader tilbyr forenklet lesing | opplesing | oversettelse.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#11\",\n    \"claim\": \"Azure AI Document Intelligence gjør strukturert tekstekstraksjon fra PDF og bilder.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#12\",\n    \"claim\": \"Azure OpenAI med modellen GPT-4o gir multimodal forståelse og lang kontekst.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#13\",\n    \"claim\": \"Copilot Studio har innebygde funksjoner for tilgjengelighet: Adaptive Cards | SSML-støtte (Speech Synthesis Markup Language) | sentiment analysis | handoff til menneskelig agent.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-accessibility-action-plan.md#14\",\n    \"claim\": \"Azure AI Speech har støtte for norsk språk i tekst-til-tale og tale-til-tekst.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/regulatory/offering-wcag-2-1\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-accessibility-action-plan.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md",
    "claim_count": 19,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#1\",\n    \"claim\": \"Microsoft Foundry har en model catalog som brukes til modellregister og lineage tracking.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#2\",\n    \"claim\": \"Budsjettmodellene for tokenforbruk i Azure OpenAI er PTU (provisioned throughput units) | pay-per-token.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#3\",\n    \"claim\": \"Azure Policy og Compliance Manager kan gjøre automatisk sjekk av AI Act-krav.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#4\",\n    \"claim\": \"Microsoft Purview dekker data governance og lineage tracking.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#5\",\n    \"claim\": \"Azure Information Protection brukes til klassifisering av sensitive data.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#6\",\n    \"claim\": \"Power Platform CoE Starter Kit inneholder AI governance workflows.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#7\",\n    \"claim\": \"Azure AI Search støtter vektorsøk | semantisk søk.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#8\",\n    \"claim\": \"Azure AI Document Intelligence brukes til strukturert ekstraksjon av innhold.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#9\",\n    \"claim\": \"Azure OpenAI Embeddings tilbyr modellen text-embedding-3-large for vektor-representasjon.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#10\",\n    \"claim\": \"Azure OpenAI Service tilbyr API for modellene GPT-4o | o1-preview.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#11\",\n    \"claim\": \"Microsoft Foundry er en felles plattform for modell | data | evaluering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#12\",\n    \"claim\": \"Azure API Management er en API-gateway med rate limiting | versjonering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#13\",\n    \"claim\": \"Event Grid og Service Bus brukes for event-drevne AI-workflows.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#14\",\n    \"claim\": \"Azure AI Content Safety dekker moderation | red teaming.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#15\",\n    \"claim\": \"Azure Machine Learning har Responsible AI Dashboard for bias-evaluering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#16\",\n    \"claim\": \"Microsoft Copilot Studio Analytics gir bruks- og kvalitetsdata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#17\",\n    \"claim\": \"Power Platform-integrasjonsmønstrene er: Instant trigger | event-driven | data consolidation | service-oriented | synchronization.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/architecture/key-concepts/integration-patterns/patterns\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#18\",\n    \"claim\": \"Data integration patterns for Microsoft industry clouds omfatter: Real-time | asynchronous | batch | presentation layer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/industry/well-architected/cross-industry/data-integration-patterns\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md#19\",\n    \"claim\": \"Integrasjonsmønstrene for Dynamics 365 finance and operations er: Synchronous | asynchronous | event-driven.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/dynamics365/guidance/techtalks/integrate-finance-operations-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-samhandling-eif-5-layers.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md",
    "claim_count": 16,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#1\",\n    \"claim\": \"Azure AI Services tilbyr verktøyene Responsible AI Dashboard | Model Cards | Transparency Notes, som kan brukes som utgangspunkt for dokumentasjonskravet.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#2\",\n    \"claim\": \"Azure Machine Learning sitt Responsible AI Dashboard tilbyr model interpretability | counterfactual analysis.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#3\",\n    \"claim\": \"Azure OpenAI har zero data retention (ingen lagret data), noe som sikrer personvern men gjør sporing og forklarbarhet vanskeligere.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#4\",\n    \"claim\": \"Azure Machine Learning — Model Monitoring tilbyr drift detection | data quality monitoring.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#5\",\n    \"claim\": \"Microsofts Responsible AI-rammeverk er bygget på seks prinsipper: Transparency | Fairness | Reliability & Safety | Privacy & Security | Accountability | Inclusiveness.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#6\",\n    \"claim\": \"Fairness assessment er tilgjengelig som en del av Responsible AI Dashboard (RAI Dashboard).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#7\",\n    \"claim\": \"Microsofts verktøy for prinsippet Privacy & Security omfatter Azure Confidential Computing | zero data retention.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#8\",\n    \"claim\": \"Azure Application Insights gir full request/response-logging.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#9\",\n    \"claim\": \"Microsoft Foundry tilbyr komplett RAI-verktøysett | model governance | prompt flow for menneske-i-sløyfen.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#10\",\n    \"claim\": \"Azure Machine Learning tilbyr fullstendig MLOps | Responsible AI Dashboard | model interpretability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#11\",\n    \"claim\": \"Power Platform AI Builder har lav kode-terskel | innebygd forklaring | bruker-UI for manuell review, og er begrenset i kompleksitet (ikke for avanserte modeller).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#12\",\n    \"claim\": \"Copilot Studio har menneske-i-sløyfen innebygd og dekker kun dialog-/samtalebaserte løsninger.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#13\",\n    \"claim\": \"Microsoft Foundry støtter RAG-arkitektur med citation for generativ AI med dokumentgrunnlag.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#14\",\n    \"claim\": \"Azure AI Document Intelligence brukes for dokumentforståelse (OCR og forståelse av opplastede dokumenter).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#15\",\n    \"claim\": \"Azure ML sin Responsible AI Dashboard dekker model interpretability | fairness assessment | error analysis.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md#16\",\n    \"claim\": \"Microsoft Responsible AI Standard er publisert i versjon 2 (v2), General Requirements.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/forvaltningsloven-ai-decisions.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md",
    "claim_count": 8,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#1\",\n    \"claim\": \"DFØ og Prosjektveiviseren opererer med tre hovedkategorier av gevinster: Ytelsesforbedrende gevinster (Performance improvement) | Kostnadsbesparelser (Direct/indirect cost savings) | Risikoreduksjon (Risk mitigation).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/adoption/business-value\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#2\",\n    \"claim\": \"Microsoft anbefaler kombinerte målemetoder for forretningsverdi: stakeholder-intervjuer (kvalitativ innsikt) | surveyer og tilbakemeldingsskjemaer (kvantitativ data) | brukeranalyse (adopsjonsrate, feature usage) | ROI-kalkulatorer | 360-graders feedback.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/adoption/business-value-methods\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#3\",\n    \"claim\": \"Verktøy for oppfølging og måling av AI-gevinster: Power BI (visualisering av KPIer og trender over tid) | Copilot Dashboard (Viva Insights) | Business Value Toolkit (Power Platform CoE).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/coe/business-value-toolkit\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#4\",\n    \"claim\": \"Copilot Dashboard er en del av Viva Insights og gir produktivitetsmetrikker og brukeropplevelsesdata for Microsoft 365 Copilot.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/copilot/microsoft-365/copilot-control-system/measurement-reporting\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#5\",\n    \"claim\": \"Business Value Toolkit (Power Platform CoE) består av et fem-trinns storytelling-rammeverk, en value calculator («average cost of things») og value cards som publiseres til SharePoint.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/coe/business-value-toolkit\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#6\",\n    \"claim\": \"Business Value Toolkits storytelling-rammeverk har fem trinn: problem | løsning | hindringer | verdi | neste steg.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/coe/business-value-toolkit\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#7\",\n    \"claim\": \"Power Platform CoE Starter Kit vedlikeholdes ikke lenger aktivt (status per 2026-06).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/coe/business-value-toolkit\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md#8\",\n    \"claim\": \"Kjernekapabilitetene fra Power Platform CoE Starter Kit er nå del av Power Platform admin center.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/power-platform/guidance/coe/business-value-toolkit\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/gevinstrealisering-ai-projects.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/norge-ai-strategy-government.md",
    "claim_count": 15,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/norge-ai-strategy-government.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/norge-ai-strategy-government.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#1\",\n    \"claim\": \"Microsofts 6 AI-prinsipper for ansvarlig AI er: Fairness | Reliability & Safety | Privacy & Security | Inclusiveness | Transparency | Accountability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#2\",\n    \"claim\": \"Microsofts Responsible AI-rammeverk er integrert i Microsoft Foundry | Copilot Studio | M365 Copilot.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#3\",\n    \"claim\": \"Med Microsoft Cloud for Sovereignty lagres data i norske Azure-regioner: Norge Øst og Norge Vest.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#4\",\n    \"claim\": \"Microsofts løsninger for offentlig sektor og deres leveransemodell er: Microsoft 365 Copilot (SaaS generativ AI) | Microsoft Foundry (PaaS/IaaS) | Copilot Studio (low-code AI-agenter) | Power Platform AI (automatisering og AI Builder).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/training/modules/enhance-public-sector-services-generative-ai/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#5\",\n    \"claim\": \"Azure Cloud Adoption Framework for AI består av fasene Strategy → Plan → Ready → Govern → Secure → Manage.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#6\",\n    \"claim\": \"Secure AI i Cloud Adoption Framework er en 3-fase prosess: Discover AI risks → Protect AI resources & data → Detect AI threats (kontinuerlig).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#7\",\n    \"claim\": \"Secure AI (CAF) anbefaler trusselmodellering med STRIDE som grunnlag, supplementert med MITRE ATLAS og OWASP Generative AI risk.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#8\",\n    \"claim\": \"For AI asset inventory anbefaler Secure AI (CAF) Azure Resource Graph for å oppdage AI-ressurser og Microsoft Defender for Cloud for å identifisere generative AI-workloads.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#9\",\n    \"claim\": \"Secure AI (CAF) anbefaler for sikre kommunikasjonskanaler: managed identities (uten lagrede credentials) | Virtual Networks for isolasjon | Azure API Management for å sikre MCP-endepunkter.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#10\",\n    \"claim\": \"Microsoft Purview Insider Risk Management dekker prompt-basert dataeksfiltrering og identifikasjon av risikabel AI-atferd.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#11\",\n    \"claim\": \"AI Security Posture Management (DSPM) i Microsoft Defender for Cloud gir automatisk deteksjon og remediation av generative AI-risikoer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#12\",\n    \"claim\": \"Microsoft Purview DLP for AI scanner og blokkerer sensitive data i AI-workflows, og støtter custom content filtering for organisasjonsspesifikke mønstre.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/secure\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#13\",\n    \"claim\": \"Microsoft Foundry støtter datasuverenitet og norsk datalagring.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#14\",\n    \"claim\": \"Microsoft utvikler produktene Copilot og Azure OpenAI i samsvar med EU AI Act.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/norge-ai-strategy-government.md#15\",\n    \"claim\": \"Brukerfeedback-løkker for AI-løsninger settes opp med Azure AI Studio og Copilot Studio Analytics.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/norge-ai-strategy-government.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md",
    "claim_count": 13,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#1\",\n    \"claim\": \"Azure RBAC tilbyr granulære AI-roller: AI Developer | AI User | Model Deployer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#2\",\n    \"claim\": \"Copilot Studio har sikkerhetsrollene Publisher | Author | Viewer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#3\",\n    \"claim\": \"Azure OpenAI Content Filters lar deg konfigurere terskelverdier for kategoriene hate | violence | sexual | self-harm.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#4\",\n    \"claim\": \"Prompt Shields i Microsoft Foundry forsvarer mot jailbreak | indirect attacks.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#5\",\n    \"claim\": \"Azure OpenAI Data Residency gir EU-region for databehandling (data processing).\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#6\",\n    \"claim\": \"Copilot Studio Authentication støtter Entra ID | OAuth | manual configuration.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#7\",\n    \"claim\": \"Azure OpenAI håndhever TLS 1.2 for kommunikasjon og krypterer data i ro med Microsoft-managed eller customer-managed keys.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#8\",\n    \"claim\": \"Azure AI Search støtter customer-managed keys (CMK) for vektorlagre (vector stores).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#9\",\n    \"claim\": \"Azure AI Search har geo-redundant backup av vektordata.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#10\",\n    \"claim\": \"Azure OpenAI kan geo-pinnes til Sverige, Norge (via Sweden) eller andre EU-regioner.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#11\",\n    \"claim\": \"Microsoft cloud security benchmark (MCSB) v1.0 er gjeldende standard, og v3.0 er også tilgjengelig for nyere baselines.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/introduction\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#12\",\n    \"claim\": \"Azure AI Content Safety og Prompt Shields er produksjonsklare tjenester med GA-status.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md#13\",\n    \"claim\": \"Private Link for Azure OpenAI er generelt tilgjengelig i alle Azure-regioner.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/security-baselines-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/nsm-grunnprinsipper-ai-mapping.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/ros-analyse-ai-systems.md",
    "claim_count": 17,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/norwegian-public-sector-governance/ros-analyse-ai-systems.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/norwegian-public-sector-governance/ros-analyse-ai-systems.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#1\",\n    \"claim\": \"Kapasitetsplanlegging for AI-tjenester i Azure gjøres med PTU (Provisioned Throughput Units).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#2\",\n    \"claim\": \"Azure Secure Score / Azure Security Benchmark kan brukes til å evaluere sikkerheten for Azure OpenAI | Azure AI Search | Azure ML.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#3\",\n    \"claim\": \"Defender for AI Workloads i Microsoft Defender for Cloud er i preview, og detekterer unormale AI-interaksjoner.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#4\",\n    \"claim\": \"Microsoft Defender for Clouds AI-relevante kapabiliteter omfatter Defender for AI Workloads | Just-in-Time (JIT) access | Threat intelligence.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#5\",\n    \"claim\": \"Microsoft Purview Compliance Manager støtter overvåking av compliance mot GDPR | AI Act | ISO 27001.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#6\",\n    \"claim\": \"Microsoft Purview Compliance Manager består av komponentene Compliance Score | Improvement Actions | Assessments.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#7\",\n    \"claim\": \"Microsoft Purview Compliance Manager inneholder den forhåndsdefinerte assessment-malen «Data Protection Baseline».\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#8\",\n    \"claim\": \"Microsoft Threat Modeling Tool gir strukturert trusselmodellering basert på STRIDE-rammeverket.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#9\",\n    \"claim\": \"Microsoft Threat Modeling Tool støtter import av Azure-arkitektur (Microsoft Foundry, Copilot Studio) | identifisering av trust boundaries | automatisk generering av trusler basert på dataflyt | eksport til Azure DevOps for sporing av tiltak.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#10\",\n    \"claim\": \"STRIDE består av Spoofing | Tampering | Repudiation | Information Disclosure | Denial of Service | Elevation of Privilege.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/mission-critical/mission-critical-security#threat-modeling\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#11\",\n    \"claim\": \"Azure OpenAI-ressurser har Azure-ressurstypen Microsoft.CognitiveServices/accounts, og offentlig nettverkstilgang styres av egenskapen Microsoft.CognitiveServices/accounts/publicNetworkAccess (verdi Enabled).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#12\",\n    \"claim\": \"Azure Blueprints kan levere standard-oppsett med private endpoints, logging og RBAC, og har compliance-preset for GDPR eller ISO 27001.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#13\",\n    \"claim\": \"Microsoft Sentinel (SIEM) brukes for AI-tjenester til anomalideteksjon | threat hunting | incident response med automatiske playbooks.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#14\",\n    \"claim\": \"Diagnostikklogger for Azure Cognitive Services / Azure OpenAI finnes i tabellen AzureDiagnostics med ResourceProvider MICROSOFT.COGNITIVESERVICES og loggkategorien RequestResponse.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#15\",\n    \"claim\": \"Azure AI Search støtter RBAC og sikkerhet på dokumentnivå (document-level security).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#16\",\n    \"claim\": \"Microsoft Cloud Security Benchmark v2 inneholder DevOps Security-kontrollen DS-1: Conduct threat modeling.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-devop-security#ds-1-conduct-threat-modeling\"\n  },\n  {\n    \"id\": \"ms-ai-governance/norwegian-public-sector-governance/ros-analyse-ai-systems.md#17\",\n    \"claim\": \"Microsoft SDL Threat Modeling Tool er et gratis verktøy for trusselmodellering.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/security/threat-model\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/norwegian-public-sector-governance/ros-analyse-ai-systems.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/ai-center-of-excellence-setup.md",
    "claim_count": 20,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/ai-center-of-excellence-setup.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/ai-center-of-excellence-setup.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#1\",\n    \"claim\": \"Microsoft anbefaler fire strukturmodeller for AI Center of Excellence: Sentralisert CoE | Unified CoE | Federated CoE | Desentralisert CoE.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#2\",\n    \"claim\": \"AI CoE-teamet består av åtte roller: AI CoE Leader | Senior Data Scientist | ML Engineer | AI Governance Expert | AI Security Specialist | AI Operations Professional | Business Leader | Data Engineer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#3\",\n    \"claim\": \"Microsoft Cloud Adoption Framework fordeler ansvar mellom tre team — Platform Team | Workload Team | AI CoE — der Platform Team er Responsible for technical foundation & guardrails og governance & security policies, Workload Team er Responsible for model deployment & lifecycle og business requirements & data curation, og AI CoE er Responsible for Responsible AI policy, training & competency building og use case prioritization.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#4\",\n    \"claim\": \"Anbefalte learning pathways for AI-kompetansebygging i CoE: Azure AI Fundamentals | Azure AI Engineer Associate | Azure Data Scientist Associate.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/paths/ai-center-excellence/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#5\",\n    \"claim\": \"Microsoft angir fire signaler (inflection points) for overgang fra sentralisert CoE til advisory-modell: Approval delays | Knowledge bottlenecks | Priority friction | Policy compliance.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#6\",\n    \"claim\": \"Modenhetsnivåene (Microsoft 365 Maturity Model tilpasset Azure AI) har fem nivåer: 100 - Initial | 200 - Managed | 300 - Defined | 400 - Predictable | 500 - Optimizing.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#7\",\n    \"claim\": \"Governance- og policy-håndhevingslaget i en AI CoE-plattform bygges på fire Microsoft-tjenester: Azure Policy (model restrictions, content filtering) | Microsoft Purview (compliance, data governance) | Microsoft Entra Agent ID (agent inventory) | Defender for Cloud (AI risk detection).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#8\",\n    \"claim\": \"Copilot Studio-compliance omfatter sertifiseringene ISO | SOC | HIPAA.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#9\",\n    \"claim\": \"Cloud Adoption Framework peker på nøkkelpolicyer for AI innenfor fire tjenester: Microsoft Foundry (model deployment restrictions, content filter enforcement) | Azure AI Services (allowed SKUs, network isolation) | Azure AI Search (encryption, network security) | Azure OpenAI (model restrictions, content filtering).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#10\",\n    \"claim\": \"For dataresidens i Norge/EU skal norsk offentlig sektor bruke Azure-regionene Norway East og Norway West, håndhevet via Azure Policy og Purview.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#11\",\n    \"claim\": \"Microsoft Foundry kan kjøres i regionen Norway East for å oppnå dataresidens i Norge.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#12\",\n    \"claim\": \"Lisensbehov per CoE-rolle: CoE Lead = M365 E5 + Azure subscription contributor | Data Scientist = M365 E3 + Microsoft Foundry + VS Enterprise | ML Engineer = M365 E3 + Azure DevOps + GitHub Copilot | AI Governance Expert = M365 E5 Compliance + Purview | Security Specialist = M365 E5 Security + Defender for Cloud.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#13\",\n    \"claim\": \"Cloud Adoption Framework sitt AI-scenario har oppdatert struktur i seks steg: Strategy → Plan → Ready → Govern → Manage → Secure, og AI CoE er referert under «Manage AI operations» steg 1.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#14\",\n    \"claim\": \"Copilot Tuning er i preview, og model retirement tracking gjøres via Foundry-portalen.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#15\",\n    \"claim\": \"CAF beskriver fem steg for å bygge AI CoE-teamet: sikre executive sponsorship | utnevne AI CoE-leder | sette sammen AI CoE-teamet | plassering i organisasjonen | definere driftsmodell.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#16\",\n    \"claim\": \"CAF definerer sju ansvarsområder for AI CoE: AI-strategi | AI-kompetanse | Pilotprosjekter | Standarder og governance | Intake og prioritering | Gjenbrukbare assets | Resultater og rapportering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/center-of-excellence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#17\",\n    \"claim\": \"AI CoE-læringsstien på Microsoft Learn består av tre moduler: Introduction to AI Center of Excellence (7 units, Beginner level) | Guide AI workload operations with an AI Center of Excellence (10 units) | Introduction to AI Landing Zones.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/paths/ai-center-excellence/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#18\",\n    \"claim\": \"CAF sin AI agents-veiledning definerer en tre-roller-modell for agent-adopsjon: Platform-team (teknisk fundament og governance-rekkverk) | Workload-team (ende-til-ende-livsløp for konkrete agenter) | AI CoE (sentral rådgivende kropp for strategi og ekspertise).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/ai-agents/organization-people-readiness-plan\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#19\",\n    \"claim\": \"Fem ferdighetsområder for agent-aktiverte team: Prompt engineering | Agent optimization | AI ethics & governance | Data engineering for AI | AI security.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/ai-agents/organization-people-readiness-plan\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-center-of-excellence-setup.md#20\",\n    \"claim\": \"Microsoft angir tre indikatorer for at CoE bør gå fra sentralisert kontroll til rådgivende rolle: approval delays / kunnskapsflaskehalser | friksjon med produktteam om prioriteringer | plattformkapabilitet der platform-team kan håndheve governance på tvers av workloads.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/ai-agents/organization-people-readiness-plan\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/ai-center-of-excellence-setup.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/ai-ethics-in-public-sector.md",
    "claim_count": 20,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/ai-ethics-in-public-sector.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/ai-ethics-in-public-sector.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#1\",\n    \"claim\": \"Microsofts Responsible AI-standard bygger på seks prinsipper: fairness | reliability and safety | privacy and security | inclusiveness | transparency | accountability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#2\",\n    \"claim\": \"Microsoft Cloud Adoption Framework anbefaler fire governance-roller for AI: AI Governance Board | AI Center of Excellence | Data Protection Officer (DPO) | AI Ethics Committee.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/govern\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#3\",\n    \"claim\": \"Azure Machine Learning Responsible AI Dashboard kan dokumentere treningsdata (datakilder, representativitet, bias-testing) | modellens beslutningslogikk (feature importance, SHAP/LIME-forklaringer) | evalueringsmetrikker (fairness, accuracy, precision/recall per demografisk gruppe).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#4\",\n    \"claim\": \"Responsible AI Scorecard kan genereres som PDF eller HTML for innsynsforespørsler.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#5\",\n    \"claim\": \"Azure Machine Learning Model Registry gir versjonshåndtering og lineage for modeller.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-model-management\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#6\",\n    \"claim\": \"Azure AI-tjenester kan deployes i regionene Norway East og West Europe.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ai-services/openai/data-privacy\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#7\",\n    \"claim\": \"Azure OpenAI Service i Europa har data residency-garanti gjennom EU Data Boundary.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ai-services/openai/data-privacy\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#8\",\n    \"claim\": \"Fairness Assessment i Responsible AI Dashboard måler disparate impact ratio og equal opportunity difference.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-fairness-ml\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#9\",\n    \"claim\": \"Fairlearn er et Python-bibliotek for fairness-analyse, open source fra Microsoft Research.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-fairness-ml\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#10\",\n    \"claim\": \"Datasuverenitet per plattform: Azure OpenAI dekkes av EU Data Boundary | Azure ML gir valgfri region | Copilot Studio dekkes av Power Platform data residency.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#11\",\n    \"claim\": \"Microsoft Foundry tilbyr tre Responsible AI-kapabiliteter: Content Safety Studio | Prompt Shields | Groundedness Detection.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#12\",\n    \"claim\": \"Content Safety Studio filtrerer fire kategorier ulovlig innhold: hat | vold | seksuelt | selvskading.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#13\",\n    \"claim\": \"Microsoft Purview har governance-kapabilitetene Data Catalog | Sensitivity Labels | Data Lineage | Compliance Manager.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/purview/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#14\",\n    \"claim\": \"Microsoft Entra ID het tidligere Azure AD.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#15\",\n    \"claim\": \"Microsoft Entra ID gir accountability gjennom Role-Based Access Control (RBAC) | Privileged Identity Management (PIM) | Audit Logs.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#16\",\n    \"claim\": \"EU Data Boundary er Azures forpliktelse til å holde persondata innenfor EU/EØS.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#17\",\n    \"claim\": \"Azure-regionene Norway East og Norway West garanterer data residency.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#18\",\n    \"claim\": \"Microsoft Purview lisensieres per bruker via F5 Security + Compliance.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/purview/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#19\",\n    \"claim\": \"Microsoft 365 E5 inkluderer Purview Information Protection | Compliance Manager | Audit Logs.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/purview/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-ethics-in-public-sector.md#20\",\n    \"claim\": \"Power Platform Premium inkluderer AI Builder | Copilot Studio | Premium connectors.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/ai-ethics-in-public-sector.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/ai-risk-taxonomy-classification.md",
    "claim_count": 25,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/ai-risk-taxonomy-classification.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/ai-risk-taxonomy-classification.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#1\",\n    \"claim\": \"Microsofts risikotaksonomi er tilpasset EU AI Acts fire hovedkategorier: unacceptable | high | limited | minimal.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/legal/ai-code-of-conduct\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#2\",\n    \"claim\": \"Kategorien Unacceptable Risk (totalt forbud) omfatter forbudte bruksområder: social scoring | real-time facial recognition (law enforcement) | subliminal manipulation | exploitation av sårbare grupper.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/legal/ai-code-of-conduct\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#3\",\n    \"claim\": \"Kategorien High Risk krever strict compliance, human oversight og impact assessment, og omfatter: critical infrastructure | employment decisions | credit scoring | healthcare diagnosis | biometric identification.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/legal/ai-code-of-conduct\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#4\",\n    \"claim\": \"Microsoft bruker en 5-nivå severitetsmodell for å prioritere sikkerhetsrisikoer: Critical | High | Medium | Low | Informational.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/ai-red-team/ai-risk-assessment\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#5\",\n    \"claim\": \"Microsofts risikomatrise for adversarial ML-angrep omfatter fem angrepstyper med likelihood/impact/exploitability: Extraction (High/Low/High) | Evasion (High/Medium/High) | Inference (Medium/Medium/Medium) | Inversion (Medium/High/Medium) | Poisoning (Low/High/Low).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/ai-red-team/ai-risk-assessment\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#6\",\n    \"claim\": \"Azure AI Content Safety og Microsoft Responsible AI Standard definerer seks primære innholdsrisiko-kategorier: Hate and Fairness | Sexual Content | Violence | Self-Harm | Protected Material | Jailbreak (User Prompt Injection).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/default-safety-policies\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#7\",\n    \"claim\": \"Kategoriene Hate and Fairness, Sexual Content, Violence og Self-Harm graderes på skalaen Safe (0) til High (6), med standardterskel Medium (blokkerer 4 og over).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/default-safety-policies\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#8\",\n    \"claim\": \"Kategoriene Protected Material (text, code) og Jailbreak (User Prompt Injection) rapporteres binært som Detected / Not Detected, og standardoppførselen er å blokkere alt som detekteres.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/default-safety-policies\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#9\",\n    \"claim\": \"Microsoft organiserer AI-sikkerhet i tre pillarer — AI Platform Security | AI Application Security | Monitor and Respond — og MCSB v2 dekker sikkerhetskontrollene AI-1 til AI-7.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#10\",\n    \"claim\": \"Pillar 1 (AI Platform Security) i MCSB v2 omfatter kontrollene: approved models / model approval process (AI-1) | network segmentation og VPN (NS-2) | identity management (IM-3) | logging og monitoring (LT-3) | data-at-rest encryption (DP-4).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#11\",\n    \"claim\": \"Pillar 2 (AI Application Security) i MCSB v2 omfatter kontrollene: multi-layered content filtering (AI-2, Azure AI Content Safety) | safety meta-prompts (AI-3) | least privilege for agent functions og plugins (AI-4) | Human-in-the-Loop (AI-5) | discover, classify, label sensitive data (DP-1).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#12\",\n    \"claim\": \"Pillar 3 (Monitor and Respond) i MCSB v2 omfatter: establish monitoring and detection (AI-6, Defender for Cloud AI threat protection) | continuous AI red teaming (AI-7, PYRIT og Azure AI Red Teaming Agent) | threat intelligence correlation (MITRE ATLAS, OWASP Top 10 for LLM) | audit trails og compliance reporting.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#13\",\n    \"claim\": \"MCSB-kontrollen AI-5 (human-in-the-loop) krever HITL for fem scenarier: external data transfer (mandatory approval) | financial transactions over terskel (mandatory approval) | healthcare diagnosis/treatment (mandatory review) | employment decisions (mandatory review) | legal/compliance decisions (mandatory approval).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#14\",\n    \"claim\": \"Microsoft Foundry tilbyr fire risk-assessment-komponenter: AI Red Teaming Agent | Safety Evaluators | Prompt Shields | Groundedness Detection.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/default-safety-policies\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#15\",\n    \"claim\": \"AI Red Teaming Agent utfører automatisert adversarial testing mot fire risikokategorier: Violence | Hate | Sexual | Self-Harm.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/ai-red-team/ai-risk-assessment\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#16\",\n    \"claim\": \"Security Dashboard for AI inventarierer både Microsoft AI-løsninger (Microsoft 365 Copilot | Copilot Studio-agenter | Microsoft Foundry-apper og -agenter) og tredjeparts AI-modeller, -apper og -agenter (Google Gemini | OpenAI ChatGPT | MCP-servere); agent-inventaret hentes fra agenter registrert i Microsoft Agent 365 via Entra Agent Registry, mens modeller, MCP-servere og øvrige AI-apper oppdages av Microsoft Defender.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/security-for-ai/security-dashboard-for-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#17\",\n    \"claim\": \"Security Dashboard for AI henter data fra fire Microsoft Security-løsninger: Microsoft Entra (identity og access governance, conditional access for AI-apper, agent identity platform) | Microsoft Defender (AI threat protection, overvåking av agenter/workloads, sky-sikkerhetsposisjon, SaaS AI-app-risiko) | Microsoft Purview (data classification, DLP for AI, insider risk) | Security Copilot (risk exploration via prompts).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/security-for-ai/security-dashboard-for-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#18\",\n    \"claim\": \"Minimumsrollen for full innsikt i Security Dashboard for AI er Security Reader i Microsoft Entra.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/security-for-ai/security-dashboard-for-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#19\",\n    \"claim\": \"Responsible AI Dashboard i Azure Machine Learning består av fire komponenter: Error Analysis | Fairness Assessment | Model Explainability | Causal Inference.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#20\",\n    \"claim\": \"Norsk data residency for AI-arbeidslast oppnås i Azure-regionene Norway East og Norway West.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#21\",\n    \"claim\": \"Red teaming (AI Red Teaming Agent) er inkludert i Microsoft Foundry / AI Foundry uten separat lisens eller tillegg.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#22\",\n    \"claim\": \"Security Dashboard for AI krever ingen egen lisens.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/security-for-ai/security-dashboard-for-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#23\",\n    \"claim\": \"Security Dashboard for AI er avhengig av fire lisensierte produkter: Microsoft Entra ID P2 (identity risk detection) | Microsoft Defender for Cloud P2 (AI workload threat protection) | Microsoft Purview Compliance (AI-accessible data classification) | Security Copilot (lisensiert per capacity unit, AI risk exploration via prompts).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/security-for-ai/security-dashboard-for-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#24\",\n    \"claim\": \"Microsoft Entra ID P2 gir identity risk detection med risikonivåene low | medium | high.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/security-for-ai/security-dashboard-for-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/ai-risk-taxonomy-classification.md#25\",\n    \"claim\": \"MCSB v2-artikkelen Artificial Intelligence Security inneholder sikkerhetskontrollene AI-1 til AI-5 samt three-pillar-modellen.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/ai-risk-taxonomy-classification.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/algorithmic-accountability-auditability.md",
    "claim_count": 23,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/algorithmic-accountability-auditability.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/algorithmic-accountability-auditability.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#1\",\n    \"claim\": \"Microsoft definerer accountability som ett av seks kjerneprinsipper for Responsible AI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai?view=azureml-api-2#accountability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#2\",\n    \"claim\": \"Azure Machine Learning MLOps gir accountability gjennom fire kapabiliteter: Model Registry | Lineage Tracking | Event Notifications | Model Monitoring.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-management-and-deployment?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#3\",\n    \"claim\": \"Microsoft Foundry tilbyr OpenTelemetry-basert distributed tracing for generative AI-applikasjoner og agenter.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/develop/trace-agents-sdk?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#4\",\n    \"claim\": \"Observability-komponentene i Foundry består av: Application Insights (samler traces, spans og telemetri) | Trace Viewer i Foundry Portal (execution timeline, input/output-data, performance metrics, error details) | Agent Identity (Microsoft Entra Agent Identity) | Centralized Logging (Azure Log Analytics Workspace).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/develop/trace-agents-sdk?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#5\",\n    \"claim\": \"Microsoft Entra Agent Identity gir hver agent en unik identitet med ownership | versjon | lifecycle status, og skiller mellom production-, development- og test-agenter.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/governance-security-across-organization#agent-observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#6\",\n    \"claim\": \"Microsoft Purview støtter compliance management for AI-apper gjennom fire funksjoner: Audit Log for AI Activities | Activity Explorer (DSPM-dashboards) | eDiscovery & Content Search | Communication Compliance.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/purview/ai-microsoft-purview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#7\",\n    \"claim\": \"Purviews audit log for AI-aktiviteter logger prompts | responses | tidspunkt | bruker | fil-referanser | sensitivity labels.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/purview/ai-microsoft-purview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#8\",\n    \"claim\": \"App-dekning for Purview audit/compliance (2026-06): «Copilot experiences and agents» omfatter Microsoft 365 Copilot | Security Copilot | Copilot in Fabric | Copilot Studio; «Enterprise AI apps» omfatter Microsoft Foundry | Entra-registrerte AI-apper | Anthropic Claude (Enterprise) | ChatGPT Enterprise; tredjeparts-LLM-er som ChatGPT | Gemini | DeepSeek dekkes som «Other AI apps» via browser-aktivitet i Defender for Cloud Apps.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/purview/ai-microsoft-purview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#9\",\n    \"claim\": \"Microsofts tjenester for security logging og threat detection av AI: Azure Monitor (metrics, logs, traces) | Azure Policy (enforcer logging- og monitoring-konfigurasjoner) | Microsoft Sentinel (SIEM) | Defender for AI Services (monitorerer model inputs, outputs og API-interaksjoner).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-6-establish-monitoring-and-detection\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#10\",\n    \"claim\": \"Microsoft Sentinel korrelerer AI-aktivitet mot kjente attack patterns fra MITRE ATLAS og OWASP.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-6-establish-monitoring-and-detection\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#11\",\n    \"claim\": \"Microsoft Security Engineering anbefaler algoritme-nivå event tracing som for hver høyverdibeslutning logger: timeframe for siste treningsevent | timestamp for nyeste dataset entry | weights og confidence levels for key classifiers | classifiers involvert i beslutningen | final decision reached.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/engineering/securing-artificial-intelligence-machine-learning\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#12\",\n    \"claim\": \"I Foundry Portal under Tracing-fanen kan traces filtreres på trace ID | start time | duration | status.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/develop/trace-agents-sdk?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#13\",\n    \"claim\": \"Hver trace i Foundry viser execution timeline | input/output-data | performance metrics | error details | custom attributes.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/develop/trace-agents-sdk?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#14\",\n    \"claim\": \"I Microsoft Purview Audit kan man søke på AI-aktivitetene CopilotInteraction | AIServiceUsed | PromptSubmitted | ResponseGenerated.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/purview/ai-microsoft-purview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#15\",\n    \"claim\": \"Copilot Studio støtter integrasjon med Azure Application Insights for sentralisert telemetri, aktivert under Settings → Logging.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#16\",\n    \"claim\": \"Microsoft Copilot Studio har ISO-, SOC- og HIPAA-sertifiseringer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-certification\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#17\",\n    \"claim\": \"Copilot Studio støtter konfigurerbare data locations for datasuverenitet.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/data-location\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#18\",\n    \"claim\": \"AI-logger kan lagres i Norge/EU ved bruk av Azure Norway-regioner og konfigurert Purview data location.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#19\",\n    \"claim\": \"Azure Log Analytics inkluderer 31 dagers retention, med mulighet for extended retention utover dette.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#20\",\n    \"claim\": \"Microsoft 365 E5 Compliance inkluderer Purview Audit (Premium) | Activity Explorer | Communication Compliance | eDiscovery (Premium).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#21\",\n    \"claim\": \"Microsoft 365 E3 med Purview Compliance add-on gir de samme kapabilitetene som E5 Compliance.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#22\",\n    \"claim\": \"Microsoft 365 E3 uten Purview Compliance add-on gir kun basic audit log med 90 dagers retention og begrenset eDiscovery.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/algorithmic-accountability-auditability.md#23\",\n    \"claim\": \"Model Registry og Lineage Tracking i Azure Machine Learning har ingen ekstra kostnad — de er inkludert i Azure ML workspace.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/algorithmic-accountability-auditability.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/bias-detection-mitigation-strategies.md",
    "claim_count": 30,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/bias-detection-mitigation-strategies.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/bias-detection-mitigation-strategies.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#1\",\n    \"claim\": \"Responsible AI Dashboard i Azure Machine Learning integrerer seks verktøy/komponenter for model debugging og bias assessment: Model Overview & Fairness | Error Analysis | Data Analysis | Model Interpretability | Counterfactual Analysis | Causal Inference.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#2\",\n    \"claim\": \"Komponentene i Responsible AI Dashboard bygger på disse verktøyene: Model Overview & Fairness → Fairlearn | Error Analysis → Error Analysis (erroranalysis.ai) | Data Analysis → Azure ML native | Model Interpretability → InterpretML | Counterfactual Analysis → DiCE | Causal Inference → EconML.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#3\",\n    \"claim\": \"Prosessen for model debugging består av tre steg: Identify (finn feil og fairness-problemer) | Diagnose (data analysis, interpretability, counterfactuals) | Mitigate (Fairlearn-algoritmer, data rebalancing).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#4\",\n    \"claim\": \"Fairlearn måler to klasser av disparity metrics: model performance disparity (accuracy rate, error rate, precision, recall, MAE) | selection rate disparity (loan approval rate, favorable classification rate).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#5\",\n    \"claim\": \"Fairlearn støtter fire parity constraints med tilhørende ML-oppgaver: Demographic parity (binary classification, regression) | Equalized odds (binary classification) | Equal opportunity (binary classification) | Bounded group loss (regression).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#6\",\n    \"claim\": \"Fairlearns reduction-algoritmer (krever retraining) er ExponentiatedGradient (binary classification, kategoriske sensitive features, demographic parity og equalized odds) og GridSearch (binary classification/regression, binære sensitive features, demographic parity, equalized odds og bounded group loss).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#7\",\n    \"claim\": \"Fairlearns post-processing-algoritme ThresholdOptimizer justerer decision threshold per gruppe for binary classification med kategoriske sensitive features, støtter demographic parity og equalized odds, og krever ingen retraining av modellen.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#8\",\n    \"claim\": \"Hver Azure AI Content Safety-kategori rapporteres med severity-nivåene Safe | Low | Medium | High.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/content-filter-severity-levels\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#9\",\n    \"claim\": \"Azure AI Content Safety filtrerer fire kategorier: Hate & Fairness | Sexual | Violence | Self-harm.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#10\",\n    \"claim\": \"Default-konfigurasjonen i Azure AI Content Safety er Medium severity threshold (blokkerer medium og høyere) for både text models og image models — samme terskel for alle fire kategorier.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/content-filter-severity-levels\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#11\",\n    \"claim\": \"Content Safety kan tilpasses med: configurable filters via «Guardrails & controls» i AI Foundry portal | custom blocklists (ord/mønstre) | meta-prompts (systemmeldinger) | threshold-justering per kategori.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#12\",\n    \"claim\": \"For serverless API deployments som ikke bruker Model Inference API er content filtering ikke automatisk aktivert; det må implementeres separat via Azure AI Content Safety SDK.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#13\",\n    \"claim\": \"Fire fairness-metrikker brukes til sammenligning av protected group: predictive_parity (precision-forskjell) | predictive_equality (false positive rate-forskjell) | equal_opportunity (true positive rate-forskjell) | statistical_parity (selection rate-forskjell).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/data-quality-monitoring/data-profiling/fairness-bias\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#14\",\n    \"claim\": \"Azure ML-komponentregisteret inneholder RAI-fairness-komponenten `microsoft_azureml_rai_tabular_fairness`, som hentes med label «latest».\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#15\",\n    \"claim\": \"Responsible AI Dashboard kan generere en Responsible AI Scorecard som PDF for deling med stakeholders.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#16\",\n    \"claim\": \"Azure AI Content Safety Prompt Shield detekterer jailbreak-forsøk i input.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#17\",\n    \"claim\": \"Databricks tilbyr DataQualityMonitor.create for fairness-/bias-overvåkning av inference-logger med parametrene table_name, inference_log, problem_type og slicing_exprs, og beregner automatisk metrikkene predictive_parity, predictive_equality og equal_opportunity.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/data-quality-monitoring/data-profiling/fairness-bias\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#18\",\n    \"claim\": \"Responsible AI Dashboard-komponentene har SDK-metodene RAIInsights.from_model() (oppretter dashboard) | add_fairness() | add_error_analysis() | add_explainer() | add_counterfactual() | add_causal().\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#19\",\n    \"claim\": \"Ved serverless API deployment i Microsoft Foundry aktiveres default content filters automatisk, konfigurerbart via «Guardrails & controls»-fanen, med Medium threshold for text og Low for images.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/content-filter-severity-levels\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#20\",\n    \"claim\": \"Custom categories i Azure AI Content Safety er i preview og trenes via Content Safety Studio.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#21\",\n    \"claim\": \"Pakken azure.ai.evaluation tilbyr HateUnfairnessEvaluator, som tar azure_ai_project, credential og en threshold-parameter (severity threshold).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#22\",\n    \"claim\": \"AI Builder-modelltypene omfatter Form Processing | Text Classification | Prediction | Object Detection.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#23\",\n    \"claim\": \"Standard_DS3_v2 er en Azure-VM-størrelse med 4 cores, brukt som compute for dashboard-generering.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#24\",\n    \"claim\": \"Responsible AI Dashboard er inkludert i Azure ML-lisensen uten tilleggslisens, og Fairlearn (Apache 2.0), InterpretML, EconML og DiCE er alle open source.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#25\",\n    \"claim\": \"Azure AI Content Safety krever egen lisens etter pay-per-use uten base fee, og faktureres separat fra Azure OpenAI/model inference.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#26\",\n    \"claim\": \"Responsible AI-funksjoner er inkludert i standard Copilot Studio-lisens, uten per-use-kostnad for bias detection-funksjoner.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#27\",\n    \"claim\": \"Responsible AI dashboard (Azure Machine Learning) har status GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#28\",\n    \"claim\": \"Fairlearns mitigation-dokumentasjon som legges til grunn er versjon v0.7.0.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#29\",\n    \"claim\": \"Azure AI Content Safety har status GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/bias-detection-mitigation-strategies.md#30\",\n    \"claim\": \"Databricks-funksjonen for å overvåke fairness og bias (data profiling) har status GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/data-quality-monitoring/data-profiling/fairness-bias\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/bias-detection-mitigation-strategies.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md",
    "claim_count": 21,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#1\",\n    \"claim\": \"MLflow 3 GenAI introduserer en ny Assessment-datamodell med to typer: Feedback (evaluerer faktisk output, f.eks. ratings og kommentarer) | Expectation (definerer ønsket/korrekt output, ground truth, brukes til å bygge evalueringsdata).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#2\",\n    \"claim\": \"MLflow 3 GenAI-assessments har tre innsamlingskilder: utvikler (dev) | domeneekspert (via Review App) | sluttbruker (produksjon).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#3\",\n    \"claim\": \"MLflow tilbyr API-et mlflow.log_feedback() for å knytte bruker-rating og kommentarer til spesifikke traces.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#4\",\n    \"claim\": \"Genie Code er en ny kapabilitet i MLflow 3 GenAI for naturspråk-analyse av trace-data.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#5\",\n    \"claim\": \"Produksjonsdata for AI-feedback samles inn via MLflow Traces / MLflow 3 GenAI | Azure Monitor & Application Insights (operational metrics, latency, error rates) | Model Data Collector (automatisk innsamling av produksjonsdata for ML-modeller) | Azure AI Content Safety logs (content moderation events).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/observability\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#6\",\n    \"claim\": \"Microsoft bruker fem automated scorers (LLM-judges) for kontinuerlig kvalitetsvurdering av produksjonstrafikk: Groundedness | Relevance | Coherence | Fluency | Safety.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/monitor-quality-safety?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#7\",\n    \"claim\": \"Eksempel-terskler for scorers: pass rate ≥ 70 % for groundedness, relevance, coherence og fluency, og pass rate ≥ 95 % for safety.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/monitor-quality-safety?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#8\",\n    \"claim\": \"MLflow Continuous Improvement Cycle for GenAI-apper består av 10 steg: Production App | User Feedback | Monitor & Score | Identify Issues | Domain Expert Review | Build Eval Dataset | Tune Scorers | Evaluate New Versions | Compare Results | Deploy or Iterate.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/overview/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#9\",\n    \"claim\": \"AI Builder feedback loop lagrer data i Microsoft Dataverse-tabellen «AI Builder Feedback Loop».\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/ai-builder/feedback-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#10\",\n    \"claim\": \"Microsoft Foundry monitoring dashboard visualiserer metrics over tid via Charts tab og Logs tab.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/monitor-quality-safety?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#11\",\n    \"claim\": \"Konfigurasjon av kontinuerlig kvalitetsmonitorering bruker Python SDK-klassene GenerationSafetyQualitySignal | GenerationSafetyQualityMonitoringMetricThreshold | MonitorSchedule | CronTrigger fra azure.ai.ml.entities.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/monitor-quality-safety?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#12\",\n    \"claim\": \"MLflow på Azure Databricks støtter automatisk tracing via mlflow.openai.autolog() for OpenAI, LangChain m.fl.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#13\",\n    \"claim\": \"Kodeeksempelet for MLflow-tracing på Azure Databricks bruker modellen gpt-4o-mini.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#14\",\n    \"claim\": \"AI Builder feedback loop har følgende begrensninger: kun for custom document processing-modeller | feedback loop-data kun via Power Automate cloud flows | samme eier kreves for modell og flow | ingen overføring av feedback loop-data på tvers av miljøer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/ai-builder/feedback-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#15\",\n    \"claim\": \"Model monitoring for GenAI i Azure Machine Learning er i public preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-monitor-generative-ai-applications?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#16\",\n    \"claim\": \"Azure Machine Learning model monitoring for GenAI har evalueringsmetrikkene groundedness | coherence | fluency | relevance | similarity, interoperable med Prompt Flow evaluation metrics.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-monitor-generative-ai-applications?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#17\",\n    \"claim\": \"Prompt flow i Microsoft Foundry og Azure Machine Learning pensjoneres 20. april 2027 og anbefales ikke for ny utvikling; GenAI-app-monitoring bygd på Prompt flow-deployments/runtime-images må migreres til Microsoft Agent Framework (MAF) før den datoen.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-monitor-generative-ai-applications?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#18\",\n    \"claim\": \"Microsoft Foundry faktureres pay-as-you-go for monitoring, evaluering og lagring, og bruker serverless Spark compute for monitoring-schedules.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/monitor-quality-safety?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#19\",\n    \"claim\": \"AI Builder krever AI Builder credits for modelltrening og inferens.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/ai-builder/feedback-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#20\",\n    \"claim\": \"AI Builder feedback loop-funksjonen er inkludert i AI Builder-lisensieringen og har preview-status.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/ai-builder/feedback-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/continuous-improvement-feedback-loops.md#21\",\n    \"claim\": \"Monitoring-kapabiliteter er inkludert i Microsoft Copilot Studio-lisensieringen, uten separat kostnad for feedback-mekanismer.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/responsible-ai\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/data-quality-responsible-ai.md",
    "claim_count": 23,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/data-quality-responsible-ai.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/data-quality-responsible-ai.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#1\",\n    \"claim\": \"Microsoft klassifiserer kilder til treningsdata i tre typer: Proprietary data | Public sources (Wikipedia, PubMed, offentlige datasett) | User-generated data.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/ai/training-data-design\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#2\",\n    \"claim\": \"Teknikker for å balansere skjeve datasett: SMOTE (Synthetic Minority Oversampling Technique) | Undersampling av majoritetsklasser | Synthetic data generation i Microsoft Foundry.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-manage-ml-pitfalls?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#3\",\n    \"claim\": \"Azure Machine Learning Responsible AI dashboard har en Data Analysis-komponent som brukes til å identifisere kvalitetsproblemer og over-/underrepresentasjon i datasett.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-data-analysis?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#4\",\n    \"claim\": \"Data Analysis-komponenten i Responsible AI dashboard tilbyr visualiseringene aggregate plots | scatter plots | cohort-basert analyse, og kan filtrere på predicted outcome | dataset features | error groups.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-data-analysis?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#5\",\n    \"claim\": \"Microsoft dokumenterer fire nøkkelteknikker for preprosessering av treningsdata: Quality filtering | Rescoping (broadening overly specific fields) | Deduplication | Sensitive data handling.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/ai/training-data-design\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#6\",\n    \"claim\": \"Azure Machine Learning AutoML Data Guardrails omfatter Class balancing detection (status Alerted/Passed) | Memory issues detection (Done/Passed, sjekker at horizon/lag/rolling window ikke forårsaker OOM) | Frequency detection (Done/Passed, verifiserer time-series alignment).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-configure-auto-features?view=azureml-api-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#7\",\n    \"claim\": \"Delta Live Tables er offisielt omdøpt til Lakeflow Spark Declarative Pipelines.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/ldp/expectations\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#8\",\n    \"claim\": \"Azure Machine Learning integrert med Microsoft Purview gir automatisk lineage tracking for data.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/platform/governance\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#9\",\n    \"claim\": \"Microsoft angir tre trusler mot treningsdata: Malicious data injection | Target leakage | Training data tampering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/engineering/securing-artificial-intelligence-machine-learning\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#10\",\n    \"claim\": \"Azure Machine Learning Model Monitoring støtter signaler for data drift og datakvalitet (data quality signals).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#11\",\n    \"claim\": \"GPT-4 kan finjusteres (fine-tuning) for domenespesifikke oppgaver, for eksempel medisinsk dokumentasjon.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#12\",\n    \"claim\": \"Azure Machine Learning-komponenter som støtter datakvalitet: Responsible AI Dashboard (data analysis, fairness, error analysis) | AutoML Data Guardrails | Model Monitoring | ML Datasets (versjonerte, registrerte datasett).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#13\",\n    \"claim\": \"Python SDK-en azure.ai.ml.entities eksponerer klassene DataQualitySignal og DataQualityMetricThreshold (med DataQualityMetricsNumerical og DataQualityMetricsCategorical) for konfigurasjon av datakvalitetssignaler.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#14\",\n    \"claim\": \"Microsoft Foundry tilbyr Evaluation tools | Synthetic data generation | Content Safety med protected material detection.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#15\",\n    \"claim\": \"Microsoft Purview tilbyr Data discovery and classification (automated tagging) | Lineage tracking (full data provenance) | Compliance policies som håndhever GDPR/dataresidens.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/platform/governance\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#16\",\n    \"claim\": \"Azure Machine Learning Responsible AI dashboard kan generere Responsible AI Scorecard som PDF for stakeholders.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#17\",\n    \"claim\": \"Dataresidens for segmenterte pipelines i Norge dekkes av Azure-regionene Norway East og Norway West.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#18\",\n    \"claim\": \"Azure ML Feature Store er inkludert i Azure Machine Learning (ingen separat tjenestelisens).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#19\",\n    \"claim\": \"Responsible AI Dashboard er inkludert i Azure Machine Learning uten ekstra kostnad.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#20\",\n    \"claim\": \"Microsoft Purview krever separat lisens som data governance add-on.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/platform/governance\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#21\",\n    \"claim\": \"Databricks Expectations krever Databricks-lisens på Premium- eller Enterprise-tier.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/ldp/expectations\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#22\",\n    \"claim\": \"Azure Cognitive Services tilbyr PII-deteksjon som alternativ til egne preprosesseringsskript for PII-fjerning.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/data-quality-responsible-ai.md#23\",\n    \"claim\": \"Azure Databricks data expectations støtter mønstrene expect_all | expect_all_or_drop | expect_all_or_fail.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/ldp/expectations\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/data-quality-responsible-ai.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/fairness-testing-measurement.md",
    "claim_count": 23,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/fairness-testing-measurement.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/fairness-testing-measurement.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#1\",\n    \"claim\": \"Fairlearn består av komponentene Disparity metrics (sammenligner modellprestasjon mellom grupper) | Mitigation algorithms (reduction og post-processing teknikker) | Dashboard integration (visualisering i Azure ML Studio) | Parity constraints.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#2\",\n    \"claim\": \"Fairlearns parity constraints er demographic parity | equalized odds | equal opportunity | bounded group loss.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#3\",\n    \"claim\": \"Disparity i model performance måles med disparity in accuracy | disparity in error rate | disparity in precision | disparity in recall (alle for Classification) | disparity in MAE (Regression).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#4\",\n    \"claim\": \"Disparity-metrikker kan uttrykkes som ratio (max/min) eller difference (max - min).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#5\",\n    \"claim\": \"Responsible AI scorecard støtter fairness-metrikkene accuracy_score | precision_score | recall_score | f1_score | error_rate | selection_rate | mean_absolute_error | mean_squared_error, hver med fairness_evaluation_kind difference eller ratio.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-insights-sdk-cli?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#6\",\n    \"claim\": \"I Responsible AI scorecard gjelder mean_absolute_error og mean_squared_error for modelltypen Regression, mens accuracy_score, precision_score, recall_score, f1_score, error_rate og selection_rate gjelder Classification.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-insights-sdk-cli?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#7\",\n    \"claim\": \"Databricks data quality monitoring støtter fairness-metrikkene predictive_parity | predictive_equality | equal_opportunity | statistical_parity for classification-modeller.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/data-quality-monitoring/data-profiling/fairness-bias\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#8\",\n    \"claim\": \"Model Overview-komponenten i Azure ML Responsible AI dashboard er GA og genererer performance metrics for hele datasettet og identifiserte kohorter, med breakdown på sensitive features.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#9\",\n    \"claim\": \"RAI-dashboard-jobben i Azure ML bruker registry-komponenten azureml://registries/azureml/components/microsoft_azureml_rai_tabular_insight_constructor/versions/<version>.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-insights-sdk-cli?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#10\",\n    \"claim\": \"RAI insight constructor-komponenten tar inputene title | task_type | model_input (type mlflow_model) | train_dataset | test_dataset | target_column_name | categorical_column_names.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-insights-sdk-cli?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#11\",\n    \"claim\": \"Fairness-seksjonen i Responsible AI scorecard-konfigurasjonen (JSON) har feltene metric | sensitive_features | fairness_evaluation_kind | threshold.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-insights-sdk-cli?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#12\",\n    \"claim\": \"Parity constraints kobles til Fairlearn-algoritmer slik: Demographic parity (binary classification, regression) med ExponentiatedGradient og GridSearch | Equalized odds (binary classification) med ExponentiatedGradient, GridSearch og ThresholdOptimizer | Equal opportunity (binary classification) med ThresholdOptimizer | Bounded group loss (regression) med GridSearch.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#13\",\n    \"claim\": \"Fairlearns mitigation-algoritmer er ExponentiatedGradient (reduction, black-box, categorical sensitive features) | GridSearch (reduction, binary sensitive features) | ThresholdOptimizer (post-processing, categorical sensitive features).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#14\",\n    \"claim\": \"HateUnfairnessEvaluator i azure.ai.evaluation instansieres med parametrene azure_ai_project, credential og threshold.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/python/api/azure-ai-evaluation/azure.ai.evaluation.hateunfairnessevaluator?view=azure-python\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#15\",\n    \"claim\": \"I Azure Machine Learning er Responsible AI dashboard (model overview med fairness metrics), Fairlearn-integrasjonen (disparity metrics og mitigation) og logging av fairness metrics som MLflow model registry-metadata GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#16\",\n    \"claim\": \"Scorecard PDF-eksport (fairness insights for stakeholders) i Azure ML er i Preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-insights-sdk-cli?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#17\",\n    \"claim\": \"HateUnfairnessEvaluator er GA i Microsoft Foundry som content safety evaluator for generative modeller.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/python/api/azure-ai-evaluation/azure.ai.evaluation.hateunfairnessevaluator?view=azure-python\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#18\",\n    \"claim\": \"Content filtering i Microsoft Foundry er GA og tilbyr pre-trained filtre for hate | fairness | violence.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#19\",\n    \"claim\": \"Power Platform AI Builder har ingen innebygd fairness assessment-UI for custom models (per 2026-02), mens fairness testing for pre-built models er utført av Microsoft.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#20\",\n    \"claim\": \"Fairlearn er gratis open-source under MIT License.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#21\",\n    \"claim\": \"Anbefalt compute for en fairness testing-pipeline er Standard_DS3_v2 med 4 cores og 14 GB RAM.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#22\",\n    \"claim\": \"Fairness metrics i Databricks er del av Lakehouse monitoring og krever Databricks Premium- eller Enterprise-tier.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/databricks/data-quality-monitoring/data-profiling/fairness-bias\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/fairness-testing-measurement.md#23\",\n    \"claim\": \"Fairlearn støtter primært binary classification; multi-class-problemer krever One-vs-Rest kombinert med post-processing.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/fairness-testing-measurement.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/human-in-the-loop-oversight.md",
    "claim_count": 24,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/human-in-the-loop-oversight.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/human-in-the-loop-oversight.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#1\",\n    \"claim\": \"Multistage- og AI-approvals i Power Automate/Copilot Studio er i Preview, og det finnes en ny 'Human in the loop'-kobling samt conditions mellom stages for dynamisk routing.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/flows-advanced-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#2\",\n    \"claim\": \"AI-stage i Power Automate-approvals bruker GPT-o3 til å gjøre Approve/Reject med begrunnelse.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/faqs-ai-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#3\",\n    \"claim\": \"Azure Logic Apps Human Approval Connectors pauser AI-prosesser for menneskelig validering og integreres med Microsoft Teams | Outlook | egne dashboards.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-5-ensure-human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#4\",\n    \"claim\": \"Azure Durable Functions pauser agentic workflows for menneskelig beslutning via External Events med `WaitForExternalEvent` og timeout.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/agent-framework/user-guide/agents/agent-types/durable-agent/features\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#5\",\n    \"claim\": \"Godkjenningstypene i approval-flyter er: First to respond | Everyone must approve | Conditional approvals | Multistage.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/flows-advanced-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#6\",\n    \"claim\": \"Azure AI Content Safety bruker severity scores i skalaen 0-7, som kan trigge menneskelig review.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#7\",\n    \"claim\": \"Microsoft Agent Framework har approval modes for funksjoner: `never` (default) | `always_require` | `confidence_based`.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#8\",\n    \"claim\": \"Sikkerhetskravet i AI-5.1 krever kryptering av review-systemer med TLS 1.2 eller nyere.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-5-ensure-human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#9\",\n    \"claim\": \"Azure AI Security Benchmark AI-5 definerer fem kriterier som alltid krever HITL: External data transfers | Processing of confidential information | Decisions impacting financial outcomes | Safety-related commands | Compliance-critical processes.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-5-ensure-human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#10\",\n    \"claim\": \"AI-5.1 sine opplæringskrav for reviewere omfatter fem punkter: AI system behavior | Potential vulnerabilities | Domain-specific risks | Decision-support tools | Escalation procedures.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-5-ensure-human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#11\",\n    \"claim\": \"For å forhindre reviewer fatigue skal en person ikke review mer enn 50 AI-beslutninger per dag.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-5-ensure-human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#12\",\n    \"claim\": \"Microsoft Foundry sine HITL-kapabiliteter er: Prompt Shields (blokkerer prompt injection) | Content Safety med severity scores 0-7 | Model Monitoring med anomaly detection | Tracing via OpenTelemetry.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/agents/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#13\",\n    \"claim\": \"Copilot Studio sine HITL-features er: Human Handoff Topic som overfører samtalen til Live Agent (Omnichannel, Dynamics 365) | Escalation Rate Tracking i analytics dashboard | Rationale Generation | Approval Topics som pauser for menneskelig input.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/deflection-topic-escalation-analysis\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#14\",\n    \"claim\": \"Power Automate Multistage Approvals har stage-typene: AI Stage (AI gjør approve/reject) | Manual Stage (menneske beslutter) | Condition Stage (logisk routing basert på verdier).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/flows-advanced-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#15\",\n    \"claim\": \"FAQ for AI Approvals anbefaler GPT-4.1 for komplekse approval-scenarioer, og o3 for advanced reasoning men da tregere.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/faqs-ai-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#16\",\n    \"claim\": \"Microsoft Agent Framework støtter HITL i orchestration-typene: Sequential | Concurrent | Group Chat | Handoff.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#17\",\n    \"claim\": \"Microsoft Agent Framework aktiverer HITL for utvalgte agenter via API-et `with_request_info(agents=[…])` på SequentialBuilder.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#18\",\n    \"claim\": \"Agent Framework HITL støtter responstypene: Feedback (agent refinerer output) | Approval (workflow fortsetter) | Rejection (workflow stopper eller re-routes).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#19\",\n    \"claim\": \"Microsofts tilgjengelighets-features for HITL-grensesnitt: Power Automate Approvals er skjermleser-kompatibel | Azure Dashboards er WCAG 2.1 AA-compliant | Copilot Studio har keyboard navigation support.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#20\",\n    \"claim\": \"Power Automate Approvals er inkludert i Power Automate per-user/per-flow-lisens, uten ekstrakostnad for standard approvals.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/flows-advanced-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#21\",\n    \"claim\": \"AI Approvals er inkludert i Copilot Studio-lisensen (per-user-lisens + AI credits) og forbruker AI credits ved bruk.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/faqs-ai-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#22\",\n    \"claim\": \"Power Automate Premium kreves for approvals.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/flows-advanced-approvals\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#23\",\n    \"claim\": \"Microsoft Agent Framework har ingen direkte lisenskostnad (open source), men krever Azure OpenAI eller Microsoft Foundry for modellene.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/human-in-the-loop\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/human-in-the-loop-oversight.md#24\",\n    \"claim\": \"Regelmessig testing av HITL-robusthet gjøres med PYRIT og Azure AI Red Teaming Agent.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-5-ensure-human-in-the-loop\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/human-in-the-loop-oversight.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/model-explainability-interpretability.md",
    "claim_count": 23,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/model-explainability-interpretability.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/model-explainability-interpretability.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#1\",\n    \"claim\": \"Azure Machine Learning sine model interpretability-/Responsible AI-kapabiliteter har status GA (generelt tilgjengelig).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#2\",\n    \"claim\": \"XAI-teknikkene i Azure Machine Learning omfatter SHAP Tree Explainer | Mimic Explainer (Global Surrogate) | SHAP Text | SHAP Vision | Guided Backprop | Guided GradCAM | Integrated Gradients | XRAI | D-RISE | Permutation Feature Importance.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#3\",\n    \"claim\": \"Fire av XAI-teknikkene er AutoML-spesifikke for bildedata: Guided Backprop | Guided GradCAM | Integrated Gradients | XRAI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#4\",\n    \"claim\": \"D-RISE er model-agnostic og gir object detection-forklaringer for YOLO | Faster-RCNN | ViT.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#5\",\n    \"claim\": \"Permutation Feature Importance er tilgjengelig for tabulære data i .NET (ML.NET) og måler impact ved feature shuffling.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/explain-machine-learning-model-permutation-feature-importance-ml-net\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#6\",\n    \"claim\": \"Responsible AI dashboard støtter tre forklaringsnivåer: Global explanations (aggregert feature importance på tvers av alle prediksjoner) | Local explanations (feature importance for enkeltprediksjoner) | Cohort explanations (feature importance for subgrupper).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#7\",\n    \"claim\": \"Responsible AI dashboard i Azure ML bruker LightGBM (LGBMExplainableModel) som global surrogate, bruker SHAP Tree Explainer for tree ensembles, og støtter både classification og regression på tabulære data.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#8\",\n    \"claim\": \"Responsible AI dashboard beskrives som seks komponenter, men listen inneholder syv: Error Analysis | Fairness | Model Overview | Data Analysis | Interpretability | Counterfactual Analysis | Causal Inference.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#9\",\n    \"claim\": \"Responsible AI dashboard er begrenset til maks 5000 datapunkter i UI-visualiseringen, og kun tabulære data / pandas DataFrame i Parquet-format.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#10\",\n    \"claim\": \"Responsible AI dashboard støtter kun MLflow sklearn-modeller som har predict() eller predict_proba().\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#11\",\n    \"claim\": \"Explainable Boosting Machines (EBM) fra InterpretML er begrenset til tabulære data.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/fabric/data-science/explainable-boosting-machines-classification\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#12\",\n    \"claim\": \"Interpretability-komponenten i Responsible AI dashboard kan genereres programmatisk via Azure ML SDK v2 / CLI v2.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#13\",\n    \"claim\": \"Generering av forklaringer i Responsible AI dashboard krever compute cluster, ikke serverless compute.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#14\",\n    \"claim\": \"Datasett i Responsible AI dashboard kan ha maks 10 000 features (kolonner).\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#15\",\n    \"claim\": \"Microsoft Fabric (Synapse ML) tilbyr TabularSHAP for distribuert SHAP-beregning på Spark | Explainable Boosting Machines som glass-box-modeller med built-in explanations | MLflow tracking av feature importance-plott som artifacts | Lakehouse-lagring av SHAP-verdier som Delta tables.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/fabric/data-science/tabular-shap-explainer\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#16\",\n    \"claim\": \"Prompt Flow i Microsoft Foundry (Azure OpenAI) tilbyr evaluators for groundedness og relevance som GPT-assisterte metrikker.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#17\",\n    \"claim\": \"Azure OpenAI transparency note angir at reasoning-modeller (o1) har intrinsic chain-of-thought men økt persuasiveness-/scheming-risiko, og at OpenAI fine-tunede modeller har redusert explainability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note?view=foundry-classic#limitations\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#18\",\n    \"claim\": \"Power Platform AI Builder gir prediction confidence scores, men ikke feature-level explanations.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#19\",\n    \"claim\": \"Azure ML kan kjøre SHAP-beregninger i Norge-regionene Norway East og Norway West.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#20\",\n    \"claim\": \"Explanation-generering i Responsible AI dashboard kjøres på compute cluster, for eksempel VM-SKU Standard_DS3_v2.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#21\",\n    \"claim\": \"Microsoft Fabric F64-kapasitet tilsvarer 64 Capacity Units (CU), og TabularSHAP kjører på Spark compute i denne kapasiteten.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/fabric/data-science/tabular-shap-explainer\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#22\",\n    \"claim\": \"Lisensmodellene for XAI-funksjonalitet er: Azure Machine Learning = pay-as-you-go (compute + storage, gir Responsible AI Dashboard, InterpretML, SHAP) | Microsoft Fabric = capacity-based CU per måned (gir TabularSHAP i Synapse ML og EBM i InterpretML) | Power BI Premium = per user eller per capacity (kan visualisere SHAP-data fra Fabric/AML) | Microsoft Foundry = pay-per-token (gir GPT-assisterte evaluators).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-explainability-interpretability.md#23\",\n    \"claim\": \"Azure ML free tier gir 4 timer compute per måned og kan brukes til utvikling av XAI.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/model-explainability-interpretability.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/model-monitoring-drift-detection.md",
    "claim_count": 30,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/model-monitoring-drift-detection.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/model-monitoring-drift-detection.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#1\",\n    \"claim\": \"Azure Machine Learning model monitoring er GA (Generally Available) for tabular data, med støtte for både online endpoints og batch/external deployments.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#2\",\n    \"claim\": \"Azure ML model monitoring skiller mellom fem kjernetyper av drift: Data drift | Concept drift | Prediction drift | Data quality drift | Feature attribution drift.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/ai/test#guidance-for-testing-model-training-and-fine-tuning\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#3\",\n    \"claim\": \"Azure Machine Learning tilbyr seks innebygde monitoring-signaler for tabular data: Data drift | Prediction drift | Data quality | Feature attribution drift | Model performance | Generation safety/quality.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#4\",\n    \"claim\": \"Data drift-signalet i Azure ML bruker metrikkene Jensen-Shannon Distance | Population Stability Index | Normalized Wasserstein Distance | Two-Sample Kolmogorov-Smirnov Test | Pearson's Chi-Squared Test, med model inputs som produksjonsdata og training data eller nylig produksjonsdata som referansedata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#5\",\n    \"claim\": \"Prediction drift-signalet bruker metrikkene Jensen-Shannon Distance | Population Stability Index | Normalized Wasserstein Distance | Chebyshev Distance | Two-Sample Kolmogorov-Smirnov Test | Pearson's Chi-Squared Test, med model outputs som produksjonsdata og validation data eller nylig produksjonsdata som referansedata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#6\",\n    \"claim\": \"Data quality-signalet bruker metrikkene Null value rate | Data type error rate | Out-of-bounds rate, med model inputs som produksjonsdata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#7\",\n    \"claim\": \"Signalene Feature attribution drift, Model performance og Generation safety/quality er merket preview i Azure ML model monitoring.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#8\",\n    \"claim\": \"Feature attribution drift-signalet bruker metrikken normalized discounted cumulative gain (NDCG) og krever training data som referansedata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#9\",\n    \"claim\": \"Model performance-signalet bruker metrikkene Accuracy | Precision | Recall for classification og MAE | MSE | RMSE for regression, og krever ground truth data som referansedata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#10\",\n    \"claim\": \"Generation safety/quality-signalet bruker metrikkene Groundedness | Relevance | Fluency | Similarity | Coherence, gjelder generativ AI (Q&A) og bruker annotation template som referansedata.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#11\",\n    \"claim\": \"Azure ML støtter opptil 0.00001 precision for data quality-beregninger.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#12\",\n    \"claim\": \"Data type error rate støtter PySpark-typene ShortType | BooleanType | BinaryType | DoubleType | TimestampType | StringType | IntegerType | FloatType | ByteType | LongType | DateType.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#13\",\n    \"claim\": \"Out-of-bounds rate støtter typene StringType | IntegerType | DoubleType | ByteType | LongType | FloatType.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#14\",\n    \"claim\": \"Serverless Spark compute for Azure ML model monitoring konfigureres med runtime_version 3.3.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#15\",\n    \"claim\": \"Custom signal-komponentens signal_metrics-output er en mltable med skjemaet group | metric_name | metric_value | threshold_value.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#16\",\n    \"claim\": \"Preprocessing-komponenten for external/batch deployments har inputene data_window_start (literal string, ISO8601) | data_window_end (literal string, ISO8601) | input_data (uri_folder), og outputen preprocessed_data (mltable).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#17\",\n    \"claim\": \"For Event Grid-integrasjon med model monitoring velges event-typen Run status changed; event-typen Dataset drift detected tilhører Azure ML v1.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#18\",\n    \"claim\": \"Event Grid advanced filter for threshold breach bruker nøkkelen data.RunTags.azureml_modelmonitor_threshold_breached med operatoren String contains.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#19\",\n    \"claim\": \"Event handlers for model monitoring-events er Azure Event Hubs | Azure Functions | Azure Logic Apps.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#20\",\n    \"claim\": \"Monitoring metrics sendes til Azure Blob Storage i JSON-format, med Application Insights for custom alerting på alle metrics og Azure Monitor Metrics for performance-visualisering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#21\",\n    \"claim\": \"Monitoring jobs kjører på Serverless Spark compute med støttede størrelser standard_e4s_v3 | standard_e8s_v3 | standard_e16s_v3 | standard_e32s_v3 | standard_e64s_v3.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#22\",\n    \"claim\": \"Azure ML model monitoring støtter IKKE AllowOnlyApprovedOutbound managed VNet isolation.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#23\",\n    \"claim\": \"Kun basic MLTable har garantert support i Azure ML model monitoring.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#24\",\n    \"claim\": \"Azure ML model monitoring støtter to autentiseringsmetoder mot datastore: credential-based | credential-less med User-Assigned Managed Identity (systemDatastoresAuthMode=identity).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#25\",\n    \"claim\": \"Serverless Spark-størrelsene har spesifikasjonene standard_e4s_v3 4 vCPU/32 GB | standard_e8s_v3 8 vCPU/64 GB | standard_e16s_v3 16 vCPU/128 GB | standard_e32s_v3 32 vCPU/256 GB | standard_e64s_v3 64 vCPU/512 GB.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#26\",\n    \"claim\": \"Model monitoring er inkludert i Azure ML uten ekstra lisens eller kostnad utover compute; Azure ML workspace krever kun Azure subscription.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#27\",\n    \"claim\": \"Serverless Spark for monitoring faktureres pay-per-use og belastes per vCPU-time.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-monitoring?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#28\",\n    \"claim\": \"Event Grid gir de første 100 000 operasjonene per måned gratis.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#29\",\n    \"claim\": \"Data drift (preview) i Azure ML v1 blir retired og erstattes av Model Monitor.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-datasets?view=azureml-api-1\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/model-monitoring-drift-detection.md#30\",\n    \"claim\": \"Python SDK-pakkene for model monitoring er azureml-datadrift (v1) og azure-ai-ml (v2).\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance?view=azureml-api-2\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/model-monitoring-drift-detection.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/red-teaming-ai-models.md",
    "claim_count": 24,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/red-teaming-ai-models.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/red-teaming-ai-models.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#1\",\n    \"claim\": \"Microsoft Security Benchmark-kontrollen AI-7 definerer kontinuerlig AI red teaming som obligatorisk best practice.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-7-perform-continuous-ai-red-teaming\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#2\",\n    \"claim\": \"PyRIT (Python Risk Identification Tool for generative AI) består av komponentene Prompt Executors | Datasets | Converters | Scorers | Memory | Targets.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/red-teaming\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#3\",\n    \"claim\": \"PyRIT Targets støtter integrasjoner mot Azure OpenAI | Hugging Face | REST APIs | lokale modeller.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/red-teaming\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#4\",\n    \"claim\": \"PyRIT med Foundry-integrasjon installeres via pakke-extraet azure-ai-evaluation[redteam] (Azure AI Evaluation SDK).\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#5\",\n    \"claim\": \"Azure AI Red Teaming Agent er en managed service i Microsoft Foundry som kombinerer PyRIT med Risk and Safety Evaluations, og er i preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#6\",\n    \"claim\": \"AI Red Teaming Agent har en tre-faset tilnærming: Automated scans for content risks | Evaluate probing success (Attack Success Rate) | Reporting and logging i Foundry.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#7\",\n    \"claim\": \"AI Red Teaming Agent har to deployment-modeller: Local red teaming (model-only testing, minimal sandboxing på klientsiden) | Cloud red teaming (agent-testing med agentic risks, purple environment med transiente kjøringer og mock tools).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#8\",\n    \"claim\": \"AI Red Teaming Agent har regionstøtte (per feb 2026) i East US2 | Sweden Central | France Central | Switzerland West.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#9\",\n    \"claim\": \"AI Red Teaming Agent er en preview-feature uten SLA og er ikke anbefalt for produksjonsarbeidslaster.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#10\",\n    \"claim\": \"Støttede risikokategorier for AI Red Teaming Agent: Hateful and Unfair Content | Sexual Content | Violent Content | Self-Harm-Related Content | Protected Materials | Code Vulnerability | Ungrounded Attributes | Prohibited Actions | Sensitive Data Leakage | Task Adherence | Indirect Prompt Injection (XPIA).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#11\",\n    \"claim\": \"Risikokategoriene Prohibited Actions | Sensitive Data Leakage | Task Adherence | Indirect Prompt Injection (XPIA) støttes kun for agenter og kun i cloud red teaming.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#12\",\n    \"claim\": \"PyRIT tilbyr 20+ attack strategies for å omgå safety alignments.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#13\",\n    \"claim\": \"Encoding-baserte attack strategies i PyRIT: Base64 | Binary | Morse | ROT13 | Atbash | Caesar | Url | UnicodeConfusable | UnicodeSubstitution | Diacritic.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#14\",\n    \"claim\": \"Obfuscation-teknikker i PyRIT: CharacterSpace | CharSwap | Flip | Leetspeak | StringJoin | AsciiArt | AsciiSmuggler | AnsiAttack.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#15\",\n    \"claim\": \"Adversarial prompting-strategier i PyRIT: Jailbreak (direct UPIA) | Indirect Jailbreak (XPIA via tool outputs) | SuffixAppend | Tense transformation.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#16\",\n    \"claim\": \"Multi-turn-strategier i PyRIT: Multi-turn (kontekstakkumulering over flere turer) | Crescendo (gradvis eskalering av kompleksitet/risiko).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#17\",\n    \"claim\": \"Microsofts anbefalte red teaming-verktøy per fase: Design → AI Red Teaming Agent (cloud) | Development → PyRIT (local) med CI/CD-integrasjon | Pre-deployment → AI Red Teaming Agent (cloud) | Post-deployment → AI Red Teaming Agent (cloud) + Azure Monitor med månedlig/kvartalsvis frekvens.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security#ai-7-perform-continuous-ai-red-teaming\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#18\",\n    \"claim\": \"Purple environment for agentic cloud red teaming: transient runs (agent-state logges ikke av Foundry Agent Service, chat completions lagres ikke) | mock tools med syntetiske data | sandboxede prohibited actions | redacted inputs i utviklersynlige resultater.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#19\",\n    \"claim\": \"Microsofts spotlighting-teknikker mot prompt injection: Delimiting (skille brukerinput fra systeminstruksjoner med spesielle tokens) | Data marking (eksplisitt merking av upålitelige data) | Encoding (encode upålitelige data før prosessering).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/red-teaming\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#20\",\n    \"claim\": \"AI Red Teaming Agent støtter Foundry-hostede prompt agents og container agents samt Azure tool calls, men støtter ikke Foundry workflow agents eller function tool calls.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#21\",\n    \"claim\": \"Prompt Shields i Azure AI Content Safety dekker to angrepstyper: User Prompt Attacks (UPIA) — direkte jailbreak-forsøk som role-play, encoding og regelendringer | Indirect Prompt Attacks (XPIA) — ondsinnede instruksjoner i eksterne datakilder.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#22\",\n    \"claim\": \"PyRIT-rammeverket er lisensiert under MIT License og er gratis å bruke.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/red-teaming\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#23\",\n    \"claim\": \"Minimum lisenskrav: Microsoft Foundry krever Azure subscription (Pay-As-You-Go eller Enterprise Agreement) | Azure OpenAI Service krever Azure subscription + godkjent søknad | Azure AI Content Safety er inkludert i Azure AI Services (pay-per-transaction) | PyRIT krever ingen lisens (MIT open-source).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/concepts/ai-red-teaming-agent\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/red-teaming-ai-models.md#24\",\n    \"claim\": \"Microsoft AI Red Team Training Series består av 10 episoder: Episode 1-2 Fundamentals | Episode 3-6 Attack techniques (direkte/indirekte prompt injection, single/multi-turn) | Episode 7 Defense strategies (Spotlighting, Prompt Shields) | Episode 8-10 Automation with PyRIT.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/ai-red-team/training\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/red-teaming-ai-models.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/responsible-ai-framework-overview.md",
    "claim_count": 24,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/responsible-ai-framework-overview.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/responsible-ai-framework-overview.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#1\",\n    \"claim\": \"Microsoft Responsible AI Framework bygger på seks kjerneprinsipper: fairness | reliability and safety | privacy and security | inclusiveness | transparency | accountability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#2\",\n    \"claim\": \"Microsoft Responsible AI Standard foreligger i versjon v2, som detaljerer hvordan de seks prinsippene integreres i engineering-team, AI-livssyklusen og verktøy.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#3\",\n    \"claim\": \"Azure ML-verktøy knyttet til prinsippet privacy and security er: Azure ML security config | SmartNoise (differential privacy) | Counterfit (adversarial testing).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#4\",\n    \"claim\": \"Responsible AI Standard dekker seks domener og 14 mål (goals) for å redusere AI-risiko og skade, der hvert mål består av requirements.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#5\",\n    \"claim\": \"Microsofts AI-utviklingslivssyklus alignes med NIST AI Risk Management Framework og består av fire kjernefaser i en iterativ loop: Govern | Map | Measure | Manage.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#6\",\n    \"claim\": \"Govern-fasen støttes av: RAI Standard | Responsible AI Council (lederskap) | Office of Responsible AI (ORA) | Product Terms.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#7\",\n    \"claim\": \"Measure-fasen evaluerer risiko med de definerte metrikkene groundedness | relevance | content safety | harmful content likelihood, via Azure AI Studio safety evaluations og adversarial test datasets.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#8\",\n    \"claim\": \"Manage-fasens mitigations omfatter Prompt Shield (jailbreak defense) | Content Credentials (provenance) | MLOps monitoring, med model-level-tiltak (fine-tuning, content filters) og app-level-tiltak (grounding, UI design, disclosures).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#9\",\n    \"claim\": \"Responsible AI Dashboard brukes til model debugging for ML-modeller for klassifikasjon/regresjon på tabulære data, med primærkomponentene Error Analysis | Fairness Assessment | Model Overview.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#10\",\n    \"claim\": \"Responsible AI Dashboard har kun delvis støtte for generative AI-modeller (tekst/bilde); for tekst og bilde henvises det til Responsible AI Toolbox (åpen kildekode).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#11\",\n    \"claim\": \"Microsoft Foundry / Azure AI Studio tilbyr RAI-kapabilitetene Safety Evaluations (groundedness, relevance, content safety) | Adversarial Test Datasets | Prompt Shield | Content Safety Service (sanntidsfiltrering av tekst/bilde).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#12\",\n    \"claim\": \"Azure Machine Learning Responsible AI Dashboard samler error analysis | fairness | interpretability | counterfactuals | causal inference i én UI, og RAI Scorecard eksporterer PDF-rapport med model health insights.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#13\",\n    \"claim\": \"Microsoft 365 Copilot grounder svar med RAG mot Microsoft Graph-kilder: e-post | dokumenter | chat, og kun data brukeren har tilgang til.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#14\",\n    \"claim\": \"Microsoft 365 Copilot-interaksjoner lagres i brukerens Exchange Online-mailbox og styres av Purview retention policies.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#15\",\n    \"claim\": \"Microsoft 365 Copilot bruker ikke kundedata til å trene foundation models, per Microsoft Product Terms.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#16\",\n    \"claim\": \"Power Platform AI dekker RAI gjennom AI Builder (fairness-vurderinger for AI Builder-modeller) | Power Automate (RAI-vurderinger for workflows med AI-komponenter) | drift-/performance-tracking via Power Platform Admin Center.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#17\",\n    \"claim\": \"Microsofts compliance-verktøy for RAI er: Microsoft Purview (dataklassifisering, governance, eDiscovery for AI-assets) | Service Trust Portal (compliance-dokumentasjon, ISO 42001-sertifikat, audit-rapporter) | Compliance Manager (vurdering mot GDPR, AI Act, NIST AI RMF).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#18\",\n    \"claim\": \"Azure Machine Learning Responsible AI Dashboard krever ingen separat lisens — den er inkludert i Azure ML workspace-kostnadene, og komponentene kjører på Azure ML compute instances (CPU/GPU).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#19\",\n    \"claim\": \"Microsoft 365 Copilot krever Microsoft 365 E3 eller E5 pluss en Copilot-lisens; RAI-funksjonene (grounding, safety filters, zero data retention, Purview governance) er inkludert.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#20\",\n    \"claim\": \"Copilot Studio lisensieres per-user eller per-session (message capacity), og RAI-funksjonene Content Safety og plugin governance er inkludert.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#21\",\n    \"claim\": \"Følgende RAI-verktøy er åpen kildekode og gratis, og kan kjøres lokalt eller i Azure ML: Fairlearn | InterpretML | DiCE | EconML | SmartNoise | Counterfit.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#22\",\n    \"claim\": \"For data residency i norsk offentlig sektor brukes Azure Norway-regionene Norway East og Norway West.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#23\",\n    \"claim\": \"Azure OpenAI har zero data retention som standardinnstilling (bør verifiseres i Product Terms).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-framework-overview.md#24\",\n    \"claim\": \"Microsoft har ISO/IEC 42001-sertifikat (AI management system-standard) for Microsoft 365.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/assurance/assurance-artificial-intelligence\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/responsible-ai-framework-overview.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/responsible-ai-policy-development.md",
    "claim_count": 19,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/responsible-ai-policy-development.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/responsible-ai-policy-development.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#1\",\n    \"claim\": \"Microsoft Responsible AI Standard dekker seks domener med 14 mål som skal redusere AI-risiko og tilhørende skader.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/strategy#develop-a-responsible-ai-strategy\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#2\",\n    \"claim\": \"Responsible AI-prinsippene som organisatoriske AI-policyer skal bygge på er seks: Accountability | Transparency | Fairness | Reliability & Safety | Privacy & Security | Inclusiveness.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/ai/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#3\",\n    \"claim\": \"Privacy & Security-domenet i Microsoft Responsible AI Standard krever at team implementerer differential privacy | data minimization | secure model deployment.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/ai/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#4\",\n    \"claim\": \"Governance-strukturen for policy-enforcement har fire nøkkelroller: AI Center of Excellence (CoE) som sentraliserer governance-ansvar og definerer standarder | Research Team som utfører risk discovery | Policy Team som utvikler workload-spesifikke policyer | Engineering Team som implementerer policyer og tester for adherence.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/ai-agents/responsible-ai-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#5\",\n    \"claim\": \"Microsofts Office of Responsible AI (ORA) setter company-wide interne policyer | definerer governance-strukturer | tilbyr ressurser for AI-praksisadopsjon | reviewer sensitive use cases | hjelper forme offentlig policy rundt AI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/compliance/assurance/assurance-artificial-intelligence\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#6\",\n    \"claim\": \"En komplett Responsible AI-policy dekker seks policy-områder: Model Selection & Onboarding | Third-party Tools & Data | Model Maintenance & Monitoring | Regulatory Compliance | User Conduct | Integration & Lifecycle.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/ai-agents/responsible-ai-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#7\",\n    \"claim\": \"Microsoft Enterprise AI Services Code of Conduct definerer mandatory requirements for alle applikasjoner bygget med Microsoft AI Services: fraud detection | input/output-kontroller | AI disclosure | watermarking for video | testing | feedback-kanaler | human oversight.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/legal/ai-code-of-conduct\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#8\",\n    \"claim\": \"Microsoft-verktøy for ansvarlig AI: Responsible AI Dashboard (Azure ML) — fairness assessment, bias detection, model explainability | Microsoft Foundry evaluation tools — safety assessment, hallucination detection, bias før deployment | Azure AI Content Safety — filtrering av skadelig tekst/bilde | PYRIT (Python Risk Identification Toolkit) — red teaming for adversarielle scenarier.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/ai/responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#9\",\n    \"claim\": \"Azure Policy-initiativer for AI omfatter: Azure OpenAI Guardrails-initiativ | Azure Machine Learning ML guardrails | Azure AI Search Cognitive Services guardrails | Azure AI Bot Service Bot guardrails.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/platform/governance\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#10\",\n    \"claim\": \"Azure AI Content Safety støtter filtrering av skadelig innhold for tekst, bilde og multimodalt innhold, med konfigurerbare severity thresholds.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ai-services/content-safety/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#11\",\n    \"claim\": \"Microsoft Foundry evaluation SDK dekker pre-deployment safety-, hallusinasjons- og bias-testing og kan integreres i CI/CD.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ai-studio/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#12\",\n    \"claim\": \"Azure ML Model Registry gir versioning | approval workflows | provenance tracking, og er RBAC-kontrollert.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-model-management-and-deployment\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#13\",\n    \"claim\": \"Foundry Agent Service metrics — overvåking av model drift, performance degradation og kvalitetsmetrikker med alert rules — er i Preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/observability/how-to/how-to-monitor-agents-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#14\",\n    \"claim\": \"Microsoft Purview-integrasjon gir data governance for AI-workloads med data lineage | sensitivity labels | DLP-policyer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/purview/ai-azure-services\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#15\",\n    \"claim\": \"Copilot Studios governance-funksjoner: data location-kontroller som respekterer datasuverenitetskrav | compliance-sertifiseringer ISO, SOC, HIPAA | analytics dashboard som overvåker token-bruk og identifiserer høykost-skills | dokumenterte best practices for sikkerhet og governance.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/microsoft-copilot-studio/guidance/sec-gov-intro\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#16\",\n    \"claim\": \"Microsoft Purviews AI-governance-kapabiliteter: Compliance Manager som oversetter reguleringer (f.eks. EU AI Act) til kontroller og vurderer compliance-posture | Purview APIs for compliance-automatisering i agent-workflows | dataklassifisering med sensitivity labels og data loss prevention | unified governance som katalogiserer AI-relaterte dataressurser.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/purview/ai-azure-services\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#17\",\n    \"claim\": \"Lisensmodeller for Microsoft governance-verktøy: Azure Policy inkludert i Azure subscription | Microsoft Purview per GB data + per bruker | Microsoft Foundry pay-as-you-go (compute, storage, API-kall) | Copilot Studio per bruker/måned eller per sesjon | Azure Monitor per GB ingested + retention | Microsoft Defender for Cloud per ressurs.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-model-management-and-deployment\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#18\",\n    \"claim\": \"Microsoft Purview har en gratis tier som kan brukes som utgangspunkt sammen med innebygd Azure Policy.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-model-management-and-deployment\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-policy-development.md#19\",\n    \"claim\": \"Reservations for Azure compute til AI-workloads gir besparelser på opptil 72 %.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-model-management-and-deployment\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/responsible-ai-policy-development.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/responsible-ai-training-awareness.md",
    "claim_count": 17,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/responsible-ai-training-awareness.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/responsible-ai-training-awareness.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#1\",\n    \"claim\": \"Microsoft definerer et trelags opplæringsopplegg for Responsible AI: AI Awareness (alle ansatte) | AI Literacy (knowledge workers, produkteiere) | AI Competency (utviklere, data scientists, arkitekter).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/plan#acquire-ai-skills\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#2\",\n    \"claim\": \"Nivået AI Competency (utviklere, data scientists, arkitekter) dekkes av sertifiseringskursene AI-900 | AI-102 | DP-100.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/plan#acquire-ai-skills\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#3\",\n    \"claim\": \"Microsoft Learn tilbyr tre strukturerte learning paths for Responsible AI: Embrace Responsible AI Principles and Practices | Apply Responsible AI Principles in Learning Environments | Implement a Responsible Generative AI Solution in Microsoft Foundry.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/modules/embrace-responsible-ai-principles-practices/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#4\",\n    \"claim\": \"Microsoft Learn-modulen på training/modules/responsible-ai-studio/ heter nå «Implement a Responsible Generative AI Solution in Microsoft Foundry» — modulnavnet er endret fra «Azure AI Studio» til «Microsoft Foundry».\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/modules/responsible-ai-studio/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#5\",\n    \"claim\": \"Microsofts Maturity Model for Cognitive Business krever på Level 400 at opplæring i cognitive business vedlikeholdes for staff, management og ledelsen, for å sikre forståelse av etikk, compliance og best practice.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/microsoft-365/community/microsoft365-maturity-model--cognitive-business\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#6\",\n    \"claim\": \"AI-900 (Azure AI Fundamentals) har ingen formell utløpsdato.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/credentials/certifications/azure-ai-fundamentals/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#7\",\n    \"claim\": \"AI-102 (Azure AI Engineer Associate) har 12-måneders fornyelsesfrekvens og pensjoneres 30. juni 2026 — sertifiseringen kan ikke oppnås eller fornyes etter denne datoen.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/credentials/certifications/azure-ai-engineer/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#8\",\n    \"claim\": \"Azure Data Scientist Associate er en gjeldende Microsoft-sertifisering (credentials/certifications/azure-data-scientist) som anbefales som minimumsbaseline for data scientists.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/credentials/certifications/azure-data-scientist/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#9\",\n    \"claim\": \"Microsoft Learn-modulen «Embrace Responsible AI Principles and Practices» består av 9 units og tar ca. 1 time.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/modules/embrace-responsible-ai-principles-practices/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#10\",\n    \"claim\": \"Microsoft Learn-modulen «AI Fluency: Explore Responsible AI» består av 7 units og er på beginner-nivå.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/modules/responsible-ai/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#11\",\n    \"claim\": \"Microsoft Learn-modulen «Implement a Responsible Generative AI Solution in Microsoft Foundry» består av 9 units og er på intermediate-nivå.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/modules/responsible-ai-studio/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#12\",\n    \"claim\": \"Microsoft Foundry inkluderer de innebygde Responsible AI-verktøyene Content Safety | Prompt Shields | Groundedness evaluation.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/plan\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#13\",\n    \"claim\": \"Microsoft Learn (self-paced) er gratis og krever kun en gratis Microsoft-konto som lisens.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/plan\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#14\",\n    \"claim\": \"Microsoft Learn Educator Program er gratis for institusjoner og krever institusjonsavtale, med løpende tilgang.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/training/educator-center/\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#15\",\n    \"claim\": \"Viva Learning (basic) krever Microsoft 365 E3/E5 eller Business Premium og er inkludert i disse lisensene.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/viva/organizational-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#16\",\n    \"claim\": \"Viva Learning premium connectors krever Viva Suite eller en separat Viva Learning-lisens, og trengs for å integrere eksterne LMS som LinkedIn Learning og Coursera.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/viva/organizational-data\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/responsible-ai-training-awareness.md#17\",\n    \"claim\": \"Maturity Model for Microsoft 365 — AI & Cognitive Business Competency definerer opplæringskrav på nivåene 200–500.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/microsoft-365/community/microsoft365-maturity-model--cognitive-business\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/responsible-ai-training-awareness.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/stakeholder-communication-ai-decisions.md",
    "claim_count": 22,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/stakeholder-communication-ai-decisions.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/stakeholder-communication-ai-decisions.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#1\",\n    \"claim\": \"Responsible AI Scorecard i Azure Machine Learning har disse konfigurerbare elementene: Dataset-helse (statistikk, distribusjoner, bias-indikatorer) | Modell-ytelse (accuracy, error rates, fairness metrics) | Target values (sammenligning mot ønskede ytelsesmål) | Fortolkningsevne (global/lokal feature importance) | Fairness assessment (ytelsesforskjeller på tvers av sensitive grupper)\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#2\",\n    \"claim\": \"Model Interpretability gir tre nivåer av forklaring med tilhørende Microsoft-verktøy: Global explanations (Azure ML Interpretability component) | Local explanations (Counterfactual What-If) | Cohort explanations (Responsible AI Dashboard)\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#3\",\n    \"claim\": \"Microsofts Transparency Notes inneholder: hva systemet gjør og ikke gjør | hvordan teknologien fungerer (high-level) | valg som påvirker systemprestasjon | kjente begrensninger og edge cases | Responsible AI-prinsipper i praksis\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#4\",\n    \"claim\": \"Microsofts governance-struktur for AI i organisasjoner består av et AI Governance Board (executive sponsorship) over tre enheter: AI Center of Excellence | Cross-Functional Governance Team | Incident Response Team\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/responsible-ai-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#5\",\n    \"claim\": \"Azure-verktøyene for feedback-loop med stakeholder-input er: Azure AI Tracing (OpenTelemetry for å koble feedback til spesifikk inference) | Application Insights (sentral logging av user feedback) | Azure ML Run History (arkivering av modellmetrikker før/etter forbedringer)\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/ai/design-methodology\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#6\",\n    \"claim\": \"Azure Machine Learning tilbyr disse Responsible AI-verktøyene: Responsible AI Dashboard (end-to-end model assessment) | Responsible AI Scorecard (PDF-eksport av dashboard-innsikt) | Model Interpretability | Fairness Assessment (group-based performance metrics) | Error Analysis (cohort-based error distribution) | Causal Inference (what-if-analyse for counterfactuals)\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#7\",\n    \"claim\": \"Model Interpretability i Azure Machine Learning er SHAP-basert feature importance, både global og lokal\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-interpretability?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#8\",\n    \"claim\": \"Microsofts løsninger for stakeholder-kommunikasjon om generativ AI i Microsoft Foundry er: Retrieval-Augmented Generation (RAG) | Metaprompt engineering | Content filters | Groundedness evaluation | Transparency Note for Azure OpenAI\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note?view=foundry-classic\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#9\",\n    \"claim\": \"Copilot Studio har disse stakeholder-kommunikasjonsfunksjonene: agent observability med unik identitet per agent (owner, version, lifecycle status) | centralized logging av nøkkelhendelser til Azure Log Analytics | cost tracking av token consumption og compute usage per agent | user disclosure der agenter identifiserer seg som AI og ikke menneske\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/governance-security-across-organization\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#10\",\n    \"claim\": \"AI Builder har transparensfunksjoner: automatiske confidence scores på predictions | innebygd explainability som viser hvilke felt som påvirket beslutningen\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#11\",\n    \"claim\": \"Power Platform CoE Starter Kit gir governance-kapabiliteter: inventory av alle AI Builder-modeller i tenant | compliance checks (er modellen i prod uten review?) | automatiske varsler til governance team ved high-risk deployments\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#12\",\n    \"claim\": \"Responsible AI Scorecard er inkludert i Azure ML Workspace uten ekstra kostnad, og PDF-eksporten genereres gratis fra dashboard-data\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#13\",\n    \"claim\": \"Azure OpenAI-modellen som brukes i kostnadseksempelet for chatbot er GPT-4\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#14\",\n    \"claim\": \"Copilot Studio lisensieres per tenant per måned og inkluderer 25 000 Copilot Credits; ekstra forbruk måles i Copilot Credits\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#15\",\n    \"claim\": \"Agent observability (Microsoft Agent 365) er inkludert i Copilot Studio-lisensen\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#16\",\n    \"claim\": \"CoE Starter Kit er gratis og open-source\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#17\",\n    \"claim\": \"Å kjøre CoE-appene krever Power Apps- eller Power Automate-lisens\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#18\",\n    \"claim\": \"Power BI for CoE-dashboards er inkludert i Power BI Pro (lisens per bruker per måned)\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#19\",\n    \"claim\": \"Dataverse storage for CoE-inventory er inkludert i base-lisensen, med overage fakturert per GB per måned\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#20\",\n    \"claim\": \"Azure Norway regions kan brukes til datalagring for å unngå dataeksport, og sensitive helseopplysninger kan lagres i norsk databank (Azure Norway regions)\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#21\",\n    \"claim\": \"Responsible AI Scorecard er dokumentert som preview — status GA med public preview for enkelte funksjoner (verifisert 2026-02)\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard?view=azureml-api-2\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/stakeholder-communication-ai-decisions.md#22\",\n    \"claim\": \"Azure AI Evaluation SDK har innebygde evaluators, blant annet RelevanceEvaluator | ViolenceEvaluator\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/stakeholder-communication-ai-decisions.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
    "claim_count": 22,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md`)\n\n[\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#1\",\n    \"claim\": \"Microsofts Transparency Note-standardformat består av komponentene What is a Transparency Note? | The basics of [system name] | Capabilities | Limitations | System performance | Evaluating and integrating | Learn more about responsible AI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#2\",\n    \"claim\": \"Azure OpenAI Transparency Note advarer om sikkerhetsrisiko ved Computer Use, som er i preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#3\",\n    \"claim\": \"Microsofts model card-implementasjon omfatter Microsoft Foundry model catalog med innebygde model cards for pretrained modeller | Hugging Face-integrasjon der model cards synkroniseres automatisk | mal for å generere egne model cards for custom modeller.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#4\",\n    \"claim\": \"Responsible AI Scorecard inneholder komponentene Model overview | Fairness assessment | Model interpretability | Error analysis | Counterfactual analysis | Causal inference | Data quality.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#5\",\n    \"claim\": \"Responsible AI Scorecard i Azure ML har status Public preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#6\",\n    \"claim\": \"Microsoft Responsible AI Standard v2 (juni 2022) inneholder sjekklister og maler for Impact Assessments.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#7\",\n    \"claim\": \"Responsible AI dashboard i Azure Machine Learning er en pakke med 7 verktøy (fairness, explainability, error analysis m.fl.).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#8\",\n    \"claim\": \"Azure OpenAI Service har egne transparency notes per modell, blant annet GPT-4 | DALL-E 3 | Whisper | o1.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#9\",\n    \"claim\": \"Azure OpenAI API-responsen inkluderer content filter-scores for kategoriene hate | violence | sexual | self-harm.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#10\",\n    \"claim\": \"Microsoft Foundry sine dokumentasjonsfunksjoner omfatter Model catalog | Evaluation tools | Transparency Notes | Tracing | Compliance integrations (eksport til Microsoft Purview).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#11\",\n    \"claim\": \"Microsoft Foundry model catalog inneholder over 1500 pretrained modeller med model cards.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#12\",\n    \"claim\": \"Microsoft Copilot Studio har innebygde disclosures: «Powered by AI»-merke i chat-vinduet | attribusjonslenker til kildedokumenter for generative answers | bekreftelsesdialoger før sensitive plugin-handlinger | lenke til personvernerklæring i bot-innstillingene.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/microsoft-copilot-studio/responsible-ai-overview\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#13\",\n    \"claim\": \"Microsoft Purview tilbyr data lineage | sensitivity labels | audit logs | data catalog som funksjoner for AI-transparens og datastyring.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#14\",\n    \"claim\": \"Microsofts Transparency Notes er tilgjengelige kun på engelsk (kan oversettes av kunden).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#15\",\n    \"claim\": \"Responsible AI Scorecard støtter ikke norsk i preview — manuell oversettelse er nødvendig.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#16\",\n    \"claim\": \"Responsible AI dashboard er inkludert i Azure ML compute-kostnaden og krever ingen ekstra lisens.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-dashboard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#17\",\n    \"claim\": \"Responsible AI Scorecard er gratis som preview-funksjon.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#18\",\n    \"claim\": \"Scorecards lagret i Azure ML Run History er gratis i 90 dager, deretter gjelder standard storage-prising.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai-scorecard\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#19\",\n    \"claim\": \"Content Safety-annotasjoner er inkludert i Azure OpenAI API-prisingen uten ekstra kostnad.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#20\",\n    \"claim\": \"Transparency note for Azure OpenAI er sist oppdatert med o3/o4-mini og Deep Research system cards.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/transparency-note\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#21\",\n    \"claim\": \"Microsofts Responsible AI bygger på seks prinsipper: fairness | reliability | privacy | inclusiveness | transparency | accountability.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/machine-learning/concept-responsible-ai\"\n  },\n  {\n    \"id\": \"ms-ai-governance/responsible-ai/transparency-documentation-standards.md#22\",\n    \"claim\": \"Microsofts ISO/IEC 42001:2023-sertifisering dekker GitHub Copilot | M365 Copilot | Copilot Health | Copilot Studio | Dragon Copilot | Dragon Copilot (Radiologist) | Microsoft Foundry | Security Copilot.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/compliance/regulatory/offering-iso-42001\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/ai-foundry-disaster-recovery-planning.md",
    "claim_count": 25,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/ai-foundry-disaster-recovery-planning.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/ai-foundry-disaster-recovery-planning.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#1\",\n    \"claim\": \"Microsoft Foundry (AI Foundry) er generelt tilgjengelig (GA).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#2\",\n    \"claim\": \"Microsoft Foundry het tidligere Azure AI Studio / Azure AI Foundry.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#3\",\n    \"claim\": \"Microsoft Foundry tilbyr ikke automatisk failover eller disaster recovery ut av boksen; dette er eksplisitt dokumentert av Microsoft, og kunden må planlegge og implementere sin egen DR-strategi.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#4\",\n    \"claim\": \"For Foundry Agent Service er tilstandsdata (samtalehistorikk, agent-definisjoner, tråd-kontekst) fordelt på tvers av flere lagringstjenester, og det finnes per i dag ingen innebygd én-klikks eksport/import-funksjon for komplett gjenoppretting.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/agent-service-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#5\",\n    \"claim\": \"Komponentene i et AI Foundry-prosjekt med tilhørende lagringssted: Prosjektkonfigurasjon (AI Foundry control plane) | Modelldeployeringer (Azure OpenAI / AI Foundry) | Agentdefinisjoner (Cosmos DB, Standard mode) | Samtalehistorikk/tråder (Cosmos DB) | Kunnskapsfiler for agent (Azure Storage) | Søkeindekser for agent (Azure AI Search) | Datasett og artefakter (Azure Storage) | Notebook-filer og kode (Azure Storage) | Tilkoblinger og secrets (Azure Key Vault) | Container images (Azure Container Registry) | Application Insights (Log Analytics workspace).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/agent-service-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#6\",\n    \"claim\": \"Agentdefinisjoner lagres i kundens Cosmos DB når Foundry Agent Service kjører i Standard deployment mode, og sikres med Cosmos DB continuous backup.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/agent-service-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#7\",\n    \"claim\": \"Samtalehistorikk (tråder) for Foundry Agent Service lagres i Cosmos DB-databasen `enterprise_memory` og gjenopprettes via Cosmos DB point-in-time restore (PITR).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/agent-service-operator-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#8\",\n    \"claim\": \"Tilkoblinger og secrets lagret i Azure Key Vault er Microsofts backup-ansvar og har automatisk failover til sekundærregion.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline#backup-and-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#9\",\n    \"claim\": \"Azure Container Registry må konfigureres for geo-replikering av kunden selv, men Microsoft håndterer selve replikeringsmekanismen.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline#backup-and-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#10\",\n    \"claim\": \"Microsoft anbefaler følgende konfigurasjon før en hendelse inntreffer: Foundry account med Purview-integrasjon for compliance | Foundry project med user-assigned managed identity | Agent Service i Standard deployment mode | Cosmos DB med continuous backup og PITR, service-managed failover og read replication til failover-region | AI Search med unikt navn for å unngå navnekollisjon | Storage account med GZRS (geo-zone-redundant).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#11\",\n    \"claim\": \"Azure Cosmos DB continuous backup støtter tier-verdien `Continuous30Days` (30 dagers point-in-time restore), satt med `--continuous-tier` sammen med `--backup-policy-type Continuous`.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/continuous-backup-restore-introduction\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#12\",\n    \"claim\": \"Cosmos DB Service-Managed Failover gjør at Cosmos DB automatisk kan bytte skriveregion fra primærregion til sekundærregion ved et langvarig regionalt utfall.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#13\",\n    \"claim\": \"Geo-zone-redundant storage account for prosjektdata opprettes med SKU-navnet `Standard_GZRS` og kind `StorageV2`.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline#backup-and-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#14\",\n    \"claim\": \"Azure Storage-redundanstyper: LRS (3 kopier i én region) | ZRS (3 kopier på tvers av soner) | GRS (LRS + asynkron kopi til sekundærregion) | GZRS (ZRS + asynkron kopi til sekundærregion) | RA-GZRS (GZRS + lesetilgang til sekundærregion).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline#backup-and-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#15\",\n    \"claim\": \"Modellen gpt-4o brukes med modellversjonsstrengen \\\"2024-11-20\\\".\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#16\",\n    \"claim\": \"`data_zone_standard` er en gyldig deployment-type (deployment_type) for en gpt-4o-deployering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#17\",\n    \"claim\": \"gpt-4o-deployeringen har Norway East som primærregion og Sweden Central som sekundærregion.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#18\",\n    \"claim\": \"Kvoten for gpt-4o-deployeringen settes til 120 000 TPM i primærregion og 120 000 TPM i sekundærregion.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#19\",\n    \"claim\": \"Global training for fine-tuning er i Public Preview og tilbyr rimeligere fine-tuning, men gir ikke datasuverenitet.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#20\",\n    \"claim\": \"Regional training for fine-tuning er tilgjengelig i Norway East og Sweden Central.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#21\",\n    \"claim\": \"Microsoft anbefaler eksplisitt å definere account | projects | capability host | avhengigheter i IaC (Bicep eller Terraform), som kilden til sannhet for å reprodusere konfigurasjon og rolletildelinger.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#22\",\n    \"claim\": \"Cosmos DB-kontoen deployeres med resource-typen `Microsoft.DocumentDB/databaseAccounts` og API-versjon 2024-05-15.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#23\",\n    \"claim\": \"Storage-kontoen deployeres med resource-typen `Microsoft.Storage/storageAccounts` og API-versjon 2023-01-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#24\",\n    \"claim\": \"AI Foundry project deployeres med resource-typen `Microsoft.MachineLearningServices/workspaces` og API-versjon 2024-04-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/ai-foundry-disaster-recovery-planning.md#25\",\n    \"claim\": \"Azure Chaos Studio-eksperimentet (`Microsoft.Chaos/experiments`) bruker feilnavnet `urn:csci:microsoft:cosmosDB:failover/1.0` som continuous-type fault for Cosmos DB-failover.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/ai-foundry-disaster-recovery-planning.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/backup-recovery-strategies-ai-workloads.md",
    "claim_count": 17,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/backup-recovery-strategies-ai-workloads.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/backup-recovery-strategies-ai-workloads.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#1\",\n    \"claim\": \"Azure AI Search-indekser kan ikke sikkerhetskopieres direkte; gjenoppretting skjer ved reindeksering fra originale kildedata i Azure Storage eller Cosmos DB.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#2\",\n    \"claim\": \"Azure Backup støtter følgende ressurser som er relevante for AI-arbeidsbelastninger: Azure Virtual Machines (GPU/compute) | Azure Managed Disks | Azure Files (SMB/NFS) | Azure Blob Storage (operational backup) | Azure SQL Database | Azure Database for PostgreSQL.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#3\",\n    \"claim\": \"Azure Cosmos DB støttes ikke av Azure Backup, men har sin egen nativ continuous backup-mekanisme med point-in-time restore (PITR).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/continuous-backup-restore-introduction\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#4\",\n    \"claim\": \"Microsoft Foundry og Azure AI Search har verken støtte i Azure Backup eller nativ backup-mekanisme.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/baselines/azure-ai-foundry-security-baseline#backup-and-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#5\",\n    \"claim\": \"Operational backup for Azure Blobs via Azure Backup gir point-in-time restore av blob-data.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/backup/blob-backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#6\",\n    \"claim\": \"Azure Cosmos DB continuous backup tilbys i to nivåer: Continuous 7-day med 7 dagers retensjonsperiode | Continuous 30-day med 30 dagers retensjonsperiode.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/continuous-backup-restore-introduction\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#7\",\n    \"claim\": \"Ved point-in-time-gjenoppretting av Azure Cosmos DB opprettes det alltid en ny konto, og følgende konfigurasjoner gjenopprettes ikke automatisk og må rekonfigureres: brannmurregler | VNet-innstillinger | RBAC-tildelinger | private endpoints | lagrede prosedyrer | triggere | UDF-er.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/continuous-backup-restore-introduction\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#8\",\n    \"claim\": \"Azure SQL Database har 7 dagers korttidsretensjon (PITR) som standard, konfigurerbar fra 1 til 35 dager.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#9\",\n    \"claim\": \"Langtidsretensjon (LTR) for Azure SQL Database er ikke aktivert som standard og kan konfigureres opptil 10 år.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#10\",\n    \"claim\": \"Azure Managed Disks bruker inkrementelle snapshots med et maksimum på 500 snapshots per disk.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/backup/disk-backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#11\",\n    \"claim\": \"Azure Files støtter maksimalt 200 share snapshots per share.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#12\",\n    \"claim\": \"Azure Blob-versjonering gir ubegrenset antall blob-versjoner (kun begrenset av lagringskostnad).\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/backup/blob-backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#13\",\n    \"claim\": \"Azure Disk Backup bruker inkrementelle snapshots begrenset til 500 per disk, noe som med daglig backup gir maksimalt ca. 450 dagers retensjon fordi 50 snapshots er reservert for on-demand-backups.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/backup/disk-backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#14\",\n    \"claim\": \"Azure Storage-redundans omfatter: LRS (1 region, 3 kopier, 99.999999999 % holdbarhet / 11 niere) | ZRS (1 region, 3 soner, 99.9999999999 % / 12 niere) | GRS (2 regioner, 6 kopier, 99.99999999999999 % / 16 niere) | GZRS (2 regioner, 6 kopier: 3 soner + 3) | RA-GRS/RA-GZRS (som GRS/GZRS med lesetilgang til sekundærregionen).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#15\",\n    \"claim\": \"Norway East og Norway West er parede Azure-regioner med datasuverenitet i Norge, mens Norway East mot Sweden Central og Sweden Central mot Norway East er manuelle pariteter innenfor EU/EØS.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#16\",\n    \"claim\": \"Norway West har begrenset tjenestestøtte, og Sweden Central anbefales som alternativ sekundærregion.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/backup-recovery-strategies-ai-workloads.md#17\",\n    \"claim\": \"Den innebygde Azure Policy-definisjonen med ID 013e242c-8828-4970-87b3-ab247555486d brukes til å konfigurere automatisk backup av virtuelle maskiner til en Recovery Services vault.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/backup/backup-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/backup-recovery-strategies-ai-workloads.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/capacity-planning-dr-configurations.md",
    "claim_count": 17,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/capacity-planning-dr-configurations.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/capacity-planning-dr-configurations.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#1\",\n    \"claim\": \"Azure AI Search dimensjoneres med to skaleringsenheter: replikaer | partisjoner.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/design-guides/disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#2\",\n    \"claim\": \"App Service-planer for AI-workloads dimensjoneres med Premium v3-SKU-ene P3v3 (primærregion) og P2v3 (warm standby).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/design-guides/disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#3\",\n    \"claim\": \"Azure OpenAI har regionalt baserte kvoter, og eksisterende kvoter i sekundærregionen kan være utilstrekkelige ved failover.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#4\",\n    \"claim\": \"Kapasitet i Azure OpenAI kan pre-provisjoneres med Provisioned Throughput Units (PTU) via az cognitiveservices account deployment create.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#5\",\n    \"claim\": \"Modellen gpt-4o er tilgjengelig i Azure OpenAI med modellversjon 2024-08-06 og model-format OpenAI.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#6\",\n    \"claim\": \"En PTU-deployment av gpt-4o provisjoneres med --sku-capacity 50, altså 50 PTU.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#7\",\n    \"claim\": \"SKU-navnet for PTU-baserte Azure OpenAI-deployments er ProvisionedManaged.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#8\",\n    \"claim\": \"Cosmos DB-autoscale i DR-regionen konfigureres med baseline 4000 RU/s og maks 10000 RU/s (--max-throughput 10000).\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/design-guides/disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#9\",\n    \"claim\": \"DR-strategier med tilhørende RTO: Active-Active (full) ~0 | Active-Active (optimized autoscale) sekunder | Warm Standby (partial) 5–15 min | Cold Standby (IaC only) 30–60 min | Backup & Restore timer–dager.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/disaster-recovery#optimize-your-recovery-costs\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#10\",\n    \"claim\": \"Kritikalitetstiers for DR-strategivalg: Tier 0: Mission Critical | Tier 1: Business Critical | Tier 2: Business Operational | Tier 3: Administrative.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/disaster-recovery#optimize-your-recovery-costs\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#11\",\n    \"claim\": \"Azure OpenAI tilbyr pay-per-token som alternativ til PTU; pay-per-token i DR-regionen gir ingen garantert kapasitet ved failover.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/high-availability-resiliency\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#12\",\n    \"claim\": \"For Azure AI Search gir 2 replikaer kun read-only (query) SLA, mens 3 replikaer gir full read-write (query + indeksering) SLA; begge SLA-nivåene er 99,9 % og det finnes ingen 99,99 %-tier.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/design-guides/disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#13\",\n    \"claim\": \"Reservasjonsanbefalinger for DR-ressurser: App Service P2v3 = 1-år RI for baseline | Cosmos DB (autoscale) = ingen RI (per-bruk) | Azure OpenAI PTU = RI kun for primær | AI Search Standard = RI for begge regioner | Storage (GZRS) = reservert kapasitet.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/disaster-recovery#optimize-your-recovery-costs\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#14\",\n    \"claim\": \"AI Search-partisjoner kjører alltid og bør derfor dekkes av reservasjon i begge regioner.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/disaster-recovery#optimize-your-recovery-costs\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#15\",\n    \"claim\": \"Kapasitetsreservasjonsgrupper kan opprettes i regionen swedencentral med tilgjengelighetssonene 1, 2 og 3.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/infrastructure/management\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#16\",\n    \"claim\": \"VM-størrelsen Standard_NC24ads_A100_v4 (A100 GPU) kan kapasitetsreserveres i swedencentral, sone 1.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/infrastructure/management\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/capacity-planning-dr-configurations.md#17\",\n    \"claim\": \"AI Search-partisjoner kan ikke skaleres ned uten å gjenopprette tjenesten, derfor må partisjoner dimensjoneres identisk i begge regioner.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/well-architected/design-guides/disaster-recovery\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/capacity-planning-dr-configurations.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/chaos-engineering-ai-systems.md",
    "claim_count": 25,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/chaos-engineering-ai-systems.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/chaos-engineering-ai-systems.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#1\",\n    \"claim\": \"Azure Chaos Studio er generelt tilgjengelig (GA) per dokumentets statusfelt (sist oppdatert 2026-02).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#2\",\n    \"claim\": \"Azure Chaos Studio tilbyr to former for feilinjeksjon: agentbasert | tjenestenivå (service-direct).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#3\",\n    \"claim\": \"Azure OpenAI signaliserer API-throttling med HTTP 429-respons, som klienten må håndtere med retry og backoff.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#4\",\n    \"claim\": \"Registrering av Chaos Studio-targets via Azure Resource Manager REST (Microsoft.Chaos/targets) bruker api-version=2024-01-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#5\",\n    \"claim\": \"App Service-targetet i Chaos Studio heter Microsoft-AppService og har capability Stop-1.0.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#6\",\n    \"claim\": \"Chaos Studio-eksperimenter kan opprettes i regionen norwayeast (Norway East).\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#7\",\n    \"claim\": \"NSG-targetet i Chaos Studio heter Microsoft-NetworkSecurityGroup, og targets i en selector angis med type ChaosTarget.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#8\",\n    \"claim\": \"Chaos Studio-feilen for NSG-sikkerhetsregler identifiseres med URN urn:csci:microsoft:networkSecurityGroup:securityRule/1.1 (versjon 1.1).\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#9\",\n    \"claim\": \"NSG securityRule-feilen kjøres som feiltypen continuous, med varighet angitt i ISO 8601-format (PT10M).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#10\",\n    \"claim\": \"Parametrene til NSG securityRule-feilen er: name | protocol | sourceAddresses | destinationAddresses | destinationPortRanges | access | priority | direction.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#11\",\n    \"claim\": \"Azure-servicetaggen for styringstrafikk til Cognitive Services / Azure OpenAI er CognitiveServicesManagement.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#12\",\n    \"claim\": \"Privat DNS-sone for private endpoints mot Azure OpenAI er privatelink.openai.azure.com.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#13\",\n    \"claim\": \"Azure Load Testing YAML-konfigurasjon bruker skjemaversjon v0.1.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#14\",\n    \"claim\": \"Felter i Azure Load Testing YAML-konfigurasjonen: version | testId | testPlan | engineInstances | configurationFiles | failureCriteria | env.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#15\",\n    \"claim\": \"failureCriteria i Azure Load Testing uttrykkes med aggregeringsfunksjoner som avg(response_time_ms) og percentage(error).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#16\",\n    \"claim\": \"Azure CLI for Azure Load Testing bruker az load test create (med --load-test-resource, --test-plan, --engine-instances) og az load test-run create (med --test-id).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#17\",\n    \"claim\": \"Chaos Studio service-direct faults støtter ressursene App Service | AKS | Cosmos DB | NSG.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#18\",\n    \"claim\": \"Agentbaserte feil i Chaos Studio injiseres via VM-agent og dekker CPU-/memory-stress | nettverksfeil.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-agent-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#19\",\n    \"claim\": \"Tilgangskontroll i Chaos Studio er RBAC-basert med en dedikert Chaos-rolle.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#20\",\n    \"claim\": \"Verktøy for chaos testing mot Azure: Azure Chaos Studio | Azure Load Testing | Litmus Chaos | Toxiproxy | PYRIT.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#21\",\n    \"claim\": \"Azure Load Testing er innebygd i Azure og JMeter-basert.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#22\",\n    \"claim\": \"PYRIT er verktøyet for AI-spesifikk red teaming, integrert mot Azure AI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#23\",\n    \"claim\": \"Azure DevOps-oppgaven for deploy til App Service er AzureWebApp@1.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/devops/deliver/shift-right-test-production\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#24\",\n    \"claim\": \"Azure DevOps-oppgaven AzureCLI@2 tar inputene azureSubscription | scriptType | scriptLocation | inlineScript.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/devops/deliver/shift-right-test-production\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/chaos-engineering-ai-systems.md#25\",\n    \"claim\": \"Start av et Chaos Studio-eksperiment gjøres med POST mot Microsoft.Chaos/experiments/{navn}/start med api-version=2024-01-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/chaos-engineering-ai-systems.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/compliance-requirements-bcdr.md",
    "claim_count": 10,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/compliance-requirements-bcdr.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/compliance-requirements-bcdr.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#1\",\n    \"claim\": \"Azure-regionen Norway East ligger i Oslo og omtales som norsk datasuverenitetsregion, anbefalt primærregion for norsk offentlig sektor.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#2\",\n    \"claim\": \"Azure-regionen Norway West ligger i Stavanger og er sekundær norsk region.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#3\",\n    \"claim\": \"Azure-regionen Sweden Central ligger i Gävle.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#4\",\n    \"claim\": \"Sweden Central er typisk DR-region (sekundærregion) for Norway East.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#5\",\n    \"claim\": \"Azure-regionen North Europe ligger i Dublin og er et alternativt EU-region-valg for DR.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#6\",\n    \"claim\": \"Azure-regionen West Europe ligger i Amsterdam og er et alternativt EU-region-valg for DR.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#7\",\n    \"claim\": \"Overføring av DR-data fra Norway East til Sweden Central er en intern EU/EØS-overføring og krever ingen tilleggstiltak.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#8\",\n    \"claim\": \"Azure GRS (geo-redundant lagring) replikerer automatisk, og hvilken sekundærregion data havner i avhenger av region-paret — sekundærregionen må verifiseres å ligge i EU/EØS.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#9\",\n    \"claim\": \"All DR-replikering skal være kryptert med TLS 1.2 eller nyere.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/compliance\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/compliance-requirements-bcdr.md#10\",\n    \"claim\": \"Microsoft har tilgang til data i DR-regionen, men tilgangen er begrenset av Customer Lockbox og JIT (just-in-time)-tilgang.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-overview-wwps\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/compliance-requirements-bcdr.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/cost-analysis-dr-configurations.md",
    "claim_count": 16,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/cost-analysis-dr-configurations.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/cost-analysis-dr-configurations.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#1\",\n    \"claim\": \"Referansen oppgir status GA for kostnadsstyringsområdet den dekker (Microsoft Cost Management / Azure cost management + billing), sist oppdatert 2026-04.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#2\",\n    \"claim\": \"GPT-4o er tilgjengelig som modell i Azure OpenAI og brukes som primærmodell i TCO-eksempelet.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#3\",\n    \"claim\": \"Azure AI Search tilbyr Standard-tieren S1, som kan kjøres med 3 replikaer.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#4\",\n    \"claim\": \"Azure Cosmos DB kan konfigureres med multi-region-replikering og provisjonert throughput målt i RU/s (eksempel: 10K RU/s).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#5\",\n    \"claim\": \"Azure App Service tilbyr Premium v3-planen P3v3, som i eksempelet kjøres med 3 instanser.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#6\",\n    \"claim\": \"Azure Storage tilbyr geo-redundante redundansnivåer GRS | GZRS for replikering av data til sekundærregion.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#7\",\n    \"claim\": \"DR-strategier for AI-workloads deles i fire konfigurasjoner: active-active | warm standby | cold standby | backup-restore.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/disaster-recovery#optimize-your-recovery-costs\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#8\",\n    \"claim\": \"Azure Reserved Instances (reservasjoner) tilbys med løpetid på 1 år og 3 år.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#9\",\n    \"claim\": \"Azure OpenAI Provisioned Throughput (PTU) kan kjøpes med 1-års reservasjon, men det finnes ingen 3-års reservasjon for PTU.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#10\",\n    \"claim\": \"Azure Cache for Redis tilbyr Premium-tieren P1.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#11\",\n    \"claim\": \"Azure Savings Plans gir 1-års eller 3-års commitment, med fleksibilitet til å bruke kapasiteten i hvilken som helst region.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#12\",\n    \"claim\": \"Azure Blob Storage har Change Feed, som kan brukes til delta-replikering og event-drevet synkronisering av kun endringer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#13\",\n    \"claim\": \"Azure CDN er tilgjengelig som CDN-tjeneste for å cache statisk innhold (dokumenter) og redusere cross-region-trafikk.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#14\",\n    \"claim\": \"Azure CLI-kommandoen az costmanagement export create støtter parametrene --name, --scope, --type med verdien ActualCost, --timeframe med verdien MonthToDate, --storage-account, --storage-container, --recurrence med verdien Monthly og --recurrence-period.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#15\",\n    \"claim\": \"Azure CLI-kommandoen az consumption budget create støtter parametrene --budget-name, --amount, --category med verdien Cost, --time-grain med verdien Annually, --time-period, --resource-groups og --notifications med operator GreaterThan og threshold-verdier.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/cost-analysis-dr-configurations.md#16\",\n    \"claim\": \"Azure OpenAI tilbyr både pay-per-token og Provisioned Throughput (PTU) som prismodeller, slik at DR-regionen kan kjøres pay-per-token.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/cost-management-billing/cost-management-billing-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/cost-analysis-dr-configurations.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/data-replication-patterns-ai.md",
    "claim_count": 20,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/data-replication-patterns-ai.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/data-replication-patterns-ai.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#1\",\n    \"claim\": \"Azure Storage tilbyr replikeringsalternativene LRS (3 kopier i samme datasenter) | ZRS (3 kopier på tvers av Availability Zones, synkron) | GRS (LRS pluss asynkron replikering til sekundærregion, LRS der) | GZRS (ZRS pluss asynkron replikering til sekundærregion, LRS der) | RA-GRS/RA-GZRS (tillegg: lesetilgang til sekundærregionen).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#2\",\n    \"claim\": \"Azure Cosmos DB med multi-region writes gir automatisk geo-replikering med tilnærmet 0 RPO.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#3\",\n    \"claim\": \"Azure SQL Database Active geo-replication er asynkron med RPO på cirka 5 sekunder.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-sql/database/active-geo-replication-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#4\",\n    \"claim\": \"Azure AI Search-indekser har ingen innebygd replikering; multi-region krever manuell dual-indexing eller rebuild fra kilden.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#5\",\n    \"claim\": \"Azure OpenAI er en stateless tjeneste for modell-konfigurasjon, der DR håndteres ved IaC-basert redeploy.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-redundancy-replication-backup\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#6\",\n    \"claim\": \"Azure Key Vault har automatisk failover via Microsoft-managed geo-replikering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-redundancy-replication-backup\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#7\",\n    \"claim\": \"Azure Cosmos DB kan replikeres mellom regionene norwayeast og swedencentral, og hver lokasjon kan gjøres sone-redundant med isZoneRedundant=true.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#8\",\n    \"claim\": \"az cosmosdb create støtter flaggene --enable-multiple-write-locations og --default-consistency-level, og --locations tar regionName | failoverPriority | isZoneRedundant.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#9\",\n    \"claim\": \"Bicep-ressurstypen Microsoft.Network/trafficmanagerprofiles brukes med API-versjon 2022-04-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-redundancy-replication-backup\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#10\",\n    \"claim\": \"En Azure Traffic Manager-profil konfigureres med trafficRoutingMethod (f.eks. Priority) og monitorConfig-feltene protocol | port | path | intervalInSeconds | timeoutInSeconds | toleratedNumberOfFailures, samt endepunkter av typen Microsoft.Network/trafficmanagerprofiles/azureEndpoints.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-redundancy-replication-backup\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#11\",\n    \"claim\": \"Azure Cosmos DB har fem konsistensnivåer: Strong (lineariserbar) | Bounded Staleness (K versjoner eller T tid) | Session (konsistent innen sesjon) | Consistent Prefix (aldri out-of-order) | Eventual (ingen garanti om rekkefølge).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#12\",\n    \"claim\": \"Cosmos DB .NET SDK (Microsoft.Azure.Cosmos) eksponerer CosmosClientOptions med ConsistencyLevel og ApplicationPreferredRegions, samt region-konstantene Regions.NorwayEast og Regions.SwedenCentral.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#13\",\n    \"claim\": \"I Cosmos DB .NET SDK hentes session-token fra response.Headers.Session og gjenbrukes via ItemRequestOptions.SessionToken for konsistent lesing i neste request.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#14\",\n    \"claim\": \"Standard konflikthåndtering i Azure Cosmos DB er Last-Writer-Wins basert på _ts-feltet (timestamp), der siste skriving vinner.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#15\",\n    \"claim\": \"Cosmos DB custom conflict resolution stored procedure har signaturen resolveConflict(incomingRecord, existingRecord, isTombstone, conflictingRecords).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#16\",\n    \"claim\": \"Cosmos DB-diagnostikklogger i AzureDiagnostics har ResourceProvider MICROSOFT.DOCUMENTDB og kategorien DataPlaneRequests.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#17\",\n    \"claim\": \"Azure Storage-operasjonen GetBlobServiceProperties returnerer GeoReplication.LastSyncTime, som er tilgjengelig i StorageBlobLogs.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/geo-redundant-design\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#18\",\n    \"claim\": \"Azure Monitor eksponerer metrikken ReplicationLatency for Cosmos DB-kontoer (Microsoft.DocumentDB/databaseAccounts).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/distribute-data-globally\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#19\",\n    \"claim\": \"Azure Monitor eksponerer metrikken GeoReplicationLag for Azure Storage-kontoer (Microsoft.Storage/storageAccounts).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/geo-redundant-design\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/data-replication-patterns-ai.md#20\",\n    \"claim\": \"Azure Storage Geo Priority Replication gir SLA-støttet RPO.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy-priority-replication\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/data-replication-patterns-ai.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/failover-testing-ai-services.md",
    "claim_count": 17,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/failover-testing-ai-services.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/failover-testing-ai-services.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#1\",\n    \"claim\": \"APIM kan konfigureres med en backend pool der Azure OpenAI i Norway East er primær og Azure OpenAI i Sweden Central er sekundær — dvs. Azure OpenAI tilbys i begge regionene.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#2\",\n    \"claim\": \"ARM-ressurstypen for et Azure Chaos Studio-eksperiment er `Microsoft.Chaos/experiments`.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-tutorial-service-direct-portal\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#3\",\n    \"claim\": \"Et Azure Chaos Studio-eksperiment kan opprettes med location `norwayeast`, dvs. Chaos Studio er tilgjengelig i Norway East.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#4\",\n    \"claim\": \"Chaos Studio-feilen for Cosmos DB-failover har capability-navnet `urn:csci:microsoft:cosmosDB:failover/1.0` og er av typen continuous (med parameteren readRegion).\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-tutorial-service-direct-portal\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#5\",\n    \"claim\": \"Chaos Studio-målet for Cosmos DB opprettes som ressursen `Microsoft.Chaos/targets/Microsoft-CosmosDB` under en `Microsoft.DocumentDB/databaseAccounts`-ressurs, og velges via selector av typen `List` med targettype `ChaosTarget`.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-tutorial-service-direct-portal\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#6\",\n    \"claim\": \"Cosmos DB kan ved regional failover bytte skriveregion fra Norway East til Sweden Central.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-tutorial-service-direct-portal\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#7\",\n    \"claim\": \"Azure OpenAI kalles med `api_version=\\\"2024-06-01\\\"` i AzureOpenAI-klienten.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#8\",\n    \"claim\": \"Modellen `gpt-4o` er tilgjengelig i Azure OpenAI og kalles via chat.completions-API-et.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#9\",\n    \"claim\": \"Application Insights-tabellen `AppRequests` i Log Analytics har feltene TimeGenerated | AppRoleName | ResultCode | DurationMs.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#10\",\n    \"claim\": \"Application Insights-tabellen `AppDependencies` har feltene TimeGenerated | DependencyType | Target | ResultCode | DurationMs, og DependencyType kan ha verdien `HTTP`.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#11\",\n    \"claim\": \"Cosmos DB-diagnostikklogger i `AzureDiagnostics` ligger under ResourceProvider `MICROSOFT.DOCUMENTDB` med kategorien `DataPlaneRequests` og feltene regionName_s | statusCode_s | duration_s.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#12\",\n    \"claim\": \"Azure Load Testing-tester opprettes med `az load test create` og flaggene --name | --resource-group | --location | --test-plan | --engine-instances.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#13\",\n    \"claim\": \"Azure Load Testing er tilgjengelig i regionen `norwayeast`.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/guide/testing/mission-critical-deployment-testing\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#14\",\n    \"claim\": \"Cosmos DB tilbyr service-managed failover som kan aktiveres for automatisk regional failover.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-tutorial-service-direct-portal\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#15\",\n    \"claim\": \"Azure Pipelines-tasken for Azure CLI-kjøring er `AzureCLI@2` med inputene azureSubscription | scriptType | inlineScript.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#16\",\n    \"claim\": \"Azure Chaos Studio-eksperimenter styres fra Azure CLI med `az chaos experiment start` og `az chaos experiment show`, med flaggene --name og --resource-group.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/failover-testing-ai-services.md#17\",\n    \"claim\": \"Azure Chaos Studio er GA og støtter service-direct faults mot Cosmos DB.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/chaos-studio/chaos-studio-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/failover-testing-ai-services.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/geo-redundancy-azure-ai-search.md",
    "claim_count": 15,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/geo-redundancy-azure-ai-search.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/geo-redundancy-azure-ai-search.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#1\",\n    \"claim\": \"Referansen angir at geo-redundans for Azure AI Search har status GA (sist oppdatert 2026-06-24).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#2\",\n    \"claim\": \"Azure AI Search er en regional tjeneste uten innebygd geo-replikering eller automatisk failover; blir regionen utilgjengelig, blir også search-tjenesten utilgjengelig.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#3\",\n    \"claim\": \"Typisk multi-region-oppsett for norsk offentlig sektor er primær search-tjeneste i Norway East med sekundær i Sweden Central, noe som holder data residency innenfor EU/EØS samtidig som det gir regional redundans.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/regions-multi-region-nonpaired\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#4\",\n    \"claim\": \"Azure AI Search har ingen innebygd mekanisme for indeksreplikering mellom regioner; man må selv implementere én av to strategier: Dual Push Indexing | Pull from Replicated Source (GRS/GZRS-replikert datakilde).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#5\",\n    \"claim\": \"Azure AI Search distribuerer automatisk replikaer på tvers av Availability Zones når tjenesten har 2 eller flere replikaer i en region som støtter AZ.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#6\",\n    \"claim\": \"Med 1 replika har Azure AI Search 99.9% SLA og støtter både lesbare spørringer og skriveoperasjoner, men gir ingen AZ-redundans.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#7\",\n    \"claim\": \"Med 2 replikaer har Azure AI Search 99.9% SLA, og replikaene AZ-distribueres automatisk.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#8\",\n    \"claim\": \"Med 3 eller flere replikaer har Azure AI Search 99.99% SLA for både lesing og skriving (read/write SLA), og dette er anbefalt for produksjon.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#9\",\n    \"claim\": \"Partisjonsdimensjonering etter indeksstørrelse i Azure AI Search: < 25 GB → 1 partisjon | 25–50 GB → 2 partisjoner | 50–150 GB → 3–6 partisjoner | > 150 GB → 6–12 partisjoner.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#10\",\n    \"claim\": \"Anbefalt SKU per region i produksjon er Standard eller Standard S2/S3, og sekundærregionen (DR) skal ha identisk SKU som primær.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#11\",\n    \"claim\": \"Azure Front Door-ressursene Microsoft.Cdn/profiles, Microsoft.Cdn/profiles/originGroups og Microsoft.Cdn/profiles/originGroups/origins deployeres med ARM/Bicep API-versjon 2023-05-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/frontdoor/front-door-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#12\",\n    \"claim\": \"Azure Front Door-SKU-en Premium_AzureFrontDoor brukes for failover-oppsettet foran Azure AI Search.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/frontdoor/front-door-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#13\",\n    \"claim\": \"Azure AI Search REST-API kalles med api-version=2024-07-01, f.eks. health probe mot /indexes/knowledge-base/docs?api-version=2024-07-01&search=*&$top=1.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#14\",\n    \"claim\": \"Synkroniseringsstrategier for indekser med typisk forsinkelse: Dual push (samtidige) ~0 | Event-driven sync sekunder | Scheduled indexer 5–60 min | Full rebuild timer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/geo-redundancy-azure-ai-search.md#15\",\n    \"claim\": \"Dokumenttelling per indeks hentes via REST-endepunktet /indexes/{index}/docs/$count?api-version=2024-07-01.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/search-multi-region\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/geo-redundancy-azure-ai-search.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/incident-response-ai-systems.md",
    "claim_count": 14,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/incident-response-ai-systems.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/incident-response-ai-systems.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#1\",\n    \"claim\": \"Referansen for incident response for AI-systemer er merket med status GA — de omtalte Azure-tjenestene for hendelseshåndtering (Azure Monitor alerts m.fl.) er generelt tilgjengelige.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#2\",\n    \"claim\": \"Azure tilbyr fire verktøy for AI-spesifikk overvåking og hendelseshåndtering: Microsoft Defender for AI Services (trusseloppdaging) | Azure Monitor (metrikk og alerting) | Microsoft Sentinel (SIEM/SOAR-kapabiliteter) | Application Insights (applikasjonslagsovervåking).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#3\",\n    \"claim\": \"Microsoft Defender for AI Services aktiveres som Defender-plan med plannavn «AI» og pristier «standard» (az security pricing create --name \\\"AI\\\" --tier \\\"standard\\\").\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/defender-for-cloud/ai-threat-protection\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#4\",\n    \"claim\": \"Azure OpenAI-diagnostikklogger i tabellen AzureDiagnostics har ResourceProvider «MICROSOFT.COGNITIVESERVICES» og loggkategori «RequestResponse».\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#5\",\n    \"claim\": \"Token-forbruk i Azure OpenAI-diagnostikklogger finnes i feltene properties_s.promptTokens og properties_s.completionTokens.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#6\",\n    \"claim\": \"Azure AI Search-diagnostikklogger i AzureDiagnostics har ResourceProvider «MICROSOFT.SEARCH», med OperationName «Indexers.Status» og resultatfeltene resultSignature_d og resultDescription_s.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#7\",\n    \"claim\": \"Azure OpenAI (ressurstype Microsoft.CognitiveServices/accounts) eksponerer metrikken ServerErrors som kan brukes i Azure Monitor metrikk-alerts.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#8\",\n    \"claim\": \"Azure OpenAI eksponerer metrikken Latency som kan brukes i Azure Monitor metrikk-alerts (eksempelet bruker terskelen avg Latency > 5000).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#9\",\n    \"claim\": \"Azure OpenAI eksponerer metrikken ProcessedPromptTokens som kan aggregeres med total i Azure Monitor metrikk-alerts.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#10\",\n    \"claim\": \"Azure Support bruker alvorlighetsgraden «Severity A» som nivå for produksjonskritiske saker (brukes ved P0-hendelser).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/incident-management\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#11\",\n    \"claim\": \"TAM (Technical Account Manager) er tilgjengelig som kontaktpunkt mot Microsoft Support for kunder med Enterprise-støtteavtale.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/incident-management\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#12\",\n    \"claim\": \"Sweden Central brukes som sekundær Azure OpenAI-region i failover-oppsettet (Traffic Manager-endepunkt «secondary-swedencentral»).\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#13\",\n    \"claim\": \"Azure Traffic Manager-endepunkter av typen «azureEndpoints» styres med prioritetsbasert ruting via --priority i az network traffic-manager endpoint update.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/incident-response-ai-systems.md#14\",\n    \"claim\": \"«AI-6: Establish monitoring and detection» er en kontroll i AI-sikkerhetsdomenet til Microsoft Cloud Security Benchmark v2 (MCSB v2).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-artificial-intelligence-security\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/incident-response-ai-systems.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/monitoring-alerting-failover-detection.md",
    "claim_count": 27,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/monitoring-alerting-failover-detection.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/monitoring-alerting-failover-detection.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#1\",\n    \"claim\": \"Kapabilitetene referansen dekker for overvåking og alerting ved failover-deteksjon (Azure Monitor alerts) har status GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#2\",\n    \"claim\": \"Modellen gpt-4o er tilgjengelig som Azure OpenAI-deployment og kan kalles via chat.completions for health check-kall.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#3\",\n    \"claim\": \"Availability tests for health check-endepunkter opprettes med Azure CLI-kommandoen az monitor app-insights web-test create, med parametrene --resource-group | --app-insights | --web-test-name | --location | --defined-web-test-name | --url | --expected-status-code | --frequency | --timeout | --enabled.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#4\",\n    \"claim\": \"Application Insights availability test (web-test) kan opprettes med location norwayeast.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#5\",\n    \"claim\": \"Azure OpenAI-diagnostikklogger ligger i AzureDiagnostics med ResourceProvider MICROSOFT.COGNITIVESERVICES og Category RequestResponse, med feltene properties_s.modelDeploymentName | duration_s | resultCode_d.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/monitor-openai\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#6\",\n    \"claim\": \"Azure AI Search-diagnostikklogger ligger i AzureDiagnostics med ResourceProvider MICROSOFT.SEARCH og OperationName Query.Search, med feltene DurationMs | Properties.ResultCount | resultSignature_d.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/search/monitor-azure-cognitive-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#7\",\n    \"claim\": \"Custom metrics til Application Insights sendes fra Python via opencensus.ext.azure.log_exporter.AzureLogHandler og applicationinsights-pakkens TelemetryClient initialisert med instrumentation_key, og emitteres med track_metric.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#8\",\n    \"claim\": \"Metric alert-regler opprettes med az monitor metrics alert create med parametrene --condition | --window-size | --evaluation-frequency | --severity | --action-group, der severity 0 er høyeste alvorlighetsgrad.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#9\",\n    \"claim\": \"Log-baserte alerts (scheduled query rules) opprettes med az monitor scheduled-query create med parametrene --condition | --condition-query | --evaluation-frequency | --window-size | --severity | --action-group.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#10\",\n    \"claim\": \"Azure Monitor-alertens webhook-payload eksponerer data.essentials med feltene severity (verdier Sev0 | Sev1) og alertRule.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#11\",\n    \"claim\": \"Azure Monitor Application Insights har dedikert støtte for AI-agenter via Agent details view.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#12\",\n    \"claim\": \"Agent details view i Application Insights omfatter funksjonene Unified agent view | End-to-end transaction details | Live metrics | Availability tests.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#13\",\n    \"claim\": \"Unified agent view monitorerer agenter fra Microsoft Foundry | Copilot Studio | tredjeparts i én visning.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#14\",\n    \"claim\": \"End-to-end transaction details sporer samtaledata: prompts | systemInstructions | tool usage.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#15\",\n    \"claim\": \"Microsoft Foundry-agenter instrumenteres ved å koble Application Insights til Foundry-prosjektet for automatisk tracing, eller ved å bruke Azure Monitor OpenTelemetry Distro med Foundry SDK.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/foundry/observability/how-to/trace-agent-setup\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#16\",\n    \"claim\": \"Copilot Studio-agenter instrumenteres via innebygd telemetri-eksport til Application Insights, konfigurert i innstillingene i Copilot Studio.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#17\",\n    \"claim\": \"Microsoft Agent Framework (self-hosted) instrumenteres med Azure Monitor OpenTelemetry Distro for telemetri til Azure Monitor.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#18\",\n    \"claim\": \"LangChain/LangGraph og OpenAI Agents SDK instrumenteres med Azure AI OpenTelemetry Tracer.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#19\",\n    \"claim\": \"Simple Log Search Alerts, som gir per-rad KQL-evaluering, er GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#20\",\n    \"claim\": \"Query-based metric alerts for Prometheus/OTel er i public preview.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#21\",\n    \"claim\": \"Alerts lagres i 30 dager i Azure Monitor.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#22\",\n    \"claim\": \"Azure Monitor Baseline Alerts (aka.ms/amba) gir policy-basert alerting i skala via Azure Policy.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#23\",\n    \"claim\": \"OpenTelemetry (OTel) er primær instrumentering i Application Insights, og de klassiske API-SDK-ene migreres til OTel.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#24\",\n    \"claim\": \"Azure Functions støtter OpenTelemetry via telemetryMode satt til OpenTelemetry i host.json.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#25\",\n    \"claim\": \"Nye views i Application Insights er Agent details view (Foundry, Copilot Studio, tredjeparts) | SDK Stats (exporter success/drop-metrikker) | Dashboards with Grafana (direkte i Azure-portalen).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#26\",\n    \"claim\": \"Evaluations i Application Insights finnes i to former: batch (local | cloud | portal) og continuous (produksjonstrafikk).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/monitoring-alerting-failover-detection.md#27\",\n    \"claim\": \"Utløste alert-instanser (fired alert instances) er read-only og kan ikke editeres etter at de er trigget.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/monitoring-alerting-failover-detection.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/multi-region-azure-openai-deployment.md",
    "claim_count": 20,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/multi-region-azure-openai-deployment.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/multi-region-azure-openai-deployment.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#1\",\n    \"claim\": \"Azure OpenAI i Norway East har begrenset modellstøtte, oppgitt som gpt-4o versjon 2024-11-20, med datasuverenitet i Norge/EU.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#2\",\n    \"claim\": \"Azure OpenAI i Sweden Central har bred modellstøtte med gpt-4o | o1 | gpt-4o-mini, og datasuverenitet i EU.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#3\",\n    \"claim\": \"Azure OpenAI i West Europe har begrenset modellstøtte med kun gpt-4o-mini, med datasuverenitet i EU.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#4\",\n    \"claim\": \"Azure OpenAI i UK South har moderat modellstøtte med gpt-4o | gpt-4o-mini, med datasuverenitet i UK.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#5\",\n    \"claim\": \"Azure OpenAI i France Central har bred modellstøtte med gpt-4o | o1, med datasuverenitet i EU.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#6\",\n    \"claim\": \"Data Zone-deployeringer lar Azure optimere ruting og prosessering på tvers av tilgjengelig compute innenfor datasonen (f.eks. EU), i stedet for selvadministrert lastbalansering mellom regionale deployeringer.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#7\",\n    \"claim\": \"Global Standard-deployering innebærer at data kan behandles globalt og gir høyest kapasitet.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#8\",\n    \"claim\": \"Azure API Management støtter fire lastbalanseringsmetoder for backend pools: Round-robin | Vektet | Prioritetsbasert | Session affinity.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/api-management/backends#load-balanced-pool\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#9\",\n    \"claim\": \"Azure OpenAI returnerer en Retry-After-header ved throttling (HTTP 429), og denne headeren skal brukes til å styre retry- og circuit-breaker-logikk.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#10\",\n    \"claim\": \"Azure har per i dag ingen native tjeneste for global server load balancer for arbeidsbelastninger som krever privat DNS-oppløsning; active/passive-mønster oppnås ved å endre DNS-posten for gatewayen.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/business-continuity-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#11\",\n    \"claim\": \"Azure OpenAI-kvote tildeles per abonnement + region + modell i enheter av Tokens-per-Minute (TPM), og TPM trekkes fra tilgjengelig kvote når en deployment opprettes.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#12\",\n    \"claim\": \"Maks antall Azure OpenAI-ressurser per region er 30.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#13\",\n    \"claim\": \"Det er ingen begrensning på antall deployeringer per modell — begrensningen ble fjernet med det nye kvotesystemet.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#14\",\n    \"claim\": \"For gpt-4o og eldre chat-modeller gir 1 kapasitetsenhet 6 RPM og 1 000 TPM.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#15\",\n    \"claim\": \"For o1 og o1-preview gir 1 kapasitetsenhet 1 RPM og 6 000 TPM.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#16\",\n    \"claim\": \"For o3-mini, o1-mini og o3-pro gir 1 kapasitetsenhet 1 RPM og 10 000 TPM.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#17\",\n    \"claim\": \"For o3 og o4-mini gir 1 kapasitetsenhet 1 RPM og 1 000 TPM.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#18\",\n    \"claim\": \"Kapasitet per modell og region kan hentes via REST-endepunktet Microsoft.CognitiveServices/modelCapacities med api-version=2024-06-01-preview.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/quota\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#19\",\n    \"claim\": \"Microsoft anbefaler å kombinere Provisioned Throughput Units (PTU) med Standard-deployeringer, der PTU dekker baseline-kapasitet med fast pris og Standard Data Zone tar spillover ved trafikktopper med pay-per-token.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-multi-backend\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/multi-region-azure-openai-deployment.md#20\",\n    \"claim\": \"APIM Premium kreves for multi-region gateway, mens APIM Standard v2 er tilstrekkelig for mange scenarier uten multi-region.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-multi-backend\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/multi-region-azure-openai-deployment.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/network-resilience-patterns-ai.md",
    "claim_count": 15,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/network-resilience-patterns-ai.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/network-resilience-patterns-ai.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#1\",\n    \"claim\": \"Azure Well-Architected Framework definerer resiliensmønstrene Circuit Breaker | Retry med exponential backoff | Bulkhead | Throttling som særlig relevante for AI-workloads.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/reliability/design-patterns\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#2\",\n    \"claim\": \"Azure Front Door (global) har innebygd DDoS-beskyttelse på edge.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#3\",\n    \"claim\": \"En ExpressRoute-krets kan opprettes med båndbredde 1000 Mbps (--bandwidth-in-mbps 1000).\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#4\",\n    \"claim\": \"ExpressRoute-peeringlokasjonen Oslo er tilgjengelig med provider Telenor.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#5\",\n    \"claim\": \"ExpressRoute-peeringlokasjonen Stavanger er tilgjengelig med provider GlobalConnect.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#6\",\n    \"claim\": \"VPN-gateway kan opprettes med SKU VpnGw2AZ (sonefordelt VPN Gateway-SKU) i regionen norwayeast.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#7\",\n    \"claim\": \"Private DNS-sonen for private endepunkter mot Azure OpenAI heter privatelink.openai.azure.com.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/private-link/private-link-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#8\",\n    \"claim\": \"Azure OpenAI chat completions kalles med modellnavnet gpt-4o.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#9\",\n    \"claim\": \"Private endpoint mot Microsoft.CognitiveServices/accounts (Azure OpenAI) bruker group-id (sub-ressurs) account.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/private-link/private-link-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#10\",\n    \"claim\": \"Azure OpenAI settes opp med norwayeast som primærregion og swedencentral som DR-region.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#11\",\n    \"claim\": \"Private endpoint mot Microsoft.Search/searchServices (Azure AI Search) bruker group-id (sub-ressurs) searchService.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/private-link/private-link-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#12\",\n    \"claim\": \"Azure DDoS Network Protection aktiveres som en DDoS-beskyttelsesplan (az network ddos-protection create) som kobles til et VNet.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#13\",\n    \"claim\": \"Service tag CognitiveServicesManagement brukes som destinasjonsprefiks i NSG-regler for utgående trafikk til Azure OpenAI.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#14\",\n    \"claim\": \"Service tag AzureCognitiveSearch brukes som destinasjonsprefiks i NSG-regler for utgående trafikk til Azure AI Search.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/network-resilience-patterns-ai.md#15\",\n    \"claim\": \"Azure Front Door Premium gir global lastbalansering med DDoS-beskyttelse og WAF i ett produkt.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/ddos-protection/ddos-protection-overview\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/network-resilience-patterns-ai.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/rto-rpo-planning-ai-services.md",
    "claim_count": 14,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/rto-rpo-planning-ai-services.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/rto-rpo-planning-ai-services.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#1\",\n    \"claim\": \"For Azure Storage med Geo Priority Replication er RPO for blobs garantert <= 15 minutter, for 99,0 % av faktureringsperioden.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy-priority-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#2\",\n    \"claim\": \"Azure OpenAI har ingen innebygd disaster recovery; tjenesten er stateless og DR må løses med manuell multi-region-oppsett der man redeployer i ny region.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/business-continuity-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#3\",\n    \"claim\": \"Azure AI Search har ingen innebygd replikering mellom regioner; DR krever manuelt multi-region-oppsett der indeksen bygges på nytt fra kilden.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#4\",\n    \"claim\": \"Azure Cosmos DB med multi-region writes gir innebygd RPO på omtrent 0 gjennom automatisk geo-replikering, og er konfigurerbar.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-business-continuity-high-availability-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#5\",\n    \"claim\": \"Azure Blob Storage med GRS (aktivert som GRS/GZRS) har innebygd RPO på omtrent 15 minutter fordi replikeringen er asynkron.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#6\",\n    \"claim\": \"Azure Blob Storage tilbyr Geo Priority Replication som et redundansalternativ med SLA-dekket RPO (SLA-backed RPO).\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy-priority-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#7\",\n    \"claim\": \"Azure SQL Database med active geo-replication har innebygd RPO på omtrent 5 sekunder med asynkron replikering.\",\n    \"claim_type\": \"tpm\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-business-continuity-high-availability-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#8\",\n    \"claim\": \"Azure Machine Learning har ingen innebygd disaster recovery; multi-region må settes opp manuelt med separat storage per region.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-business-continuity-high-availability-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#9\",\n    \"claim\": \"Azure OpenAI kan deployes i regionene norwayeast (primær) og swedencentral (sekundær) i et multi-region failover-oppsett.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/business-continuity-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#10\",\n    \"claim\": \"Azure OpenAI-klienten kalles med api_version «2024-10-21».\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/business-continuity-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#11\",\n    \"claim\": \"Modell-deploymentet gpt-4o brukes som Azure OpenAI-deployment i både primær og sekundær region.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/business-continuity-disaster-recovery\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#12\",\n    \"claim\": \"Azure AI Search-tjenester kan opprettes i regionene norwayeast og swedencentral.\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#13\",\n    \"claim\": \"Azure AI Search har SKU-en «standard», som brukes med --sku «standard» i az search service create.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/rto-rpo-planning-ai-services.md#14\",\n    \"claim\": \"Azure Traffic Manager støtter rutingsmetoden Priority (--routing-method Priority) for failover-routing.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/concept-business-continuity-high-availability-disaster-recovery\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/rto-rpo-planning-ai-services.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/service-level-documentation-dr.md",
    "claim_count": 14,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/service-level-documentation-dr.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/service-level-documentation-dr.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#1\",\n    \"claim\": \"Referansefilen for Service Level Documentation og DR-runbooks (kategori Business Continuity & Disaster Recovery for Azure AI-tjenester) er merket med status GA.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#2\",\n    \"claim\": \"Azure Well-Architected Framework understreker at en DR-plan må inkludere tre essensielle komponenter: klar runbook | veldefinert kommunikasjonsplan | strukturert eskaleringsvei.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/well-architected/design-guides/disaster-recovery#document-your-dr-plan\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#3\",\n    \"claim\": \"DR-mønsteret for AI-plattformen bruker Norway East som primærregion og Sweden Central som sekundærregion (NE→SC) for Azure OpenAI, Azure AI Search, App Service, Azure Key Vault, Blob Storage, Redis Cache og App Configuration, mens Cosmos DB kjører aktivt i begge regioner (NE+SC).\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#4\",\n    \"claim\": \"DR-strategien for Azure AI Search er dual-indexing mot sekundær region, med RTO 15 minutter og RPO 30 minutter i komponentoversikten.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#5\",\n    \"claim\": \"Azure Cosmos DB støtter multi-region writes, som gir tilnærmet null RTO og RPO (~0/~0) ved regional feil.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#6\",\n    \"claim\": \"Azure Key Vault har Microsoft-managed failover, slik at gjenoppretting skjer automatisk (RTO og RPO = Auto) uten kundekonfigurert failover.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#7\",\n    \"claim\": \"Blob Storage for kildedokumenter bruker redundansnivået GZRS (geo-zone-redundant storage) som DR-strategi.\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#8\",\n    \"claim\": \"Azure Cache for Redis støtter geo-replication som DR-strategi mellom primær- og sekundærregion.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#9\",\n    \"claim\": \"Azure App Configuration støtter geo-replication som DR-strategi mellom regioner.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#10\",\n    \"claim\": \"App Service-planen i DR-regionen skaleres opp til SKU P3v3 ved failover og ned igjen til P2v3 etter verifisert failback (az appservice plan update --sku).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#11\",\n    \"claim\": \"Azure OpenAI chat completions kalles mot deployment gpt-4o på stien /openai/deployments/gpt-4o/chat/completions med api-version=2024-10-21.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#12\",\n    \"claim\": \"Azure Traffic Manager-endepunkter av typen azureEndpoints styres ved failover/failback med endpoint-status Enabled | Disabled (az network traffic-manager endpoint update).\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#13\",\n    \"claim\": \"Azure AI Search REST API brukes med api-version=2024-07-01 for indexes, datasources, skillsets, indexers og docs/search.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/reliability/reliability-ai-search\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/service-level-documentation-dr.md#14\",\n    \"claim\": \"Azure AI Search indexer-status-endepunktet (/indexers/{navn}/status) returnerer feltene lastResult.status og lastResult.itemsProcessed.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/reliability/reliability-ai-search\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/service-level-documentation-dr.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  },
  {
    "file": "skills/ms-ai-infrastructure/references/bcdr/state-management-failover.md",
    "claim_count": 14,
    "prompt": "# Per-claim groundedness judge — bake-off **v3.1** (recall-hardened over v3's 3 confirmed FNs)\n\nv3.1 of `judge-claim-prompt-v3.md`. Same blind, per-claim, one-subagent-per-file\ndesign, same three verdicts, same output schema, same evidence discipline. v3.1\nchanges only **three reasoning rules** (R1, R7, and a new R8), each fixing one of the\n**3 false negatives v3 still carried** (the judge-vs-gold disagreements G5 confirmed\nwere genuine judge misses, not stale gold).\n\n**Why v3.1 exists — and what it is NOT (transparent, not p-hacking).** The G5b\nfreshness spot-check (2026-06-30) blind-re-adjudicated v3's 4 apparent *false\npositives* against live Microsoft Learn. **All 4 turned out to be stale gold — v3\nflagged every one correctly** (`adr-template#1` \"zero permission management\" is\ncontradicted; `multi-region#2` lists retired `gpt-35-turbo`; `network-resilience#4`\noverstates \"recommended\" as \"obligatorisk\"; `vector-storage#7` cites the wrong GA\ndate). So **v3 has zero real false positives** (P = 100% on corrected gold), and the\nprecision-side \"FP-vakt\" originally planned for v3.1 is **dropped — there is nothing to\ndefend.** v3.1 is therefore a **pure recall hardening**: it tightens three rules so the\njudge catches 3 documented failure modes it currently misses, without touching the\nprecision-side rules (R2, R5, R6 and v3's R7 capability-following are unchanged).\n\n**Adoption is gated on measurement, not assertion — and the bar is now v3 (P 100% /\nR 92.9% on corrected gold), not v2.** v3 sits at the precision ceiling, so v3.1 can only\nbe adopted if it **holds P = 100% AND lifts R above 92.9%** (catches FNs without\nintroducing a single new false positive). Any new FP drops P below 100% and fails the\ngate — keep v3. Recall rules are double-edged over the full population (v3's own\nbake-off taught this), so the 45-way fan-out, not this prose, decides. v3 results stay\nfrozen; v3.1 writes to `judge-bakeoff-results-v3.1.json` and is graded against the\ncorrected `gold-correctness-set.json`.\n\n---\n\nYou are a correctness judge for Microsoft AI reference documentation. You verify\nfactual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).\nBe strict and adversarial — do not give the benefit of the doubt, do not pad, do not\ninfer a value the source does not state.\n\nYou are judging claims extracted from `skills/ms-ai-infrastructure/references/bcdr/state-management-failover.md`. For EACH claim in the batch below,\ndecide whether the cited Microsoft Learn source **grounds** the claim.\n\n## The three verdicts (exhaustive, mutually exclusive)\n\n- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed\n  value(s). The page supports the claim. (Maps to gold `correct`.)\n- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a\n  **different / contradicting / superseded** value for what the claim asserts. The\n  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)\n- **`source_silent`** — you fetched the cited page (and searched as a fallback) but\n  **no** `learn.microsoft.com` page states the claimed value at all. You cannot\n  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure\n  pages typically lands here — that is expected, not a failure. **Exception: existence\n  claims — see Rule R2.**\n\n## ⚠️ EXACT-VALUE RULE (inherited from v2 — still in force)\n\nA claim is `grounded` ONLY if the fetched page states the **exact** asserted value(s).\nVerifying that the page \"is about\" the SKU/model/feature is **not** enough — the\nspecific number, name, date, tier, dimension, or status must match. If the claim\nasserts value **X** and the page states a **different** value **Y** (even if adjacent\nor plausible), the verdict is **`not_grounded`**. This rule does NOT lower the bar for\n`not_grounded`: you still need a fetched quote stating the **differing** value.\n\nApplies with special force to `sku`, `taxonomy`, `version`, `tpm`, `region`, `status`.\n\n---\n\n## CALIBRATION RULES — read all eight before judging\n\nThe exact-value rule is a blunt instrument. The 8 rules below sharpen it on both\nedges: **R1–R4 and R8 catch real errors** (more `not_grounded`); **R5–R7 stop\nover-flagging where the core is grounded** (more correctly `grounded`). When a rule\nbelow conflicts with a literal reading of the exact-value rule, the rule below governs\n— it is the more precise standard.\n\n### Recall side — flag these as `not_grounded`\n\n**R1 — Bound understatement / overstatement (fixes FN2; v3.1 splits lower vs upper).**\nA claim may assert a **bound**. Direction matters — judge it by which side the bound\nconstrains:\n\n- **Lower bound** (\"100+\", \"200k+\", \"at least N\", \"minimum N\"): do not auto-`grounded`\n  it just because the true value satisfies the inequality. Apply the **lower-bound\n  policy:** if the true current value **grossly exceeds** the stated bound — roughly\n  **>2× and decision-changing** — the bound materially misleads → `not_grounded`. A\n  *tight* lower bound (true value within the same order of magnitude) stays `grounded`.\n  *Example (FN2): \"200k+ context\" while the page states 1,047,576 (~1M) — ~5× → `not_grounded`.*\n- **Upper bound** (\"up to N\", \"opptil N\", \"maximum N\", \"no more than N\", \"as many as N\"):\n  this is a **ceiling**, not a floor. The lower-bound leniency does **NOT** apply. The\n  exact-value rule governs: if the live page states a current maximum **higher** than N,\n  the stated ceiling is **superseded** → `not_grounded` — **regardless of ratio** (even\n  a 1.1× exceedance breaks a ceiling). The claim tells the reader the limit is N when it\n  is really higher. *Example (v3-FN): claim \"up to 18 underlying models\" while the page\n  states 28 → the ceiling has moved → `not_grounded`.* (Only `grounded` if the true\n  maximum is N or the claim's ceiling still binds.)\n\n**R2 — `source_silent` does NOT excuse an existence claim (fixes FN3, FN5).** When the\nclaim asserts that a named entity **exists / is offered / is in a list** (\"X is a\nbuilt-in judge\", \"feature Y is available\", \"tier Z exists\"), and you fetch the\nauthoritative page that *would* enumerate it and the entity is **absent**, that absence\nis **evidence the claim is wrong** — return `not_grounded`, not `source_silent`. Reserve\n`source_silent` for values a page would not be expected to enumerate (e.g. JS-rendered\nprices). State in `reason` that you checked the canonical enumerating page and the\nentity was not present. *Example (FN5): claim \"99.99% SLA tier\" while the reliability\npage lists only 99.9% → absence of any 99.99% tier = `not_grounded`.*\n\n**R3 — Frame/unit replacement (fixes FN4).** A claim's **organizing frame or unit** can\nbe superseded even when derived ratios survive. If the page shows the claim's framing\nhas been **replaced** (e.g. \"1 Unit Capacity\" → \"Quota Tiers\"; a renamed/retired\nmetric), the claim is `not_grounded` even if some embedded numbers still appear\nsomewhere — the claim describes a world that no longer exists. Check that the *unit and\nstructure* the claim assumes still match the current page, not just the digits.\n\n**R4 — Current row, never a legacy row (fixes FN6).** Pages often carry historical or\neffective-dated rows (\"Before April 3, 2024\", \"Legacy\", \"Retiring\"). A claim is\n`grounded` only if it matches the **current/effective** row. Matching a clearly\ntime-stamped *past* row is `not_grounded` (the value has since changed). Always locate\nthe row that applies *today*. *Example (FN6): storage limits matching only the\n\"Before April 3, 2024\" row while current limits differ → `not_grounded`.*\n\n**R8 — Multi-part claims: every load-bearing part must hold (v3.1 — fixes the\n`ai-foundry-dr#9` FN).** A single claim often bundles **several load-bearing\nsub-assertions** (a status AND a region; a capability AND a named target; a date AND a\nGA level). Verify **each load-bearing part separately**. If **any one** load-bearing\npart is contradicted by the source, the whole claim is `not_grounded` — even when the\nother parts check out. Do not let a correct first half earn a `grounded` for a wrong\nsecond half. *Example (v3-FN): \"Global training (Public Preview), cheaper, no data\nresidency; use regional in Norway East\" — the GA-vs-Preview part and the \"no residency\"\npart hold, but **Norway East is a Global (non-residency) training region, not a regional\none** → one load-bearing part is wrong → `not_grounded`.* (R8 is the mirror of R6:\nR6 forgives an **omitted, non-load-bearing** detail; R8 condemns a **stated,\nload-bearing** part that is wrong. Decide first whether the part is load-bearing — if\nthe claim *asserts* it and a reader would act on it, it is.)\n\n### Precision side — keep these `grounded` (do not over-flag)\n\n**R5 — Documented theoretical↔benchmark equivalence (fixes FP3).** Do not flag a\nnumeric claim merely because the exact string is not verbatim, when the asserted value\nis the **documented theoretical or benchmark equivalent** of what the page states and\nboth trace to Microsoft sources (e.g. a theoretical max vs a measured benchmark of the\nsame technique, same order of magnitude, same direction). The exact-value rule targets\n*drifted/contradicting* values — not two Microsoft-sourced expressions of the same\nfact. If the page substantiates the magnitude and the technique, keep `grounded` and\nnote the equivalence in `reason`.\n\n**R6 — Core grounded, detail omitted ≠ ungrounded (fixes FP4).** Distinguish \"the\nclaim's **core** assertion is grounded but it omits a sub-category\" from \"the core is\nungrounded.\" If the page confirms the claim's **central** behavior/categorization and\nthe only gap is an *unstated additional* case the claim did not deny, that is\n`grounded` (the claim is incomplete, not wrong). Reserve `not_grounded` for when the\npage **maps the core differently** or the claim **asserts** something the page\ncontradicts. Omission ≠ contradiction. (Contrast R8: an omitted case is forgiven here;\na *stated* but wrong load-bearing part is not — that is R8's domain.)\n\n**R7 — Follow the capability to its canonical page; don't punish illustrative numbers\n(fixes FP5; v3.1 sharpens the load-bearing carve-out).** If a claim asserts a **real\ncapability** and the cited `evidence_url` does not foreground it, search for the\n**canonical** page that documents the capability before judging — do not return\n`not_grounded` merely because the *cited* page was a weak choice. And when a capability\nis solidly grounded, do **not** flag it over an *illustrative* attached number (e.g.\n\"~0 RTO/RPO\", \"≈15 min\") that the claim offers as an order-of-magnitude illustration\nrather than a cited spec. Judge the **capability**; treat an illustrative figure as\ngrounded if the capability is.\n\n> **⚠️ Load-bearing carve-out (v3.1, fixes the `token-usage#3` FN).** R7's leniency\n> covers only *illustrative* values. It does **NOT** cover a value or **exact string**\n> that **IS the assertion** — a metric name, an API field, an SDK identifier, an enum\n> value, a specific date/version. When the claim's load-bearing content is the literal\n> name/string itself (e.g. \"the metrics are `PromptTokens` and `CompletionTokens`\"),\n> the exact-value rule applies in full: if the live page names them differently\n> (`ProcessedPromptTokens` / `InputTokens` / `GeneratedTokens` / `OutputTokens`), the\n> claim is `not_grounded`. \"Follow to the canonical page\" means find the **right\n> names**, not rescue wrong ones. A reader would copy that string into code; an\n> illustrative magnitude they would not.\n\n---\n\n## Procedure (per claim)\n\n1. **Identify the volatile assertion(s)** in the claim text — and when the claim\n   bundles several (R8), enumerate **each load-bearing part**. The `claim_type` tells\n   you what to check:\n   - `version` → model/API version, GA date, context window, max output, training cutoff\n   - `tpm` → tokens-per-minute / throughput / quota numbers\n   - `sku` → SKU name, tier, PTU minimums, deployment type\n   - `region` → regional availability\n   - `status` → GA / preview / retirement / deprecation status\n   - `taxonomy` → categorization, capability mapping, which-feature-does-what\n2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's\n   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not\n   address the assertion, run `microsoft_docs_search` to find the authoritative page.\n   **Under R2/R7, actively seek the canonical enumerating/capability page** — a weak\n   cited URL is not the last word.\n3. **Exact-value entailment check** each checkable value (and each load-bearing part\n   under R8), then apply the calibration rules R1–R8. Classify which rule(s), if any,\n   govern the claim. For R1, first decide whether the bound is a **lower** bound (floor)\n   or an **upper** bound (ceiling) — they invert.\n4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a verbatim\n   quote you actually fetched from a `learn.microsoft.com` URL. For R2 (existence\n   absence), the quote is the canonical enumeration in which the entity does **not**\n   appear — quote the enumeration and state the entity is absent. No quote → `source_silent`.\n\n## Hard rules\n\n- Verify against the fetched page only. Do not rely on prior knowledge of model\n  specs / prices — those are exactly what may have drifted.\n- Stable identifiers are not volatile and are not your job to refute: regulation year\n  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10\n  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on\n  whatever volatile value it carries, else `source_silent`.\n- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.\n- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the\n  verdict (empty string for `source_silent`). `evidence_url` = the page you actually\n  used (may differ from the cited one if you fell back to search).\n- `rule` = which calibration rule governed, if any (`R1`–`R8`), else empty.\n\n## Batch to judge (from `skills/ms-ai-infrastructure/references/bcdr/state-management-failover.md`)\n\n[\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#1\",\n    \"claim\": \"Azure Cosmos DB Python SDK eksponerer en asynkron `CosmosClient` i namespace `azure.cosmos.aio`, og asynkron `DefaultAzureCredential` i `azure.identity.aio`.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/architecture/patterns/retry\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#2\",\n    \"claim\": \"Cosmos DB-dokumenter kan settes til automatisk utløp med et `ttl`-felt angitt i sekunder (her 3600 * 24, altså 24 timer).\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/architecture/patterns/retry\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#3\",\n    \"claim\": \"Den asynkrone Cosmos DB-klienten støtter betinget oppdatering via `replace_item(..., match_condition=...)` basert på dokumentets `_etag` for optimistisk låsing.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/cosmos-db/consistency-levels\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#4\",\n    \"claim\": \"Azure Cache for Redis er under retirement for alle SKU-er, annonsert oktober 2025.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#5\",\n    \"claim\": \"For Azure Cache for Redis Basic, Standard og Premium blokkeres ny-provisjonering 1. april 2026 for nye kunder og 1. oktober 2026 for eksisterende kunder, og gjenværende instanser slås av fra 1. oktober 2028.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#6\",\n    \"claim\": \"Azure Cache for Redis Enterprise og Enterprise Flash retires 31. mars 2027.\",\n    \"claim_type\": \"status\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#7\",\n    \"claim\": \"Azure Managed Redis anbefales for nye løsninger og tilbyr active geo-replication | zone-redundant by default | Entra ID-autentisering.\",\n    \"claim_type\": \"taxonomy\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#8\",\n    \"claim\": \"Azure Cache for Redis Premium kan opprettes i regionene norwayeast (primær) og swedencentral (sekundær geo-replica).\",\n    \"claim_type\": \"region\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#9\",\n    \"claim\": \"Azure Cache for Redis tilbyr Premium-SKU med vm-size P1 (`--sku Premium --vm-size P1` i `az redis create`).\",\n    \"claim_type\": \"sku\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#10\",\n    \"claim\": \"`az redis create` støtter parameteren `--minimum-tls-version` med verdien 1.2 for å sette minimum TLS-versjon.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#11\",\n    \"claim\": \"Geo-replikering mellom to Azure Cache for Redis-instanser opprettes med `az redis server-link create` med parametrene `--server-to-link` og `--replication-role Secondary`.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-geo-replication\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#12\",\n    \"claim\": \".NET-grensesnittet `IDistributedCache` tilbyr `GetStringAsync`/`SetStringAsync`, og `DistributedCacheEntryOptions` har egenskapen `AbsoluteExpirationRelativeToNow`.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/architecture/patterns/retry\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#13\",\n    \"claim\": \"`text-embedding-3-large` er et gyldig modellnavn for embedding-generering via embeddings-API-et.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/architecture/patterns/retry\"\n  },\n  {\n    \"id\": \"ms-ai-infrastructure/bcdr/state-management-failover.md#14\",\n    \"claim\": \"Chat completion kan gjøres idempotent/deterministisk ved bruk av `seed`-parameteren.\",\n    \"claim_type\": \"version\",\n    \"evidence_url\": \"https://learn.microsoft.com/azure/architecture/patterns/retry\"\n  }\n]\n\n## Output (strict JSON, no fence)\n\n```\n{\"file\":\"skills/ms-ai-infrastructure/references/bcdr/state-management-failover.md\",\"results\":[\n  {\"id\":\"<claim id>\",\"judge_verdict\":\"grounded|not_grounded|source_silent\",\"rule\":\"<R1-R8 or empty>\",\"evidence_url\":\"<url actually used>\",\"evidence_quote\":\"<verbatim quote or empty>\",\"reason\":\"<one sentence: what the source said vs the claim>\"}\n]}\n```\n"
  }
]