ms-ai-architect/docs/r11-flag-format-2026-07.md

R11 flag format — the judge-pass → fix-work-list contract

Spec for the flag record the R7–R10 judge pass emits and R11 (the fix step) consumes. A flag is a judged claim the pass could NOT ground against its cited Microsoft Learn source. The pass never fixes — it stamps (born-verified) or flags; R11 does the human-confirmed fix.

Status: design spec (R6 Step 5). Builds on the frozen v3.1 claim judge (scripts/kb-eval/judge-claim-prompt-v3.1.md) and the judge-pass ledger (scripts/kb-eval/judge-pass-manifest.mjs, R6 Step 4). Consumed by R11.

---

Where a flag comes from

The v3.1 judge returns, per claim, a strict-JSON result (no fence):

{"file":"<FILE>","results":[
  {"id":"<claim id>","judge_verdict":"grounded|not_grounded|source_silent",
   "rule":"<R1-R8 or empty>","evidence_url":"<url actually used>",
   "evidence_quote":"<verbatim quote or empty>","reason":"<one sentence>"}
]}

The pass augments each non-grounded result (judge_verdict !== 'grounded') into a flag record. This is a minimal augmentation, not a pass-through — the judge output is a subset of the flag record; the pass adds four fields from the fan-out context and the claim manifest:

Added field	Source
file	the fan-out context (one subagent per file) — also echoed by the judge output's top-level file
line	the claim manifest (scripts/kb-eval/extract-judge-claims.mjs output)
claim	the claim manifest (the verbatim claim text that was judged)
disposition	mapped from judge_verdict (see the vocabulary mapping below)

The flag record

A flag record is the union of the judge result and the four augmented fields:

{
  "id":            "<claim id>",
  "judge_verdict": "not_grounded" | "source_silent",   // 'grounded' is never flagged
  "rule":          "R1".."R8" | "",
  "evidence_url":  "<url the judge actually used>",
  "evidence_quote":"<verbatim quote or empty>",
  "reason":        "<one sentence: what the source said vs the claim>",
  "file":          "skills/.../x.md",
  "line":          <number>,
  "claim":         "<verbatim claim text>",
  "disposition":   "outdated" | "wrong" | "unsourced"   // R11 fix disposition
}

In the ledger (R6 Step 4), a flagged file is a manifest record with per_file_verdict: "flagged" and one flags[] entry per non-grounded claim — the structural mirror of spor0-fix-manifest.json's "fix-record with applied:false". A file is flagged if any judgeable claim is non-grounded (the born-verified rule is AND: pass requires EVERY claim grounded).

The two vocabularies + the mapping

There are two distinct enumerations, and conflating them is the trap this spec exists to prevent:

1. judge_verdict — the judge's per-claim output (v3.1 prompt):

   • grounded — the source confirms the claim → never flagged.
   • not_grounded — the source contradicts / does not support the claim.
   • source_silent — the source does not address the claim at all.

2. disposition — the R11 fix disposition, the correctness vocabulary from scripts/kb-eval/lib/base-rate.mjs (VERDICTS): correct, outdated, wrong, unsourced. Its ERROR_VERDICTS subset — outdated and wrong — are the only ones R11 treats as fix targets. unsourced is recorded but is not, on its own, a content error (no MS source confirms it either way).

Mapping (judge_verdict → disposition):

judge_verdict	disposition	R11 fix target?
grounded	correct	no (never flagged)
not_grounded	outdated or wrong	yes — the human assigns which at R11
source_silent	unsourced	no (flagged, but not a fix target)

not_grounded → {outdated, wrong} is deliberately a one-to-two mapping: the judge establishes the claim is not grounded; whether it is stale-but-once-true (outdated) or never-true (wrong) is a human call at R11, re-verified against the live source (verification duty). The pass never auto-fixes — every flag is human-confirmed in R11, so a judge false-positive becomes a human review, never silent corruption of a public file.