feat(ms-ai-architect): S1 judge bake-off harness + forhåndsregistrert gate (TDD) [skip-docs]

Deterministisk de-risk-harness for Fase 3-judgen (kjøres på frosset 373-påstands gull-sett):
- lib/judge-bakeoff.mjs: P-filter (volatil+fetchbar, price ekskl.), confusion-matrix
  for 3 armer (staleness/judge/hybrid), Wilson-bånd, forhåndsregistrert gate. 14 tester.
- extract-judge-claims.mjs: blind manifest (255 påstander, 0 label-lekkasje — testet invariant).
- judge-claim-prompt.md: blind per-påstands groundedness-judge (Opus xhigh, microsoft_docs_fetch).
- run-judge-bakeoff.mjs: join gull+results på id, gate-rapport (.json/.md).

Gate FORHÅNDSREGISTRERT (operatørvalg, før fan-out): recall ≥0.80, presisjon ≥0.70,
OG slår staleness (0/38). Evalueringspop P = 240 verifiserbare, 38 positive.
Suite 551/551 (538 + 13 nye).

scripts/kb-eval/extract-judge-claims.mjs

@@ -0,0 +1,51 @@
+#!/usr/bin/env node
+// extract-judge-claims.mjs — S1 step: emit the BLIND claim manifest the judge
+// subagents are graded on. Reads the frozen gold set, filters to the eval
+// population P (volatile + fetchable, price excluded) via the tested lib, strips
+// every claim to the fields the judge may see (no gold verdict / notes /
+// lastmod_changed / stratum — no label leakage), and writes a flat manifest.
+//
+// The fan-out groups these by file at dispatch time (one judge subagent per file,
+// mirroring how the gold set was built). The harness later joins the judge results
+// back to gold by id and runs run-judge-bakeoff.mjs.
+//
+// Usage:  node scripts/kb-eval/extract-judge-claims.mjs [--write]
+//   (default: print population summary; --write persists data/judge-bakeoff-claims.json)
+
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { blindClaims, evalPopulation } from './lib/judge-bakeoff.mjs';
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const DATA = path.join(__dirname, 'data');
+
+const gold = JSON.parse(fs.readFileSync(path.join(DATA, 'gold-correctness-set.json'), 'utf8'));
+const blind = blindClaims(gold.claims);
+
+// File -> claim count, for fan-out sizing (read-only summary).
+const byFile = {};
+for (const c of blind) (byFile[c.file] ||= 0), (byFile[c.file] += 1);
+const files = Object.keys(byFile).length;
+const withUrl = blind.filter((c) => c.evidence_url).length;
+
+if (process.argv.includes('--write')) {
+  const out = path.join(DATA, 'judge-bakeoff-claims.json');
+  const payload = {
+    _meta: {
+      source: 'gold-correctness-set.json',
+      population: 'volatile + fetchable claim_type (price excluded)',
+      blind: 'gold verdict/notes/lastmod_changed/stratum withheld — no label leakage',
+      claim_count: blind.length,
+      files,
+    },
+    claims: blind,
+  };
+  fs.writeFileSync(out, JSON.stringify(payload, null, 2) + '\n');
+  console.log(`wrote ${out} (${blind.length} blind claims across ${files} files)`);
+} else {
+  console.log(`eval population P: ${evalPopulation(gold.claims).length} claims`);
+  console.log(`blind manifest: ${blind.length} claims across ${files} files`);
+  console.log(`with evidence_url: ${withUrl} | without (judge falls back to docs_search): ${blind.length - withUrl}`);
+  console.log('(dry run — pass --write to persist judge-bakeoff-claims.json)');
+}

scripts/kb-eval/judge-claim-prompt.md

@@ -0,0 +1,80 @@
+# Per-claim groundedness judge — S1 bake-off (Fase 3 de-risk)
+
+Canonical instruction for the per-claim correctness judge. Runs as an Opus 4.8
+xhigh subagent, one subagent per reference file (it judges every claim in that
+file's batch). The dispatcher fills `<FILE>` and the `<CLAIMS>` batch from
+`data/judge-bakeoff-claims.json` (the BLIND manifest — it carries no gold verdict).
+
+The judge is **blind**: it never sees the gold label. Its verdict is joined back to
+the gold set by `id` in `run-judge-bakeoff.mjs` and scored as a detection task. This
+mirrors how the gold set itself was built (strict v2 evidence rule), so the judge is
+graded against a like-for-like standard.
+
+---
+
+You are a correctness judge for Microsoft AI reference documentation. You verify
+factual claims against **live, official Microsoft Learn** (`learn.microsoft.com`).
+Be strict and adversarial — do not give the benefit of the doubt, do not pad, do not
+infer a value the source does not state.
+
+You are judging claims extracted from `<FILE>`. For EACH claim in the batch below,
+decide whether the cited Microsoft Learn source **grounds** the claim.
+
+## The three verdicts (exhaustive, mutually exclusive)
+
+- **`grounded`** — you fetched a `learn.microsoft.com` page that states the claimed
+  value(s). The page supports the claim. (Maps to gold `correct`.)
+- **`not_grounded`** — you fetched a `learn.microsoft.com` page that states a
+  **different / contradicting / superseded** value for what the claim asserts. The
+  claim disagrees with the source. (Maps to gold `outdated` + `wrong`.)
+- **`source_silent`** — you fetched the cited page (and searched as a fallback) but
+  **no** `learn.microsoft.com` page states the claimed value at all. You cannot
+  confirm or refute it. (Maps to gold `unsourced`.) Pricing on JS-rendered Azure
+  pages typically lands here — that is expected, not a failure.
+
+A claim is `not_grounded` if the source contradicts **any** checkable value in it.
+It is `grounded` only if the source supports **all** checkable values. If the source
+states none of them, it is `source_silent`.
+
+## Procedure (per claim)
+
+1. **Identify the volatile assertion(s)** in the claim text. The `claim_type` tells
+   you what to check:
+   - `version` → model/API version, GA date, context window, max output, training cutoff
+   - `tpm` → tokens-per-minute / throughput / quota numbers
+   - `sku` → SKU name, tier, PTU minimums, deployment type
+   - `region` → regional availability
+   - `status` → GA / preview / retirement / deprecation status
+   - `taxonomy` → categorization, capability mapping, which-feature-does-what
+2. **Fetch the cited source** with `microsoft_docs_fetch` on the claim's
+   `evidence_url`. If the claim has no `evidence_url`, or the fetched page does not
+   address the assertion, run `microsoft_docs_search` to find the authoritative page.
+3. **Entailment check** each checkable value against the fetched text.
+4. **Strict evidence rule:** a `grounded` or `not_grounded` verdict REQUIRES a
+   verbatim quote you actually fetched from a `learn.microsoft.com` URL that states
+   the relevant value. No quote → `source_silent`. Never quote from memory.
+
+## Hard rules
+
+- Verify against the fetched page only. Do not rely on prior knowledge of model
+  specs / prices — those are exactly what may have drifted.
+- Stable identifiers are not volatile and are not your job to refute: regulation year
+  (2024/1689), case numbers (C-311/18), standard version names (OWASP LLM Top 10
+  2025, MADR v3.0), file names. If a claim is purely such an identifier, judge it on
+  whatever volatile value it carries, else `source_silent`.
+- One verdict per claim. Return EXACTLY the JSON below — no prose, no markdown fence.
+- `evidence_quote` = the verbatim sentence/value from the fetched page that drove the
+  verdict (empty string for `source_silent`). `evidence_url` = the page you actually
+  used (may differ from the cited one if you fell back to search).
+
+## Batch to judge (from `<FILE>`)
+
+<CLAIMS>
+
+## Output (strict JSON, no fence)
+
+```
+{"file":"<FILE>","results":[
+  {"id":"<claim id>","judge_verdict":"grounded|not_grounded|source_silent","evidence_url":"<url actually used>","evidence_quote":"<verbatim quote or empty>","reason":"<one sentence: what the source said vs the claim>"}
+]}
+```

scripts/kb-eval/lib/judge-bakeoff.mjs

@@ -0,0 +1,206 @@
+// scripts/kb-eval/lib/judge-bakeoff.mjs — S1 judge bake-off aggregation (deterministic).
+//
+// De-risks the Fase 3 correctness judge BEFORE production scaffolding (S3) is built:
+// grade a per-claim groundedness judge against the frozen gold set and decide whether
+// it beats the cheap staleness baseline (recall 0/40) by enough to justify ~2700
+// non-batchable microsoft_docs_fetch calls per full pass. Pure functions only — no
+// I/O, no Date.now/Math.random.
+//
+// Eval population P = volatile stratum + fetchable claim_type (price excluded). This
+// is the only population that matters: it is where the real errors concentrate, and
+// scoping to it structurally avoids the "inverted leverage" trap (a judge that wins
+// only by auto-scoring cheap stable claims it was never at risk on).
+//
+// Detection confusion matrix vs the gold verdicts:
+//   gold positive  = verdict ∈ {outdated, wrong}   (a real error the judge should catch)
+//   gold negative  = verdict === 'correct'
+//   excluded       = verdict === 'unsourced'        (no ground-truth value to grade against)
+//   predicted positive (flag) = judge_verdict === 'not_grounded'
+// judge_verdict vocabulary: 'grounded' | 'not_grounded' | 'source_silent'.
+
+import { wilson } from './base-rate.mjs';
+
+export const FETCHABLE_TYPES = new Set([
+  'taxonomy', 'status', 'version', 'tpm', 'sku', 'region',
+]);
+const ERROR_VERDICTS = new Set(['outdated', 'wrong']);
+
+// P = the claims the bake-off is measured on. A claim is in P iff it is volatile and
+// of a fetchable claim_type (price is excluded — 74% unsourced, JS-rendered Azure
+// pages defeat microsoft_docs_fetch, so a fetch-based judge cannot reach it either).
+export function evalPopulation(claims) {
+  return claims.filter(
+    (c) => c.stratum === 'volatile' && FETCHABLE_TYPES.has(c.claim_type),
+  );
+}
+
+// The BLIND manifest handed to the judge subagents: every claim in P, stripped to
+// the fields the judge is allowed to see. The gold verdict, notes, lastmod_changed
+// and stratum are deliberately withheld so the eval is blind (no label leakage).
+// All of P is included (unsourced too) — the judge must not know which claims the
+// human could not source; reproducing that boundary as source_silent is itself a
+// measured signal.
+const BLIND_FIELDS = ['id', 'file', 'skill', 'claim', 'claim_type', 'evidence_url'];
+export function blindClaims(claims) {
+  return evalPopulation(claims).map((c) => {
+    const out = {};
+    for (const k of BLIND_FIELDS) out[k] = c[k];
+    return out;
+  });
+}
+
+export function stalenessFlag(claim) {
+  return claim.lastmod_changed === true;
+}
+export function judgeFlag(claim) {
+  return claim.judge_verdict === 'not_grounded';
+}
+export function hybridFlag(claim) {
+  return stalenessFlag(claim) || judgeFlag(claim);
+}
+
+// Grade one detection arm over a list of *verifiable* claims (gold correct/outdated/
+// wrong only — unsourced must be filtered out before calling). flagFn(claim) -> bool
+// is the arm's predicted-error rule. Returns the confusion matrix plus precision /
+// recall / F1. precision is null when nothing is flagged (0/0, not NaN); recall is
+// null when there are no positives at all.
+export function gradeArm(verifiableClaims, flagFn) {
+  let tp = 0, fp = 0, fn = 0, tn = 0;
+  for (const c of verifiableClaims) {
+    const isError = ERROR_VERDICTS.has(c.verdict);
+    const flagged = flagFn(c);
+    if (isError && flagged) tp += 1;
+    else if (!isError && flagged) fp += 1;
+    else if (isError && !flagged) fn += 1;
+    else tn += 1;
+  }
+  const positives = tp + fn;
+  const negatives = fp + tn;
+  const flagged = tp + fp;
+  const precision = flagged ? tp / flagged : null;
+  const recall = positives ? tp / positives : null;
+  const f1 =
+    precision != null && recall != null && precision + recall > 0
+      ? (2 * precision * recall) / (precision + recall)
+      : null;
+  // Wilson 95% bands surface sampling noise — the positive sample is small (≈38),
+  // so a point estimate near the threshold should not be read as crisp. Bands are
+  // reported context; the gate itself uses the point estimate.
+  const recallWilson = positives ? wilson(tp, positives) : null;
+  const precisionWilson = flagged ? wilson(tp, flagged) : null;
+  return {
+    tp, fp, fn, tn, positives, negatives, flagged,
+    precision, recall, f1, recallWilson, precisionWilson,
+  };
+}
+
+// The PRE-REGISTERED gate. thresholds = { minRecall, minPrecision }. Pass requires
+// the judge to clear both thresholds AND strictly beat the staleness baseline's
+// recall (the directional point of S1: staleness recall is 0/40, so any positive
+// recall beats it — but the threshold guards against a judge too weak to justify
+// the fetch cost). A null judge precision (judge flagged nothing) cannot clear a
+// positive minPrecision.
+export function gateDecision(judgeArm, stalenessArm, thresholds) {
+  const { minRecall, minPrecision } = thresholds;
+  const recall = judgeArm.recall ?? 0;
+  const precision = judgeArm.precision; // may be null
+  const staleRecall = stalenessArm.recall ?? 0;
+
+  const recallOk = recall >= minRecall;
+  const precisionOk = precision != null && precision >= minPrecision;
+  const beatsStaleness = recall > staleRecall;
+  const pass = recallOk && precisionOk && beatsStaleness;
+
+  const reasons = [];
+  if (!recallOk) reasons.push(`recall ${fmt(recall)} < minRecall ${minRecall}`);
+  if (!precisionOk) {
+    reasons.push(
+      precision == null
+        ? 'judge flagged nothing (precision undefined)'
+        : `precision ${fmt(precision)} < minPrecision ${minPrecision}`,
+    );
+  }
+  if (!beatsStaleness) {
+    reasons.push(`recall ${fmt(recall)} does not beat staleness ${fmt(staleRecall)}`);
+  }
+  if (pass) reasons.push('all criteria met');
+  return { pass, recallOk, precisionOk, beatsStaleness, thresholds, reasons };
+}
+
+function fmt(x) {
+  return x == null ? 'n/a' : x.toFixed(3);
+}
+
+// source_silent is the judge's "I fetched but the page does not state this value"
+// verdict. It is diagnostic, not a flag. Two populations matter:
+//   - on verifiable claims (the human DID find a value): every source_silent is a
+//     judge verification miss. Split by whether the gold value was correct vs an error
+//     (a source_silent on a real error is a missed catch via "can't verify").
+//   - on unsourced-in-P claims (the human ALSO found no value): source_silent here is
+//     agreement — a good sign the judge reproduces the unverifiable boundary.
+function sourceSilentDiagnostics(P) {
+  let onVerifiableNegative = 0;
+  let onVerifiableError = 0;
+  let agreesWithUnsourced = 0;
+  let disagreesWithUnsourced = 0;
+  for (const c of P) {
+    const silent = c.judge_verdict === 'source_silent';
+    if (c.verdict === 'unsourced') {
+      if (silent) agreesWithUnsourced += 1;
+      else disagreesWithUnsourced += 1;
+    } else if (silent) {
+      if (ERROR_VERDICTS.has(c.verdict)) onVerifiableError += 1;
+      else onVerifiableNegative += 1;
+    }
+  }
+  return {
+    onVerifiableNegative,
+    onVerifiableError,
+    agreesWithUnsourced,
+    disagreesWithUnsourced,
+  };
+}
+
+// Per claim_type judge confusion matrix over the verifiable subset (so the report can
+// show where the judge earns its keep — sku/version/tpm carry the highest error rates).
+function judgeByClaimType(verifiable) {
+  const byType = {};
+  for (const c of verifiable) {
+    (byType[c.claim_type] ||= []).push(c);
+  }
+  const out = {};
+  for (const [t, arr] of Object.entries(byType)) {
+    out[t] = gradeArm(arr, judgeFlag);
+  }
+  return out;
+}
+
+// Top-level: join already done by the caller (each claim carries judge_verdict).
+// Filters to P, splits verifiable vs unsourced, grades all three arms over the same
+// verifiable set, computes diagnostics + the gate, returns one report object.
+export function computeBakeoff(joinedClaims, thresholds) {
+  const P = evalPopulation(joinedClaims);
+  const verifiable = P.filter((c) => c.verdict !== 'unsourced');
+  const unsourcedInP = P.length - verifiable.length;
+
+  const arms = {
+    staleness: gradeArm(verifiable, stalenessFlag),
+    judge: gradeArm(verifiable, judgeFlag),
+    hybrid: gradeArm(verifiable, hybridFlag),
+  };
+  const gate = gateDecision(arms.judge, arms.staleness, thresholds);
+
+  return {
+    population: {
+      total: P.length,
+      verifiable: verifiable.length,
+      positives: verifiable.filter((c) => ERROR_VERDICTS.has(c.verdict)).length,
+      negatives: verifiable.filter((c) => c.verdict === 'correct').length,
+      unsourcedInP,
+    },
+    arms,
+    sourceSilent: sourceSilentDiagnostics(P),
+    byClaimType: judgeByClaimType(verifiable),
+    gate,
+  };
+}

scripts/kb-eval/run-judge-bakeoff.mjs

@@ -0,0 +1,155 @@
+#!/usr/bin/env node
+// run-judge-bakeoff.mjs — S1 final glue: join the blind judge results back to the
+// frozen gold set, grade the judge / staleness / hybrid arms, apply the
+// PRE-REGISTERED gate, and write the bake-off report. All math lives in tested
+// lib/judge-bakeoff.mjs; this CLI is thin wiring (same shape as compute-base-rate.mjs).
+//
+// The thresholds are required flags, not defaults: the gate must be locked BEFORE the
+// judge fan-out runs (pre-registration), and the locked values are recorded in the
+// report _meta so the decision is auditable.
+//
+// Usage:
+//   node scripts/kb-eval/run-judge-bakeoff.mjs --min-recall 0.70 --min-precision 0.60 [--write]
+//
+// Inputs:  data/gold-correctness-set.json  (answer key)
+//          data/judge-bakeoff-results.json  ({ results: [{id, judge_verdict, ...}] })
+// Outputs: data/judge-bakeoff-report.{json,md}  (with --write)
+
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { computeBakeoff, evalPopulation } from './lib/judge-bakeoff.mjs';
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const DATA = path.join(__dirname, 'data');
+
+function flag(name) {
+  const i = process.argv.indexOf(name);
+  return i >= 0 ? process.argv[i + 1] : undefined;
+}
+const minRecall = Number(flag('--min-recall'));
+const minPrecision = Number(flag('--min-precision'));
+if (!Number.isFinite(minRecall) || !Number.isFinite(minPrecision)) {
+  console.error('error: --min-recall and --min-precision are required (pre-registered gate)');
+  process.exit(2);
+}
+const thresholds = { minRecall, minPrecision };
+
+const gold = JSON.parse(fs.readFileSync(path.join(DATA, 'gold-correctness-set.json'), 'utf8'));
+const resultsPath = path.join(DATA, 'judge-bakeoff-results.json');
+if (!fs.existsSync(resultsPath)) {
+  console.error(`error: ${resultsPath} not found — run the judge fan-out first`);
+  process.exit(2);
+}
+const judge = JSON.parse(fs.readFileSync(resultsPath, 'utf8'));
+
+// Join judge verdicts back to gold by id.
+const verdictById = new Map((judge.results || []).map((r) => [r.id, r.judge_verdict]));
+const joined = gold.claims.map((c) => ({ ...c, judge_verdict: verdictById.get(c.id) }));
+
+// Completeness check: every claim in P must have a judge verdict.
+const missing = evalPopulation(joined).filter((c) => c.judge_verdict == null);
+if (missing.length && !process.argv.includes('--allow-incomplete')) {
+  console.error(`error: ${missing.length} P-claims have no judge verdict (incomplete fan-out).`);
+  console.error('first few:', missing.slice(0, 5).map((c) => c.id).join(', '));
+  console.error('pass --allow-incomplete to grade anyway (missing claims count as not-flagged).');
+  process.exit(2);
+}
+
+const r = computeBakeoff(joined, thresholds);
+
+const pct = (x) => (x == null ? 'n/a' : `${(x * 100).toFixed(1)}%`);
+const num = (x) => (x == null ? 'n/a' : x.toFixed(3));
+const band = (w) => (w == null ? 'n/a' : `[${pct(w.low)}, ${pct(w.high)}]`);
+
+function armRow(name, a) {
+  return `| ${name} | ${a.tp} | ${a.fp} | ${a.fn} | ${a.tn} | ${pct(a.precision)} | ${pct(a.recall)} | ${band(a.recallWilson)} | ${num(a.f1)} |`;
+}
+function typeRow(t, a) {
+  return `| ${t} | ${a.positives} | ${a.tp} | ${a.fp} | ${a.fn} | ${pct(a.precision)} | ${pct(a.recall)} |`;
+}
+
+const p = r.population;
+const g = r.gate;
+const ss = r.sourceSilent;
+const verdict = g.pass ? '✅ PASS — bygg S3' : '❌ FAIL — stopp, ikke bygg S3';
+
+const md = `# Judge bake-off-rapport — S1 (Fase 3 de-risk)
+
+_Generert deterministisk av \`run-judge-bakeoff.mjs\` over \`gold-correctness-set.json\` + \`judge-bakeoff-results.json\`. Tall fra testet \`lib/judge-bakeoff.mjs\`. Ikke rediger for hånd — regenerer._
+
+**Forhåndsregistrert gate (låst FØR fan-out):** recall ≥ ${minRecall}, presisjon ≥ ${minPrecision}, OG judge-recall > staleness-recall.
+
+## Evaluerings-populasjon (P)
+
+Volatil stratum + fetchbare claim_types (price ekskludert) — der feilene bor; unngår «invertert leverage».
+
+| metrikk | verdi |
+|---|---|
+| P totalt | ${p.total} |
+| Verifiserbare (correct/outdated/wrong) | ${p.verifiable} |
+| Positive (reelle feil å fange) | ${p.positives} |
+| Negative (correct) | ${p.negatives} |
+| Unsourced i P (kjørt, men utenfor P/R) | ${p.unsourcedInP} |
+
+## Arm-sammenligning (detektering over de ${p.verifiable} verifiserbare)
+
+| arm | TP | FP | FN | TN | presisjon | recall | recall Wilson 95% | F1 |
+|---|---|---|---|---|---|---|---|---|
+${armRow('staleness (billig baseline)', r.arms.staleness)}
+${armRow('judge (per-påstand groundedness)', r.arms.judge)}
+${armRow('hybrid (union)', r.arms.hybrid)}
+
+## Judge per claim_type (verifiserbar delmengde)
+
+| claim_type | positive | TP | FP | FN | presisjon | recall |
+|---|---|---|---|---|---|---|
+${Object.entries(r.byClaimType).sort((a, b) => b[1].positives - a[1].positives).map(([t, a]) => typeRow(t, a)).join('\n')}
+
+## source_silent-diagnostikk
+
+Judgen hentet siden men fant ikke verdien. Diagnostisk, ikke et flagg.
+
+| signal | antall | tolkning |
+|---|---|---|
+| På verifiserbar feil | ${ss.onVerifiableError} | judge-bom: reell feil oversett via «kan ikke verifisere» |
+| På verifiserbar correct | ${ss.onVerifiableNegative} | judge reproduserte ikke et korrekt faktum mennesket fant |
+| Enig med unsourced | ${ss.agreesWithUnsourced} | judge reproduserer den uverifiserbare grensen (godt) |
+| Uenig med unsourced | ${ss.disagreesWithUnsourced} | judge hevdet grunnet/ugrunnet der mennesket ikke fant kilde |
+
+## GATE: ${verdict}
+
+- recall ${num(r.arms.judge.recall)} ≥ ${minRecall}? **${g.recallOk ? 'ja' : 'nei'}**
+- presisjon ${num(r.arms.judge.precision)} ≥ ${minPrecision}? **${g.precisionOk ? 'ja' : 'nei'}**
+- slår staleness (recall ${num(r.arms.staleness.recall)})? **${g.beatsStaleness ? 'ja' : 'nei'}**
+- begrunnelse: ${g.reasons.join('; ')}
+`;
+
+if (process.argv.includes('--write')) {
+  const jsonOut = path.join(DATA, 'judge-bakeoff-report.json');
+  const mdOut = path.join(DATA, 'judge-bakeoff-report.md');
+  fs.writeFileSync(
+    jsonOut,
+    JSON.stringify(
+      {
+        _meta: {
+          source: 'gold-correctness-set.json + judge-bakeoff-results.json',
+          thresholds,
+          judged: (judge.results || []).length,
+        },
+        ...r,
+      },
+      null,
+      2,
+    ) + '\n',
+  );
+  fs.writeFileSync(mdOut, md);
+  console.log(`wrote ${jsonOut}`);
+  console.log(`wrote ${mdOut}`);
+} else {
+  console.log(`P=${p.total} verifiable=${p.verifiable} positives=${p.positives}`);
+  console.log(`judge:     precision=${num(r.arms.judge.precision)} recall=${num(r.arms.judge.recall)} f1=${num(r.arms.judge.f1)}`);
+  console.log(`staleness: recall=${num(r.arms.staleness.recall)}`);
+  console.log(`GATE: ${g.pass ? 'PASS' : 'FAIL'} — ${g.reasons.join('; ')}`);
+  console.log('(dry run — pass --write to persist judge-bakeoff-report.json + .md)');
+}