docs(ms-ai-architect): R11 §9.3 — hele-fila-sjekken kjørt på de 15; 9 rene, 4 motsagt, 2 operatørkall [skip-docs]

Sveipet den ratifiserte brede cond2-lesningen over de 15 O2-kandidatene utenom
idx 8 og 17, én om gangen, over 10 filer. Ingen KB-fil redigert.

Ekstraktoren ble kalibrert på idx 8 FØR sveipet: den må hente fram
"Managed Compute Cluster" (linje 263) fra det slettede "eller managed compute
cluster". Den gjør det. Grep alene er ikke nok — idx 14s overlevende påstand er
norsk "automatisk" mot slettet engelsk "Automatically", null leksikalsk overlapp.
Hver kandidat ble derfor også lest på stedet.

Motsagt (ikke ratifiserbar som skrevet): 14, 18, 26, 27. Alle utenom 18 har en
redusert subtraksjon — ny streng, må gjennom check-o2-returns.mjs på nytt.
Operatørkall: 33, 36 — slettet innhold overlever i fila uten at resten blir falsk.
Rene: 7, 9, 19, 28, 31, 38, 40, 42, 45.

Verifisert skår: 2 av 46 (idx 17 + idx 19), ikke klassifikatorens opprinnelige 2.
Klassifikatorens cond2 tok feil i én retning: den skrev den motsigende linja inn i
sitt eget evidence-felt uten å behandle den som en defeater. Promptgap, ikke
modellfeil.

Suite 1021/1021.

docs/r11-pilot-results.md

@@ -437,6 +437,118 @@ idx 17 survives. That is the number §10 measurement #2 should be read with —
 classifier's own "2 of 46" counted idx 8 on a cond-2 justification that was
 half-unchecked.
 
+### 9.3 The whole-file check run on the remaining 15
+
+Run 2026-08-03, one candidate at a time, over the 15 O2 candidates other than
+idx 8 and 17 — ten distinct files. No KB file was edited.
+
+**Method, and its one calibration.** For each record, the deletion segments were
+recovered by diffing `file_text_verbatim` against `proposed_remainder` (word-level
+LCS), content phrases were extracted from each segment (markdown stripped,
+Norwegian and English stopwords dropped, contiguous content runs of 2-3 words kept
+as noun-phrase units), and each phrase was matched case-insensitively against every
+line of the file *outside* the verbatim block. The extractor was calibrated on
+idx 8 before the sweep: it must surface `Managed Compute Cluster` at line 263 from
+the deleted `eller managed compute cluster`. It does, as the top-ranked multi-word
+hit. Without that calibration a narrower extractor would have returned a clean
+bill on idx 8 — and silently on others.
+
+**The grep is necessary and not sufficient.** Two of the findings below have no
+lexical overlap with the deleted tokens at all. Idx 14's surviving claim is the
+Norwegian `automatisk` restating a deleted English `Automatically`; no token
+search finds it. Every candidate was therefore also read in place — the enclosing
+section around `real_line`, plus every hit line with context — and asked the
+second question the grep cannot: *does the surviving text now claim something
+broader or narrower than before, and does anything else in the file depend on the
+version that was there?*
+
+**Result: 9 clean · 4 contradicted · 2 operator calls.**
+
+| idx | file:line | outcome | the line that decides it |
+|---|---|---|---|
+| 7 | `document-intelligence-prebuilt-models.md:79` | clean | `prebuilt-document` occurs only in the deleted row; the one `General` hit (192) is `generalisering`; no count binds the table |
+| 9 | `data-drift-monitoring-detection.md:218` | clean | the only other Foundry/RAG reference (320) asserts exactly `groundedness, relevance` — the remainder — and never claims drift detection over grounding data |
+| 14 | `feedback-loops-continuous-improvement.md:555` | **contradicted (partial)** | 566: `Reviewed documents automatisk tilgjengelige i "Feedback loop" data source når modellen retraines` |
+| 18 | `rag-caching-optimization.md:29` | **contradicted** | 303-318: a whole section `### Azure AI Search - Built-in Caching`, plus 510: `Azure AI Search caching \| **Verified**` |
+| 19 | `rag-caching-optimization.md:297` | clean | the deleted bullet is the file's only indexing statement; the code sample's silence about explicit vector indexes predates the deletion |
+| 26 | `transparency-documentation-standards.md:117` | **contradicted (partial)** | 300: `\| **Risk assessment** \| Responsible AI Scorecard: Error analysis, fairness assessment \|` |
+| 27 | `transparency-documentation-standards.md:426` | **contradicted (partial)** | 216: `- **Copilot Studio**: "Powered by AI" disclosure i chat interface` |
+| 28 | `transparency-documentation-standards.md:83` | clean | `Hugging` occurs only in the deleted bullet; no model-card template is claimed anywhere else |
+| 31 | `ai-incident-response-procedures.md:139` | clean | `legalHold` occurs only in this block; `enabled` at 148 is Blob versioning; 82's "Legal hold på alle artifacts" is consistent with a tags-only hold |
+| 33 | `ai-threat-modeling-stride.md:211` | operator call | 357 restates both deleted capabilities — but about the CAF *document*, not about Defender AISPM |
+| 36 | `ai-threat-modeling-stride.md:38` | operator call | 310: `Backdoored models og data poisoning er Critical-severity trusler` — unqualified, while the remainder narrows the register to `(targeted)` |
+| 38 | `data-leakage-prevention-ai.md:396` | clean | both deleted policy templates occur only here; the following one-click-policy block names neither |
+| 40 | `supply-chain-security-ai-models.md:133` | clean | the CVSS band definition occurs only here; 158's `critical vulnerabilities` is container-image scanning, a different tool |
+| 42 | `supply-chain-security-ai-models.md:200` | clean (strengthened) | every other HuggingFace reference (32, 240, 492, 513) treats it as an *unverified* source; the deleted bullet was the outlier |
+| 45 | `semantic-caching-patterns.md:436` | clean (strengthened) | `Norway West` appears nowhere else; 303, 451, 488 and 628 are all Norway East |
+
+**The four contradictions are one shape.** In each, an enumerating passage would
+lose a member that the file continues to assert elsewhere — a section (18), a
+mapping row (26), a bullet in a sibling section (27), a prose restatement (14).
+That is idx 8's shape exactly: a list narrowed against a file that documents what
+was removed. Idx 18 is the hardest of them, because the surviving claim is not a
+stray sentence but a titled section *and* a row in the verification table stamping
+it `**Verified**`. No deletion confined to line 29 can fix that file; the correct
+edit is larger than the O2 envelope permits.
+
+**Three of the four admit a reduced subtraction**, on the same terms as idx 8:
+
+- **idx 14** — drop the `Automatically`→`Add` half, keep `/ SharePoint` (which is
+  the file's only occurrence). This also removes the V2b machine flag, since the
+  recapitalisation was the flagged part.
+- **idx 26** — drop item 4 (Error analysis), keep item 5 (Counterfactual analysis);
+  324, 475 and 693 attach counterfactuals to the dashboard and to GDPR, never to
+  the scorecard. The renumbering artifact (`1,2,3,4,6,7`) survives either way.
+- **idx 27** — drop the Chat-interface row, keep the Plugin-actions row; `grep -niE
+  "confirmation|plugin"` returns line 428 alone.
+
+Every one of these is a **different remainder string** than the one V2b attested,
+so none is machine-clean until `check-o2-returns.mjs` is re-run against it. Idx 18
+has no reduction: it is a single deletion.
+
+**The two operator calls are a distinct class, and are not being called
+contradictions.** In both, the deleted content survives elsewhere in the file
+without the remainder becoming false:
+
+- **idx 33** — line 357 does say `AI asset inventory via Azure Resource Graph` and
+  `Microsoft Purview Insider Risk Management for prompt-basert data
+  exfiltration-deteksjon`, stamped `*(Verified MCP 2026-04)*`. But it says it about
+  what the *Cloud Adoption Framework document* now covers, while the deleted
+  bullets attributed those capabilities to *Defender for Cloud AISPM*. Different
+  subjects, so no contradiction — but the edit's benefit is smaller than it looks,
+  because the content it removes stays in the file under another attribution.
+- **idx 36** — the remainder narrows the severity register to `Data Poisoning
+  (targeted)` while line 310 still justifies a recommendation with the unqualified
+  `data poisoning er Critical-severity`. A narrower statement does not contradict a
+  broader one; it is subsumed by it. What the edit produces is a file whose
+  severity table is more precise than the prose that cites it. Whether that is
+  acceptable, or whether 310 needs the same qualifier, is a judgement about the
+  file — and a companion edit at 310 exceeds the single-locator O2 envelope.
+
+**What this does to the score.** The sweep settles the whole-file dimension of
+cond 2 for all 15: affirmative for 9, negative for 4, operator for 2. It settles
+nothing about cond 3, which stands at `human_must_confirm` for ten of them. Only
+one candidate moves into the both-conditions-affirmative class: **idx 19**, whose
+cond-2 doubt was itself whole-file-shaped (the classifier worried that the
+preceding code sample's silence about explicit vector indexes might read as "no
+setup needed") and whose cond 3 the classifier already marked `yes`. The silence
+predates the deletion, and the deletion removes an affirmative false claim, so the
+remainder asserts nothing the file denies.
+
+**Verified score: 2 of 46 — idx 17 and idx 19.** Up from 1, and by a different
+route than the classifier's original 2: idx 8 left the class and idx 19 entered it.
+
+**What the sweep says about the method.** The classifier's cond-2 column was
+wrong in one direction only. Of the eleven candidates it marked `cond 2 = yes`,
+the whole-file check overturns two (27, 36) and confirms nine. Of the four it
+marked `human_must_confirm`, the check clears one (19) and confirms the doubt on
+three (14, 18, 26) — in every one of those three the classifier had already
+written the contradicting line number into its own evidence field without
+treating it as a defeater. The information was in the returns; the contract just
+never asked the classifier to act on it. That is a prompt gap, not a model
+failure, and it is cheap to close in a later wave: name internal consistency as a
+cond-2 dimension and require the citation.
+
 ## Appendix A — the 15 admitted proposals, hand-verified
 
 Every proposal the classifier (§4 + context condition) admitted over the whole
@@ -531,15 +643,22 @@ remainder, are in `scripts/kb-eval/data/r11-o2-returns/`.
 additionally carries a machine flag: its remainder recapitalises rather than
 deletes (§9.1, V2b), so it is a text change and must be reviewed as one.
 
-⚠️ **The `cond 2 = yes` column is the classifier's claim, not a verified fact.**
+⚠️ **The `cond 2` column above is the classifier's claim, not a verified fact —
-Hand-checking the two affirmative rows (§9.2) showed that half of idx 8's cond-2
+and it has now been corrected.** All 17 rows have had the whole-file check (idx 8
-justification was never checked, and that the same file documents the deleted
+and 17 in §9.2, the other 15 in §9.3). Read the column together with §9.3's table,
-alternative in its cost section. Read every `yes` in this column as "the
+which overrides it:
-classifier believed this". Only **idx 17** has been hand-verified. Idx 8 is not
+
-ratifiable as written: cond 2 is measured against the whole file (operator
+- **Contradicted, not ratifiable as written:** idx 8, 14, 18, 26, 27. All but 18
-ratification, 2026-08-03, §9.2), and the file contradicts the remainder at lines
+  admit a reduced subtraction; every reduction is a new remainder string and must
-263-265. **The verified score is 1 of 46, not 2.** The other 15 rows have not had
+  be re-run through `check-o2-returns.mjs` before it counts as verified.
-the whole-file check.
+- **Operator call:** idx 33, 36 — deleted content survives elsewhere in the file
+  without the remainder becoming false.
+- **Clean on the whole-file dimension:** idx 7, 9, 17, 19, 28, 31, 38, 40, 42, 45.
+  Cond 3 is still `human_must_confirm` for most of them; clean here means cond 2
+  only.
+
+**The verified score is 2 of 46 — idx 17 and idx 19**, not the classifier's
+original 2 (idx 8 left the class, idx 19 entered it).
 
 Reproduce the tally and the machine checks:
 `node scripts/kb-eval/check-o2-returns.mjs`.