open/ms-ai-architect
1
0
Fork 0
Code Issues Releases Activity

fix(ms-ai-architect): G7 idx 26 lukket — scorecard-segmenter rettet, dashboard-komponenter merket

Browse source 
Error analysis og Counterfactual analysis sto som item 4/5 i 'Komponenter i
Scorecard'. Begge kilder hentet live denne okten og bekrefter malingen
uavhengig av 9.6: how-to-responsible-ai-scorecard enumererer summary/model
overview, data analysis, model performance, cohorts, top important factors,
fairness insights og causal insights; concept-responsible-ai-dashboard lister
Error analysis og Counterfactual what-if som *dashboard*-komponenter.

Form (b), operatorratifisert: relabel framfor fjerning. Lista renummererer
rent til 1-5 — 1,2,3,4,6,7-artefakten var tvungen kun inne i delete-only-
konvolutten, ikke for en ordinaer Edit. De to kapabilitetene beholdes i et
sitatblokk eksplisitt merket som dashboard-komponenter, sa kildebekreftet
informasjon overlever og leseren advares mot nettopp den forvekslingen som
skapte defekten. Lokator 2: Risk assessment-raden leser na 'fairness insights'.

Confidence-stempelet tolv linjer under vouchet for den falske lista og la
utenfor enhver maskinsjekk (V1/V2/V2b/V3 er strenginvarianter; check-g7-queue
tester kun ankere). Beholdt, men datert 2026-08-03 for a fore re-verifiseringen.
Dokument-koherens lest etter editen (c569bdc-laerdommen).

Nabodefekt funnet av filsveipet bokfort separat som idx-26b, ikke foldet inn:
'Quantitative analyses' attribueres til Scorecard, men er en Model Card-seksjon
(samme fil, linje 77) — kryss-attribuering mellom to standarder.

Suite 1047/1047.

[skip-docs]
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 21:23:35 +02:00
commit 8cde10f09a
2 changed files with 20 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

15
scripts/kb-eval/data/g7-review-queue.json
View file

@ -35,13 +35,26 @@
"id": "idx-26",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "multi-locator",
"status": "open",
"status": "resolved",
"raised": "2026-08-03",
"summary": "The Responsible AI Scorecard component list names Error analysis (item 4) and Counterfactual analysis (item 5). Both were measured false against first-party docs 2026-08-03: the canonical scorecard segments are summary/model overview, data analysis, model performance, cohorts, top important factors, fairness insights and causal insights; Error analysis and Counterfactual analysis are Responsible AI *dashboard* components. The delete-only reduction could only remove item 5, because line 300 asserts Error analysis as scorecard content too — so a partial fix would have left a known-false claim standing while introducing a renumbering artifact (1,2,3,4,6,7). Operator declined the partial fix 2026-08-03 and sent the whole case here. Correct repair spans both locators.",
"evidence": "docs/r11-pilot-results.md §9.6; https://learn.microsoft.com/azure/machine-learning/how-to-responsible-ai-scorecard",
"anchors": [
"4. **Error analysis**: Error rates per cohort, confusion matrices",
"| **Risk assessment** | Responsible AI Scorecard: Error analysis, fairness assessment |"
],
"resolution": "Operator-ratified 2026-08-03, form (b) — relabel rather than remove. Both source pages were re-fetched live this session and confirm the measurement independently of §9.6: how-to-responsible-ai-scorecard enumerates summary/model overview, data analysis, model performance, cohorts, top important factors, fairness insights and causal insights; concept-responsible-ai-dashboard lists Error analysis and Counterfactual what-if among the dashboard components. Locator 1: items 4 and 5 removed from the numbered scorecard list, which renumbers cleanly to 1-5 — the 1,2,3,4,6,7 artifact was forced only inside the delete-only envelope and does not apply to an ordinary Edit. The two capabilities are retained in a blockquote explicitly marked as dashboard components rather than scorecard segments, so genuine source-confirmed information survives and the reader is warned off precisely the conflation that produced the defect. Locator 2: the Risk assessment row now reads 'fairness insights' alone. The **Confidence:** Verified stamp twelve lines below vouched for the false list and was silently outside every machine check (V1/V2/V2b/V3 are string invariants; check-g7-queue only tests anchors); it is kept but dated to 2026-08-03 to record the re-verification. Document coherence around both locators was read after the edit, per the c569bdc lesson. A neighbouring defect surfaced by the file sweep is booked separately as idx-26b rather than folded in here."
},
{
"id": "idx-26b",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"raised": "2026-08-03",
"summary": "Surfaced by the post-edit file sweep for idx 26, in the same compliance-mapping table as idx 26's second locator, but outside both of its anchors — so booked separately rather than folded in (gap discipline; operator-ratified 2026-08-03). The Accuracy metrics row attributes 'Quantitative analyses' to the Responsible AI Scorecard. That is not a scorecard segment name: how-to-responsible-ai-scorecard calls the corresponding segment 'model performance'. The defect is a cross-attribution between two different standards rather than mere imprecision — 'Quantitative Analyses' is a canonical Model Card section, and this same file lists it as one at line 77. Repair is a replacement, so it is outside the delete-only envelope. Whether the right fix is to rename the segment or to drop the row is not yet decided.",
"evidence": "docs/r11-pilot-results.md §9.6; https://learn.microsoft.com/azure/machine-learning/how-to-responsible-ai-scorecard",
"anchors": [
"| **Accuracy metrics** | Responsible AI Scorecard: Quantitative analyses |"
]
},
{