open/ms-ai-architect
1
0
Fork 0
Code Issues Releases Activity

fix(ms-ai-architect): G7 idx-26b + idx-26c lukket — segmentnavn rettet, enumerasjonen komplettert

Browse source 
Kilden re-hentet live for skriving; §9.7 behandlet som premiss, ikke faktum.
how-to-responsible-ai-scorecard enumererer SJU segmenter og navngir
accuracy-segmentet 'model performance'. Begge malingene bekreftet uavhengig.

idx-26b (ratifisert form a: rename framfor fjerning): Accuracy metrics-raden
leser na 'Responsible AI Scorecard: model performance'. Sletting var
tilgjengelig og avvist — EU AI Act-mappingen er ekte og kildestottet, sa a
fjerne raden ville kastet sann informasjon for a reparere en navnedefekt.

idx-26c (ratifisert form a: legg til framfor nedgrader): 'Model performance' og
'Cohorts' lagt til som punkt 6-7; de fem eksisterende urort, sa endringen er
begrenset til det malte gapet. Punkt 7 formulert 'automatisk uttrukket' for a
holde det atskilt fra Cohort analysis i Customization-blokka, som er
operator-definert og ikke enumererer et segment.

STATE-PAASTAND FALSIFISERT: 'ankeret SKAL slutte a matche, ellers feiler
sjekken' er feil. lib/g7-queue.mjs:69-75 returnerer FOR ankersjekken for
resolved; unntaket er totalt, og anker-drift feller kun en entry som star apen.
Ukontrollert ville pastanden tvunget innsetting i kilderekkefolge pa
maskingrunner som ikke finnes. idx-26c er forste resolved entry hvis anker
fortsatt matcher ordrett — unntaket er na utovd for ekte.

To tillitsmarkorer avgjort i motsatt retning: stempelet pa linje 129 beholdt
uendret (scope sjekket; dekker na en komplett sju-punkts enumerasjon
re-verifisert samme dato det allerede barer), stempelet under
compliance-tabellen bevisst IKKE oppgradert (det dekker seks rader, fem umalte).

idx-26d bokfort, ikke foldet inn: parafrase-drift i punkt 1-5 mot kildens
segmentnavn. Den forelaa begge editene — det reparasjonen endret er
SYNLIGHETEN, siden punkt 6-7 barer kildenavn og lista na blander to dialekter.
Full kanonisering ble lagt fram som tredje alternativ og ikke valgt.

Ko: 5 apne / 4 resolved. Suite 1047/1047.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 21:40:21 +02:00
commit 63644c1791
2 changed files with 21 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

22
scripts/kb-eval/data/g7-review-queue.json
View file

@ -49,25 +49,27 @@
"id": "idx-26c",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"status": "resolved",
"raised": "2026-08-03",
"summary": "The scorecard enumeration is incomplete, and idx 26's repair made that assertion active rather than latent. The live fetch performed for idx 26 lists seven canonical segments: summary/model overview, data analysis, model performance, cohorts, top important factors, fairness insights, causal insights. The list after the idx 26 edit carries five — model overview, fairness assessment, model interpretability, causal inference, data quality — so **model performance** and **cohorts** are absent. Note for whoever repairs this: 'Cohort analysis' does appear a few lines below, but inside the *Customization* block, not as an enumerated segment, so it does not cure the omission. This was a latent incompleteness under an undated stamp before 2026-08-03; dating the stamp to 2026-08-03 as part of idx 26 turned it into a positive claim that this enumeration was verified that day, over content the same day's verification showed to be missing two members — the same class of defect §9.7 describes, where an anchor-correct edit leaves a false claim where no check reaches. Booked rather than folded into idx 26, whose ratified scope was the falsity of items 4 and 5, not the completeness of the list. The date on the stamp is deliberately NOT re-litigated here: the relabel it covers WAS verified 2026-08-03, and the operator may keep it once this entry closes the completeness gap. Adding the two missing segments is a content change requiring its own operator ratification.",
"evidence": "docs/r11-pilot-results.md §9.7; https://learn.microsoft.com/azure/machine-learning/how-to-responsible-ai-scorecard",
"anchors": [
"5. **Data quality**: Dataset statistics, missing values, outlier analysis"
]
],
"resolution": "Operator-ratified 2026-08-03, form (a) - add the two missing segments rather than downgrade the list to a selection. The source was re-fetched live this session and enumerates seven segments in order: summary/model overview, data analysis, model performance, cohorts, top important factors, fairness insights, causal insights. Model performance and Cohorts were appended as items 6 and 7; the existing five were left untouched, so the change is confined to the measured gap. Appending rather than inserting in source order was a free choice, not a machine-forced one: STATE asserted that a resolved entry anchor must stop matching or the check fails, and that is wrong - lib/g7-queue.mjs:69-75 returns before the anchor check for resolved entries, so resolved is fully exempt and anchor drift only bites an entry left open. The list carries no ordering claim, and the existing five were already not in source order, so appending introduces no new falsity. Item 7 is worded \"automatisk uttrukket av scorecard-en\" to keep it distinct from the Cohort analysis bullet in the Customization block, which is operator-defined and does not enumerate a segment. The **Confidence:** Verified (MCP: microsoft-learn, 2026-08-03) stamp at line 129 is kept unchanged and its scope was checked explicitly rather than left implicit: it now vouches for a complete seven-member enumeration re-verified against the live source on the date it already carries. Post-edit sweep of both regions found the surrounding prose coherent; the paraphrase drift it did surface is booked as idx-26d, not folded in."
},
{
"id": "idx-26b",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"status": "resolved",
"raised": "2026-08-03",
"summary": "Surfaced by the post-edit file sweep for idx 26, in the same compliance-mapping table as idx 26's second locator, but outside both of its anchors — so booked separately rather than folded in (gap discipline; operator-ratified 2026-08-03). The Accuracy metrics row attributes 'Quantitative analyses' to the Responsible AI Scorecard. That is not a scorecard segment name: how-to-responsible-ai-scorecard calls the corresponding segment 'model performance'. The defect is a cross-attribution between two different standards rather than mere imprecision — 'Quantitative Analyses' is a canonical Model Card section, and this same file lists it as one at line 77. Repair is a replacement, so it is outside the delete-only envelope. Whether the right fix is to rename the segment or to drop the row is not yet decided.",
"evidence": "docs/r11-pilot-results.md §9.6; https://learn.microsoft.com/azure/machine-learning/how-to-responsible-ai-scorecard",
"anchors": [
"| **Accuracy metrics** | Responsible AI Scorecard: Quantitative analyses |"
]
],
"resolution": "Operator-ratified 2026-08-03, form (a) - rename rather than drop the row. The source page was re-fetched live this session (how-to-responsible-ai-scorecard) rather than taken from §9.6 as a premise, and it names the segment model performance: \"The model performance segment displays your model most important metrics and characteristics of your predictions and how well they satisfy your desired target values.\" The Accuracy metrics row now reads \"Responsible AI Scorecard: model performance\". Rename was chosen over deletion because the EU AI Act accuracy-metrics mapping is genuine and source-supported; dropping the row would have removed true information to repair a naming defect. Line 77 keeps Quantitative analyses as a Model Card section, which is correct there and is what made the cross-attribution visible. The **Confidence:** Verified (Baseline + MCP-inferred) stamp at the foot of the compliance table was deliberately NOT upgraded: it covers all six rows and only one was measured against the source this session, so re-dating or strengthening it would have extended a verification claim over unmeasured content - the §9.7 defect class."
},
{
"id": "idx-27",
@ -106,6 +108,18 @@
"**Automatic Caching Behavior:**",
"| Azure AI Search caching | **Verified** | Microsoft Learn docs (4, 6) |"
]
},
{
"id": "idx-26d",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"raised": "2026-08-03",
"summary": "Surfaced by the post-edit sweep for idx-26c. The five original members of the scorecard component list carry paraphrased names rather than the source segment names, and idx-26c added two members that DO carry source names, so the list is now mixed. Concretely: item 5 Data quality maps to the source segment data analysis; item 3 Model interpretability maps to top important factors; item 1 Model overview is described as \"Architecture, training data, intended use\", which is Model Card content - the source summary segment is a model overview plus the key target values the user set. That last one is the same cross-attribution class as idx-26b rather than mere imprecision, since Model details / Intended use / Training data are canonical Model Card sections listed in this same file at lines 71-76. Full canonicalisation was offered to the operator as part of the idx-26c decision and was NOT chosen; the ratified scope there was completeness only, so this is booked rather than folded in. Open choice: rename the five to the source segment names in source order, or rewrite item 1 alone (the only member where the drift produces a false attribution rather than a recognisable paraphrase) and leave the rest.",
"evidence": "docs/r11-pilot-results.md §9.8; https://learn.microsoft.com/azure/machine-learning/how-to-responsible-ai-scorecard",
"anchors": [
"1. **Model overview**: Architecture, training data, intended use"
]
}
]
}