open/ms-ai-architect
1
0
Fork 0
Code Issues Releases Activity

fix(ms-ai-architect): G7 idx-27b + idx-26f lukket — scorecard-punkt 2/5 til kildeordlyd, transparensmelding kopiert

Browse source 
idx-27b: Moenster 3-bulleten paastod en 'Powered by AI'-disclosure; erstattet med
den allerede ratifiserte standardmeldingen fra Copilot Studio-seksjonen, grep-
verifisert byte-for-byte. Ingen audience-tiering lagt til (scope-klassen fra 66fb567).

idx-26f: punkt 2 og 5 omskrevet til kildens ordlyd i idx-26d-dialekten framfor ren
sletting av navngitt spesifisitet - den minimale diffen ville etterlatt 'Dataset
statistics' (ogsaa ukildet) og 'across sensitive groups' (gammel dialekt), og dermed
arvet begge defektklassene. Form (b), stempel-innsnevring i fila, avvist av operatoer.

idx-26d-resolutionens sluttklausul supersedert med datert tillegg; idx-27c/27d
bokfoert for to naboer som ikke kunne repareres ved kopi.

Ko: 6 aapne / 8 resolved. Suite 1047/1047.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 22:26:49 +02:00
commit 9013d250bd
3 changed files with 104 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

34
scripts/kb-eval/data/g7-review-queue.json
View file

@ -121,19 +121,20 @@
"anchors": [
"1. **Model overview**: Architecture, training data, intended use"
],
"resolution": "Operator-ratified 2026-08-03, full canonicalisation. The source was re-fetched live before writing (how-to-responsible-ai-scorecard) and enumerates seven segments: summary/model overview, data analysis, model performance, cohorts, top important factors, fairness insights, causal insights. All five original member NAMES were renamed to the source segment names, closing the defect this entry was actually booked under - that the list mixed two dialects after idx-26c added two source-named members. Rewriting item 1 alone was rejected for exactly that reason: it removes the false attribution but leaves the booked defect standing. DESCRIPTIONS were rewritten for two members only: item 1 (was \"Architecture, training data, intended use\", which is Model Card content - Model details / Intended use / Training data are canonical Model Card sections listed in this same file at lines 71-76 - now \"Modelloversikt og de target-verdiene du har satt\", matching the source summary segment) and item 3 (was \"Feature importance (global/local explanations)\", RAI *dashboard* vocabulary inside a scorecard enumeration, contradicting this file's own callout that dashboard components are not scorecard segments - now \"Faktorene som påvirker modellens prediksjoner mest\"). Descriptions of items 2, 4 and 5 were deliberately left UNTOUCHED even though they carry specifics the source does not state, because that is a distinct defect class (unsourced specificity under a verification marker) booked as idx-26f; rewriting them here would have silently resolved an entry raised the same session and left the queue incoherent. Source order was NOT imposed: idx-26c established the list carries no ordering claim and the five were already not in source order, so reordering would be a larger diff with no measured defect behind it. Cross-file check run before editing: the five names occur elsewhere in the corpus, but in dashboard/interpretability contexts, not as a mirror of this enumeration - except stakeholder-communication-ai-decisions.md:57-62, which is a parallel five-member list in the same paraphrase dialect and is booked as idx-26e rather than swept in. What the **Confidence:** Verified (MCP: microsoft-learn, 2026-08-03) stamp closing §3 now vouches for, stated explicitly rather than left implicit: a seven-member enumeration whose NAMES are all the source's own segment names and whose descriptions for items 1, 3, 6 and 7 were verified against the live source on the date the stamp already carries. Items 2, 4 and 5 descriptions are NOT covered by that widening - idx-26f is open against them."
"resolution": "Operator-ratified 2026-08-03, full canonicalisation. The source was re-fetched live before writing (how-to-responsible-ai-scorecard) and enumerates seven segments: summary/model overview, data analysis, model performance, cohorts, top important factors, fairness insights, causal insights. All five original member NAMES were renamed to the source segment names, closing the defect this entry was actually booked under - that the list mixed two dialects after idx-26c added two source-named members. Rewriting item 1 alone was rejected for exactly that reason: it removes the false attribution but leaves the booked defect standing. DESCRIPTIONS were rewritten for two members only: item 1 (was \"Architecture, training data, intended use\", which is Model Card content - Model details / Intended use / Training data are canonical Model Card sections listed in this same file at lines 71-76 - now \"Modelloversikt og de target-verdiene du har satt\", matching the source summary segment) and item 3 (was \"Feature importance (global/local explanations)\", RAI *dashboard* vocabulary inside a scorecard enumeration, contradicting this file's own callout that dashboard components are not scorecard segments - now \"Faktorene som påvirker modellens prediksjoner mest\"). Descriptions of items 2, 4 and 5 were deliberately left UNTOUCHED even though they carry specifics the source does not state, because that is a distinct defect class (unsourced specificity under a verification marker) booked as idx-26f; rewriting them here would have silently resolved an entry raised the same session and left the queue incoherent. Source order was NOT imposed: idx-26c established the list carries no ordering claim and the five were already not in source order, so reordering would be a larger diff with no measured defect behind it. Cross-file check run before editing: the five names occur elsewhere in the corpus, but in dashboard/interpretability contexts, not as a mirror of this enumeration - except stakeholder-communication-ai-decisions.md:57-62, which is a parallel five-member list in the same paraphrase dialect and is booked as idx-26e rather than swept in. What the **Confidence:** Verified (MCP: microsoft-learn, 2026-08-03) stamp closing §3 now vouches for, stated explicitly rather than left implicit: a seven-member enumeration whose NAMES are all the source's own segment names and whose descriptions for items 1, 3, 6 and 7 were verified against the live source on the date the stamp already carries. Items 2, 4 and 5 descriptions are NOT covered by that widening - idx-26f is open against them. [Superseded 2026-08-03 by idx-26f's closure: items 2 and 5 were rewritten to source wording and item 4 adjudicated a recognisable paraphrase; see idx-26f's resolution for what the stamp covers now.]"
},
{
"id": "idx-27b",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"status": "resolved",
"raised": "2026-08-03",
"summary": "Surfaced by the neighbourhood sweep for idx-27. The Mønster 3 \"Azure implementasjon\" list asserts a Copilot Studio \"Powered by AI\" disclosure in the chat interface - the same claim idx-27 removed from the Copilot Studio section, and false for the same reason: faqs-generative-orchestration documents a default transparency message (\"Just so you are aware, I sometimes use AI to answer your questions.\"), not a badge. Different section, outside idx-27's anchors, so booked separately rather than swept in. Repair is a replacement, outside the delete-only envelope. Note for whoever repairs this: after idx-27 the corrected wording already exists a few hundred lines below, so the repair is a copy of an already-ratified formulation rather than a fresh judgement.",
"evidence": "docs/r11-pilot-results.md §9.6; https://learn.microsoft.com/microsoft-copilot-studio/faqs-generative-orchestration",
"anchors": [
"- **Copilot Studio**: \"Powered by AI\" disclosure i chat interface"
]
],
"resolution": "Operator-ratified 2026-08-03. The Moenster 3 \"Azure implementasjon\" bullet was replaced with exactly what the already-ratified row at the Copilot Studio section asserts: the standard transparency message \"Just so you are aware, I sometimes use AI to answer your questions.\" delivered in the chat interface. This is a copy of a formulation ratified earlier the same session (idx-27), not a fresh judgement, and the quote was grep-verified byte-for-byte against that row after the edit - hand-copying a verbatim string is where quote-style drift enters. Deliberately NOT carried over: any audience-tiering or layered-disclosure language from the surrounding Moenster 3 pattern. The bullet sits under an audience-layering table while the source row sits under \"Built-in disclosures\"; adding reach the source does not state is exactly the scope class caught in 66fb567, and this repair would have inherited it. Standing matches idx-27's addendum: the standard message is documented in both faqs-generative-orchestration and faqs-generative-answers (the latter with the broader \"best practice to communicate to users that the agent uses artificial intelligence\" framing), so it carries no scope qualifier - unlike the confirmation safeguard, which does. Cross-file grep before editing: \"Powered by AI\" occurred nowhere else in the corpus. Two neighbours were checked and deliberately NOT swept in - the Foundry agent-transparency bullet asserting a \"This chatbot uses AI\" embeddable component, and the scenario-1 tooling line naming a \"Copilot Studio disclosure widget\" where this file's own Copilot Studio section documents a pre-built \"AI disclosure\" topic in the generative AI toolkit, not a widget. Both are unverified product artefacts in other sections against other sources; they are booked as idx-27c and idx-27d rather than repaired here."
},
{
"id": "idx-26e",
@ -152,13 +153,38 @@
"id": "idx-26f",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"status": "resolved",
"raised": "2026-08-03",
"summary": "Surfaced while writing idx-26d, and deliberately excluded from it. Items 2 and 5 of the scorecard enumeration carry specifics the live source does not state: the fairness insights segment is described with \"(gender, ethnicity, age)\" where the source says only \"your desired sensitive groups\", and the data analysis segment with \"missing values, outlier analysis\" where the source says only that it \"shows you characteristics of your data\". Neither is false on its face - both are plausible instances - but both sit under the **Confidence:** Verified (MCP: microsoft-learn, 2026-08-03) stamp, which now vouches for an enumeration re-verified against the live source. This is a distinct class from idx-26d's name drift and idx-26b's cross-attribution: unsourced SPECIFICITY under a verification marker, where the defect is the marker's reach rather than the sentence. idx-26d's resolution states explicitly that its widening of the stamp does not cover items 2, 4 and 5, so the stamp is currently honest only because that exclusion is written down - closing this entry is what would make it honest on its own terms. Item 4's description was checked in the same pass and is a recognisable paraphrase of the source, not added specificity, so it is named here as checked-and-clear rather than left ambiguous.",
"evidence": "docs/r11-pilot-results.md §9.8; https://learn.microsoft.com/azure/machine-learning/how-to-responsible-ai-scorecard",
"anchors": [
"2. **Fairness insights**: Performance disparities across sensitive groups (gender, ethnicity, age)",
"5. **Data analysis**: Dataset statistics, missing values, outlier analysis"
],
"resolution": "Operator-ratified 2026-08-03, form (a): descriptions rewritten to the source's own wording, NOT surgical deletion of the named parentheticals. The source was re-fetched live before writing (how-to-responsible-ai-scorecard) and states only \"how well your model is satisfying the fairness target values you set for your desired sensitive groups\" and \"The data analysis segment shows you characteristics of your data\". Item 2 is now \"Hvor godt modellen moeter fairness-target values du har satt for de sensitive gruppene du velger\" and item 5 \"Karakteristikker ved dataene dine\" (both written with correct Norwegian diacritics in the corpus file). Deleting only the specificity this entry NAMED was rejected on two counts, each of which would have made the repair inherit the defect class it was raised against: it leaves item 5 as \"Dataset statistics\", which the source does not state either and which this entry did NOT adjudicate as checked-and-clear the way it did item 4; and it leaves item 2 as \"across sensitive groups\", dropping the you-choose-them agency and stranding item 2 in the pre-idx-26d dialect while items 1, 3, 6 and 7 carry \"du har satt\" - reopening the mixed-dialect defect idx-26d was booked under, in the file idx-26d had just canonicalised. Form (b) - narrowing the stamp's stated reach in the file instead - was put to the operator and declined: it is honest-by-annotation relocated from this queue's resolution field into the corpus, i.e. the same mechanism this entry exists to remove the need for. Item 4 was left untouched per this entry's own adjudication. Cross-file grep before editing: \"gender, ethnicity, age\", \"outlier analysis\" and \"Dataset statistics\" occurred nowhere else in the corpus. The enumeration was re-read whole after the edit for coherence, not just for the two edited strings. SUPERSEDES the closing clause of idx-26d's resolution: what the **Confidence:** Verified (MCP: microsoft-learn, 2026-08-03) stamp vouches for is now a seven-member enumeration whose names are all the source's own segment names, whose descriptions for items 1, 2, 3, 5, 6 and 7 were verified against the live source on the date the stamp carries, and whose item 4 description was adjudicated a recognisable paraphrase of the source's causal-insights passage. No exception is written down anywhere for the stamp to remain honest."
},
{
"id": "idx-27c",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"raised": "2026-08-03",
"summary": "Surfaced by the neighbourhood sweep for idx-27b and deliberately excluded from it. The Microsoft Foundry \"Agent transparency\" list asserts \"Disclosure widgets: \\\"This chatbot uses AI\\\" embeddable component\" - a named product artefact carrying a verbatim-looking user-facing string. Same class as idx-27 and idx-27b (a plausible disclosure phrasing asserted as a built-in surface), but a different platform section against a different source, so it cannot be repaired by copying the ratified Copilot Studio formulation. Needs its own live fetch against Foundry documentation to establish whether an embeddable disclosure widget exists and, if so, what string it actually carries. Repair is a replacement, outside the delete-only envelope.",
"evidence": "docs/r11-pilot-results.md §9.10",
"anchors": [
"- Disclosure widgets: \"This chatbot uses AI\" embeddable component"
]
},
{
"id": "idx-27d",
"file": "skills/ms-ai-governance/references/responsible-ai/transparency-documentation-standards.md",
"class": "replacement",
"status": "open",
"raised": "2026-08-03",
"summary": "Surfaced by the neighbourhood sweep for idx-27b and deliberately excluded from it. The scenario 1 tooling line names a \"Copilot Studio disclosure widget\", but this same file's Copilot Studio section documents a pre-built \"AI disclosure\" TOPIC in the generative AI toolkit - not a widget. This is intra-file name drift of the same shape as idx-26d, not a source-attribution error: the file contradicts itself about what the artefact is called. Note the context differs from idx-27/idx-27b - this line sits inside a worked scenario recommendation rather than a product-capability claim, so the repair should decide whether to name the toolkit topic or drop the artefact name, not merely restate a source. Repair is a replacement, outside the delete-only envelope.",
"evidence": "docs/r11-pilot-results.md §9.10",
"anchors": [
"**Tooling:** Azure OpenAI Transparency Note + Copilot Studio disclosure widget"
]
}
]