open/ms-ai-architect
1
0
Fork 0
Code Issues Releases Activity

fix(ms-ai-architect): idx-27 scope-addendum — orkestrerings-FAQ-ens rekkevidde skilt fra generative answers

Browse source 
Adversarielt gjenlest resolution avdekket at BEGGE reparerte påstander ble
skrevet inn under overskriften «Microsoft Copilot Studio» uten kvalifikator,
mens FAQ-ens egen scope-linje er «generative orchestration». Samme utvidings-
klasse som idx-27 ble reist for — reprodusert av sin egen reparasjon.

Sjekket mot dokumentasjonen, og de to påstandene falt fra hverandre:
- bekreftelses-sikringen står KUN i orkestrerings-FAQ-en → kvalifikator lagt til
- standardmeldingen står ordrett også i faqs-generative-answers, med bredere
  ramme → raden beholder ingen kvalifikator, og den andre FAQ-en er lagt til
  som sitert kilde så lesningen kan etterprøves

Standing: grounded-as-cited, ikke established-for-all-agents.
Kø: 6 åpne / 6 resolved. Suite 1047/1047.

[skip-docs]
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 22:14:42 +02:00
commit 66fb567ec6
3 changed files with 30 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

4
scripts/kb-eval/data/g7-review-queue.json
View file

@ -78,12 +78,12 @@
"status": "resolved",
"raised": "2026-08-03",
"summary": "Failed out of O2 on cond 3 (§9.6): the cited source DOES establish the mechanism — faqs-generative-orchestration states 'Makers can require user confirmation before executing tools that modify data' — so deleting the Plugin-actions row would destroy source-confirmed information. The defect is modality, not fabrication: the file presents confirmation prompts as a built-in disclosure, whereas the source makes them maker-configured. The Chat-interface row in the same table is imprecise for the same reason: the FAQ documents a default transparency message ('Just so you are aware, I sometimes use AI to answer your questions.'), not a 'Powered by AI' badge. Repair is a replacement, outside the delete-only envelope.",
"evidence": "docs/r11-pilot-results.md §9.6; https://learn.microsoft.com/microsoft-copilot-studio/faqs-generative-orchestration",
"evidence": "docs/r11-pilot-results.md §9.6; https://learn.microsoft.com/microsoft-copilot-studio/faqs-generative-orchestration; https://learn.microsoft.com/microsoft-copilot-studio/faqs-generative-answers",
"anchors": [
"| **Plugin actions** | Confirmation prompts før sensitive actions (send email, delete file) |",
"| **Chat interface** | \"Powered by AI\" badge i chat window |"
],
"resolution": "Operator-ratified 2026-08-03. Open question #8 (does a hub page's linked children count as \"the source\") was answered NO - the strict reading R11 has held throughout stands, and no new standing grounding rule was introduced. idx-27 was instead resolved by making the child a CITED source: the Responsible AI FAQ block now lists both the hub (responsible-ai-overview) and faqs-generative-orchestration, naming the latter as the source for the two repaired claims. The FAQ was fetched live before writing; both quotes are verbatim from it. Chat-interface row: \"Powered by AI\" badge replaced with the default transparency message the source actually documents - \"Just so you are aware, I sometimes use AI to answer your questions.\" - which the source states agents include, so it belongs under Built-in disclosures. Plugin-actions row: the source says \"Makers can require user confirmation before executing tools that modify data\", a configurable safeguard, so the row was MOVED OUT of the Built-in disclosures table into a new \"Maker-konfigurerte kontroller (ikke innebygde disclosures)\" block. Repairing the text in place would have left the false modality asserted by the table heading; weakening the heading instead was rejected because it would have silently restated the modality of the two unmeasured rows (Generative answers, Data usage), and adding a modality column would have asserted \"built-in\" about them outright - creating new unverified claims to fix an old one. The deleted \"(send email, delete file)\" examples were not carried over: the source names neither. The same \"Powered by AI\" imprecision at line 218, in the Azure implementasjon list of Mønster 3, is outside this entry's anchors and is booked as idx-27b rather than swept in - the idx-26c/26d precedent. No **Confidence:** stamp covers the Copilot Studio section, so no marker obligation was triggered by this edit."
"resolution": "Operator-ratified 2026-08-03. Open question #8 (does a hub page's linked children count as \"the source\") was answered NO - the strict reading R11 has held throughout stands, and no new standing grounding rule was introduced. idx-27 was instead resolved by making the child a CITED source: the Responsible AI FAQ block now lists both the hub (responsible-ai-overview) and faqs-generative-orchestration, naming the latter as the source for the two repaired claims. The FAQ was fetched live before writing; both quotes are verbatim from it. Chat-interface row: \"Powered by AI\" badge replaced with the default transparency message the source actually documents - \"Just so you are aware, I sometimes use AI to answer your questions.\" - which the source states agents include, so it belongs under Built-in disclosures. Plugin-actions row: the source says \"Makers can require user confirmation before executing tools that modify data\", a configurable safeguard, so the row was MOVED OUT of the Built-in disclosures table into a new \"Maker-konfigurerte kontroller (ikke innebygde disclosures)\" block. Repairing the text in place would have left the false modality asserted by the table heading; weakening the heading instead was rejected because it would have silently restated the modality of the two unmeasured rows (Generative answers, Data usage), and adding a modality column would have asserted \"built-in\" about them outright - creating new unverified claims to fix an old one. The deleted \"(send email, delete file)\" examples were not carried over: the source names neither. The same \"Powered by AI\" imprecision at line 218, in the Azure implementasjon list of Mønster 3, is outside this entry's anchors and is booked as idx-27b rather than swept in - the idx-26c/26d precedent. No **Confidence:** stamp covers the Copilot Studio section, so no marker obligation was triggered by this edit. Scope addendum, same session, after an adversarial read of the resolution above: both repaired claims were taken from a FAQ whose own scope line reads \"the AI impact of generative orchestration for custom agents built in Copilot Studio\", and they were written into a section headed Microsoft Copilot Studio with no qualifier - the same widening class this entry was raised for. Re-checked against the docs rather than reasoned about, and the two claims came apart. The maker-confirmation safeguard occurs ONLY in the generative-orchestration FAQ, in a safeguards list about tool execution, so it is orchestration-scoped and the bullet now says so explicitly (\"for agenter med generativ orkestrering\"). The default transparency message occurs in that FAQ AND, verbatim, in faqs-generative-answers under \"What protections are in place within Copilot Studio for responsible AI?\", framed there as a general best practice rather than an orchestration feature. Two independent feature FAQs stating it without a feature qualifier is why the Built-in disclosures row carries no scope qualifier; faqs-generative-answers was added as a third cited source so the reader can check that reasoning rather than take it on trust. Neither FAQ is a Copilot-Studio-wide statement of record, so this is grounded-as-cited, not established-for-all-agents - if a stronger claim is ever wanted, it needs a source that says so."
},
{
"id": "idx-36",