open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 4 Activity

docs(architect): KB follow-up — batch 3 content updates

Browse source 
Additional factual updates from batch 3 research:

- responsible-ai-training-awareness.md: module renamed
  "Azure AI Studio" → "Microsoft Foundry" (3 occurrences)
- transparency-documentation-standards.md: ISO/IEC 42001 scope expanded
  to include Copilot Studio, Microsoft Foundry, Security Copilot,
  GitHub Copilot, Dragon Copilot
- ai-act-compliance-guide.md: same ISO 42001 scope expansion
- human-in-the-loop-oversight.md: AI approval stages in Copilot Studio
  (GPT-o3 as AI approver, new Human in the loop connector)
- continuous-improvement-feedback-loops.md: MLflow 3 Feedback vs
  Expectation assessment types, Genie Code trace analysis

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-09 22:43:12 +02:00
commit ecc5433818
5 changed files with 12 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

6
skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md
View file

@ -30,8 +30,12 @@ Microsoft implementerer feedback loops gjennom hele AI-livssyklusen – fra utvi
**Tracing og logging:**
- **MLflow Traces** / **MLflow 3 GenAI**: Fanger detaljerte execution traces med inputs, outputs og alle mellomsteg for hver interaksjon. *(Verified MCP 2026-04)*
- MLflow 3 GenAI introduserer ny **Feedback/Expectation-datamodell** for strukturert lagring av human feedback
- MLflow 3 GenAI introduserer ny **Assessment-datamodell** med to typer:
- **Feedback** assessments: evaluerer faktisk output (ratings, kommentarer — "Var agentens svar bra?")
- **Expectation** assessments: definerer ønsket/korrekt output (ground truth — "Hva burde ha blitt produsert"); brukes til å bygge evalueringsdata
- Tre innsamlingskilder: utvikler (dev), domeneekspert (via Review App), sluttbruker (produksjon)
- `mlflow.log_feedback()` API for å knytte bruker-rating og kommentarer til spesifikke traces
- Ny kapabilitet: **Genie Code** for naturspråk-analyse av trace-data
- Integrert tracing for Databricks agentic applikasjoner
- **Azure Monitor & Application Insights**: Logger operational metrics, latency, error rates
- **Model Data Collector**: Automatisk innsamling av production data for ML-modeller