open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity
ms-ai-architect/skills/ms-ai-advisor/references/architecture/poc-template.md
Kjell Tore Guttormsen 5a0e8d774a feat(ms-ai-architect): R21 — Status-backfill på 14 advisor-ref-filer (redusert scope etter premiss-korreksjon) [skip-docs] 
Manifest-drevet applier (scripts/kb-update/backfill-status.mjs) over den testede
insertMetaField-primitiven + ny ren regel statusForFile (filnavn-token → Reference/
Established Practice, operatør-godkjent vokabular). 7 Reference + 7 Established Practice.
Hard per-fil-invariant (én linje, body byte-identisk), idempotent, isMain-guard. 7 tester.

Premiss-korreksjon (auditHeaders, ground-truth 2026-07-06): ekte not-due-restanse er
21 Status + 26 Last-updated (STATE sa 21/22). «0 har norsk dato» var falskt — 21/26
Last-updated-gap bærer allerede Sist oppdatert/Dato → relabel-residual; 5 datoløse gir
«i dag» ved naiv git → utsatt. 4 dual-header ai-act-*-filer med plain-text «Status: GA»
fanget i diff-inspeksjon → revertert (unngår duplikat/motsigelse) → residual (samme
plain-blokk ga R20 duplikat Category). Korrigert residual logget i roadmap §R21.

Verifisering: test-backfill-status 7/7; git diff +14/-0 (kun Status-linjer, body
byte-identisk); not-due Status-missing 21→3; full suite 771/771 exit 0.
2026-07-06 09:41:40 +02:00

947 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# POC Template - Microsoft AI Projects
**Last updated:** 2026-06-24 (research via microsoft-learn MCP)
**Category:** Solution Architecture & Advisory
**Status:** Reference
---
Dette dokumentet tilbyr en strukturert mal for å planlegge, gjennomføre og evaluere Proof of Concept (POC) prosjekter for Microsoft AI-løsninger. Malen er tilpasset Microsoft Foundry, Copilot Studio, Power Platform AI, og andre Microsoft AI-plattformer.
## Innhold
1. [POC Plan Template](#poc-plan-template)
2. [Success Criteria Framework](#success-criteria-framework)
3. [Evaluation Rubric](#evaluation-rubric)
4. [Platform-Specific Checklists](#platform-specific-checklists)
5. [Risk Assessment Template](#risk-assessment-template)
6. [Timeline Templates](#timeline-templates)
7. [Stakeholder Communication Template](#stakeholder-communication-template)
8. [Go/No-Go Decision Framework](#gono-go-decision-framework)
9. [Example POC Plan](#example-poc-plan)
---
## POC Plan Template
Bruk denne malen for å strukturere din POC-plan. Fyll ut hver seksjon basert på ditt spesifikke use case.
### 1. Executive Summary
**Hensikt med POC:**
_[1-2 setninger: Hva skal POC bevise eller validere?]_
**Forventet varighet:** _[1 uke / 2 uker / 4 uker]_
**Estimert ressursbehov:** _[Antall personer, roller, budsjett]_
**Beslutningspunkt:** _[Dato for go/no-go beslutning]_
---
### 2. Business Case
#### 2.1 Problem Statement
_[Beskriv forretningsproblemet eller ineffektiviteten som AI kan løse.]_
**Eksempel:**
> Kundestøtte bruker 40% av tiden på repetitive spørsmål om ordrestatus, produktreturer og leveringsinformasjon. Dette binder opp ressurser som kunne brukes på mer komplekse kundehenvendelser.
#### 2.2 Target Outcome
_[Hva er det ønskede resultatet? Vær konkret og målbar.]_
**Eksempel:**
> Automatisere 60% av repetitive kundehenvendelser via chatbot, redusere gjennomsnittlig responstid fra 15 minutter til 2 minutter, og frigjøre 16 timer per uke for støtteteamet.
#### 2.3 Strategic Value
Ranger strategisk verdi (1-5, hvor 5 er høyest):
- **Business Impact:** _[1-5]_ — Hvor stor påvirkning har dette på forretningen?
- **User Value:** _[1-5]_ — Hvor mye verdi gir dette til sluttbrukere?
- **Innovation Potential:** _[1-5]_ — Hvor innovativt er dette for organisasjonen?
- **Strategic Alignment:** _[1-5]_ — Hvor godt aligner dette med organisasjonens AI-strategi?
---
### 3. Technical Scope
#### 3.1 AI Maturity Assessment
Identifiser din organisasjons AI-modningsnivå (basert på Microsoft CAF):
| Level | Skills Required | Data Readiness | Feasible Use Cases |
|-------|-----------------|----------------|-------------------|
| **Level 1** | Basic AI-forståelse, data-integrasjon | Minimal data, enterprise data tilgjengelig | Azure quickstart, Copilot-løsninger |
| **Level 2** | Model selection, deployment, data cleaning | Små strukturerte datasett, domene-spesifikk data | Analytical AI (Foundry Tools), Custom gen AI chat uten RAG, Fine-tuning |
| **Level 3** | Prompt engineering, data chunking, preprocessing | Store historiske datasett, domene-spesifikk data | Gen AI med RAG, ML model training, small AI models på VMs |
| **Level 4** | Advanced AI/ML, infra management, orchestration | Store treningsdatasett | Large gen AI/ML apps på VMs, AKS, Container Apps |
**Din organisasjon er på:** _[Level 1 / 2 / 3 / 4]_
#### 3.2 Chosen AI Solution
_[Velg én eller flere:]_
- [ ] **Microsoft 365 Copilot** (extensions/agents)
- [ ] **Copilot Studio** (custom agents)
- [ ] **Microsoft Foundry** (custom gen AI apps)
- [ ] **Power Platform AI** (AI Builder, Power Automate AI)
- [ ] **Azure Machine Learning** (custom ML models)
- [ ] **Analytical AI** (Content Safety, Document Intelligence, Custom Vision)
**Rationale:**
_[Hvorfor er denne løsningen valgt? Hva gjør den til det beste valget for dette use case?]_
#### 3.3 Data Requirements
**Data Sources:**
1. _[Source 1: Type, format, quality, accessibility]_
2. _[Source 2: Type, format, quality, accessibility]_
3. _[Source 3: ...]_
**Data Preparation Needed:**
- [ ] Data cleaning/normalization
- [ ] Data labeling
- [ ] Data chunking (for RAG)
- [ ] Privacy/security review (PII removal, anonymization)
- [ ] Data governance approvals
**Estimated Data Volume:**
_[Small (<100 MB) / Medium (100 MB - 10 GB) / Large (>10 GB)]_
#### 3.4 Infrastructure Requirements
**Compute:**
- [ ] Azure OpenAI capacity (model, region, quota)
- [ ] Azure Machine Learning compute (SKU, vCPUs, GPU)
- [ ] Power Platform capacity (Copilot Studio, AI Builder credits)
**Storage:**
- [ ] Azure Storage (type, size)
- [ ] Vector database (Azure AI Search, Cosmos DB)
**Network:**
- [ ] VNet integration
- [ ] Private endpoints
- [ ] Bandwidth requirements
**Security & Compliance:**
- [ ] Azure Policy enforcement
- [ ] Content Safety filters
- [ ] Data residency requirements
- [ ] Authentication/authorization (Entra ID, RBAC)
---
### 4. Success Criteria
Definer spesifikke, målbare kriterier for POC-suksess. Se [Success Criteria Framework](#success-criteria-framework) for detaljerte KPIer.
#### 4.1 Technical Success Criteria
1. **Functional Requirements:**
- _[Eksempel: Chatbotten skal kunne svare på 80% av testspørsmålene korrekt.]_
- _[Eksempel: Systemet skal kunne håndtere 100 samtidige forespørsler.]_
2. **Performance Metrics:**
- **Response Time:** _[Target: < 3 sekunder for 95% av forespørslene]_
- **Accuracy:** _[Target: 85% nøyaktighet på validasjonsdatasett]_
- **Availability:** _[Target: 99% uptime under testperioden]_
3. **Quality Metrics (for Gen AI):**
- **Groundedness:** _[Target: 90% av svarene skal være faktabaserte]_
- **Relevance:** _[Target: 85% av svarene skal være relevante for brukerens spørsmål]_
- **Content Safety:** _[Target: 0% harmful content, 100% moderate risk filtered]_
#### 4.2 Business Success Criteria
1. **Efficiency Gains:**
- _[Eksempel: Redusere behandlingstid med 50%]_
2. **Cost Savings:**
- _[Eksempel: Redusere driftskostnader med 20%]_
3. **User Satisfaction:**
- _[Eksempel: Oppnå 70% user satisfaction score]_
#### 4.3 Responsible AI Criteria
- [ ] **Fairness:** Løsningen skal ikke diskriminere basert på alder, kjønn, etnisitet, etc.
- [ ] **Transparency:** Brukere skal forstå når de interagerer med AI
- [ ] **Privacy:** Persondata skal beskyttes i henhold til GDPR/compliance-krav
- [ ] **Accountability:** Klare roller og ansvar for AI-beslutninger
- [ ] **Safety:** Content Safety filters implementert og testet
- [ ] **Inclusiveness:** Løsningen skal fungere for alle brukergrupper
---
### 5. Implementation Plan
#### 5.1 Phases & Milestones
**Phase 1: Prepare (Duration: _[X dager]_)**
- [ ] Data collection and preparation
- [ ] Environment setup (Azure, Power Platform)
- [ ] Team onboarding
- [ ] Security/compliance approvals
**Deliverable:** _[Data ready for use, infrastructure provisioned]_
---
**Phase 2: Build (Duration: _[X dager]_)**
- [ ] Develop initial prototype/POC
- [ ] Implement core functionality
- [ ] Integrate data sources
- [ ] Configure AI model/agent
**Deliverable:** _[Working prototype in dev environment]_
---
**Phase 3: Evaluate & Iterate (Duration: _[X dager]_)**
- [ ] Functional testing
- [ ] Performance testing
- [ ] Responsible AI testing (fairness, safety, bias)
- [ ] User acceptance testing (UAT)
- [ ] Iterate based on feedback
**Deliverable:** _[Validated POC with test results]_
---
**Phase 4: Document & Decide (Duration: _[X dager]_)**
- [ ] Document lessons learned
- [ ] Compile evaluation report
- [ ] Prepare go/no-go recommendation
- [ ] Present to stakeholders
**Deliverable:** _[POC report + go/no-go decision]_
---
#### 5.2 Team Roles & Responsibilities
| Role | Responsible For | Time Commitment |
|------|-----------------|-----------------|
| **Project Lead** | Overall POC coordination, stakeholder communication | _[X hours/week]_ |
| **Solution Architect** | Technical design, platform selection | _[X hours/week]_ |
| **Data Scientist/Engineer** | Data preparation, model evaluation | _[X hours/week]_ |
| **Developer/Maker** | Building prototype (Copilot Studio, Power Platform, code) | _[X hours/week]_ |
| **Subject Matter Expert (SME)** | Domain knowledge, validation | _[X hours/week]_ |
| **Security/Compliance Officer** | Responsible AI review, compliance validation | _[X hours/week]_ |
| **End-User Representative** | User testing, feedback | _[X hours/week]_ |
---
### 6. Testing & Validation Plan
#### 6.1 Functional Testing
- [ ] Unit tests for individual components
- [ ] Integration tests for data pipelines
- [ ] End-to-end scenario testing
**Test Cases:** _[Liste av testscenarier, f.eks. "User asks about order status"]_
#### 6.2 Performance Testing
- [ ] Load testing (concurrent users/requests)
- [ ] Latency testing (response times)
- [ ] Throughput testing (requests per second)
#### 6.3 Responsible AI Testing
- [ ] **Fairness Assessment:** Test på diverse brukergrupper
- [ ] **Content Safety:** Test adversarial prompts (jailbreak, harmful content)
- [ ] **Bias Detection:** Evaluate model outputs for bias
- [ ] **Explainability:** Validate that model decisions are understandable
**Tools:**
- Microsoft Foundry evaluation tools
- Azure AI Content Safety
- Responsible AI Dashboard (Azure ML)
#### 6.4 User Acceptance Testing (UAT)
- [ ] Recruit representative users
- [ ] Define UAT scenarios
- [ ] Collect qualitative feedback (surveys, interviews)
- [ ] Measure user satisfaction (NPS, CSAT)
---
### 7. Risk Management
Se [Risk Assessment Template](#risk-assessment-template) for detaljert risikovurdering.
**High-Priority Risks:**
1. _[Risk 1: Description + Mitigation Plan]_
2. _[Risk 2: Description + Mitigation Plan]_
3. _[Risk 3: Description + Mitigation Plan]_
---
### 8. Budget & Resources
**Estimated Costs:**
| Category | Estimated Cost | Notes |
|----------|---------------|-------|
| **Azure Compute** | _[NOK/USD]_ | OpenAI quota, VM SKUs, AML compute |
| **Storage** | _[NOK/USD]_ | Blob Storage, AI Search |
| **Licensing** | _[NOK/USD]_ | Copilot Studio, Power Platform |
| **Personnel** | _[NOK/USD]_ | Team member time (internal/external) |
| **Contingency (20%)** | _[NOK/USD]_ | Buffer for unexpected costs |
| **TOTAL** | _[NOK/USD]_ | |
---
### 9. Go/No-Go Decision Criteria
Ved slutten av POC, evaluer mot disse kriteriene:
- [ ] **Technical Feasibility:** Løsningen fungerer som forventet (>80% success criteria oppfylt)
- [ ] **Business Value:** ROI er positiv, eller verdi er dokumentert
- [ ] **User Acceptance:** Brukere er fornøyde (>70% satisfaction)
- [ ] **Responsible AI:** Ingen kritiske fairness/safety issues
- [ ] **Risk Acceptable:** Identifiserte risikoer kan håndteres
- [ ] **Budget Viable:** Production deployment er innenfor budsjett
**Decision:** _[GO / NO-GO / CONDITIONAL GO (specify conditions)]_
---
## Success Criteria Framework
### Technical KPIs (Generative AI)
| Metric | Definition | Target Range | Measurement Method |
|--------|------------|--------------|-------------------|
| **Groundedness** | % of responses supported by source data | >85% | Microsoft Foundry evaluation |
| **Relevance** | % of responses relevant to user query | >80% | Microsoft Foundry evaluation |
| **Fluency** | % of responses that are coherent and grammatical | >90% | Microsoft Foundry evaluation |
| **Content Safety** | % of harmful content blocked | 100% | Azure AI Content Safety |
| **Response Time** | Average latency (seconds) | <3s (p95) | Application Insights |
| **Throughput** | Requests per second handled | >100 rps | Load testing |
| **Availability** | Uptime during test period | >99% | Azure Monitor |
### Business KPIs
| Metric | Definition | Target | Measurement Method |
|--------|------------|--------|-------------------|
| **Time Saved** | Hours saved per week | _[X hours]_ | Before/after comparison |
| **Cost Reduction** | % reduction in operational costs | _[X%]_ | Financial analysis |
| **User Satisfaction (CSAT)** | Customer satisfaction score (1-5) | >4.0 | Survey |
| **Net Promoter Score (NPS)** | Likelihood to recommend (0-10) | >7.0 | Survey |
| **Task Completion Rate** | % of user tasks successfully completed | >80% | Analytics |
| **Adoption Rate** | % of target users actively using solution | >60% | Usage analytics |
### Responsible AI KPIs
| Metric | Definition | Target | Measurement Method |
|--------|------------|--------|-------------------|
| **Fairness (Demographic Parity)** | Max difference in positive prediction rates across groups | <10% | Responsible AI Dashboard |
| **Bias Detection** | No significant bias detected in outputs | 0 critical issues | Manual review + automated tools |
| **Privacy Compliance** | % of PII correctly handled (removed/anonymized) | 100% | Data audit |
| **Content Safety Pass Rate** | % of responses passing content safety filters | 100% | Azure AI Content Safety |
| **Explainability Score** | % of users who understand AI decisions | >70% | User survey |
---
## Evaluation Rubric
Bruk denne matrisen for å score POC-resultater:
### Technical Performance
| Criterion | Score 1 (Poor) | Score 3 (Fair) | Score 5 (Good) | Score 7 (Excellent) | Score |
|-----------|---------------|----------------|----------------|---------------------|-------|
| **Accuracy/Quality** | <60% | 60-74% | 75-89% | ≥90% | _[X]_ |
| **Performance** | Frequent failures, >5s latency | Occasional failures, 3-5s latency | Stable, 2-3s latency | Highly stable, <2s latency | _[X]_ |
| **Reliability** | <95% uptime | 95-97% uptime | 97-99% uptime | >99% uptime | _[X]_ |
| **Scalability** | Cannot scale beyond POC | Limited scalability | Scales to production | Easily scales | _[X]_ |
**Technical Score:** _[Sum / 28]_ → _[%]_
---
### Business Value
| Criterion | Score 1 (Poor) | Score 3 (Fair) | Score 5 (Good) | Score 7 (Excellent) | Score |
|-----------|---------------|----------------|----------------|---------------------|-------|
| **Efficiency Gains** | <20% improvement | 20-40% | 40-60% | >60% | _[X]_ |
| **User Satisfaction** | <50% satisfied | 50-65% | 65-80% | >80% | _[X]_ |
| **Cost-Effectiveness** | ROI negative | ROI break-even | ROI 1-2x | ROI >2x | _[X]_ |
| **Strategic Fit** | Misaligned | Partially aligned | Well aligned | Critical priority | _[X]_ |
**Business Score:** _[Sum / 28]_ → _[%]_
---
### Responsible AI
| Criterion | Score 1 (Poor) | Score 3 (Fair) | Score 5 (Good) | Score 7 (Excellent) | Score |
|-----------|---------------|----------------|----------------|---------------------|-------|
| **Fairness** | Significant bias issues | Minor bias detected | Fair across groups | Highly fair | _[X]_ |
| **Safety** | Harmful content generated | Moderate safety issues | Safe with minor exceptions | 100% safe | _[X]_ |
| **Privacy** | PII leaks detected | Minor privacy concerns | Privacy compliant | Exceeds compliance | _[X]_ |
| **Transparency** | Opaque, users confused | Somewhat transparent | Transparent | Highly transparent | _[X]_ |
**Responsible AI Score:** _[Sum / 28]_ → _[%]_
---
### Overall POC Score
| Dimension | Weight | Score (%) | Weighted Score |
|-----------|--------|-----------|----------------|
| Technical Performance | 40% | _[X%]_ | _[X]_ |
| Business Value | 40% | _[X%]_ | _[X]_ |
| Responsible AI | 20% | _[X%]_ | _[X]_ |
| **TOTAL** | **100%** | | **_[X%]_** |
**Recommendation:**
- **>80%:** Strong GO — Proceed to production
- **60-80%:** Conditional GO — Address gaps before production
- **<60%:** NO-GO — Re-evaluate or pivot
---
## Platform-Specific Checklists
### Copilot Studio POC Checklist
**Pre-Development:**
- [ ] Define agent scope (which topics/intents)
- [ ] Identify data sources for grounding (SharePoint, Dataverse, APIs)
- [ ] Determine deployment channels (Teams, website, custom)
- [ ] Configure Copilot Studio environment (dev, test, prod)
- [ ] Set up authentication (if required)
**Development:**
- [ ] Build initial topics using conversational design best practices
- [ ] Configure generative orchestration (if using gen AI)
- [ ] Integrate data sources (connections, AI Search)
- [ ] Implement content moderation (Azure AI Content Safety)
- [ ] Test conversation flows with representative users
**Evaluation:**
- [ ] Test intent recognition accuracy
- [ ] Measure conversation abandonment rate
- [ ] Validate grounding accuracy (if using data sources)
- [ ] Test escalation paths (handoff to human)
- [ ] Collect user feedback via surveys
**Governance:**
- [ ] Apply content filters (Azure Policy)
- [ ] Configure security groups (Entra ID)
- [ ] Review compliance (data residency, privacy)
- [ ] Document agent behavior and limitations
---
### Microsoft Foundry POC Checklist
**Pre-Development:**
- [ ] Select foundation model (GPT-4o, GPT-4, custom)
- [ ] Provision Azure OpenAI capacity (region, quota)
- [ ] Define prompt engineering strategy
- [ ] Identify grounding data (if RAG)
- [ ] Set up Azure AI Search (if RAG)
**Development:**
- [ ] Build prompt flow orchestration
- [ ] Implement RAG pipeline (chunking, embedding, retrieval)
- [ ] Configure content safety filters
- [ ] Develop evaluation dataset (test queries + expected outputs)
- [ ] Deploy to pre-production endpoint
**Evaluation:**
- [ ] Run Microsoft Foundry evaluation suite (groundedness, relevance, fluency)
- [ ] Test adversarial prompts (jailbreak attempts)
- [ ] Measure latency and throughput
- [ ] Validate cost per request
- [ ] Collect SME feedback on output quality
**Governance:**
- [ ] Enforce Azure Policy (allowed models, regions)
- [ ] Configure RBAC for deployment
- [ ] Enable monitoring (Application Insights, Azure Monitor)
- [ ] Document model version and configuration
---
### Power Platform AI (AI Builder) POC Checklist
**Pre-Development:**
- [ ] Identify AI Builder capability (document processing, text classification, object detection)
- [ ] Prepare training data (labeled datasets)
- [ ] Validate Power Platform capacity (AI Builder credits)
- [ ] Define integration points (Power Apps, Power Automate)
**Development:**
- [ ] Train AI Builder model
- [ ] Validate model accuracy on test dataset
- [ ] Build Power Automate flow or Power App integration
- [ ] Test end-to-end automation
**Evaluation:**
- [ ] Measure model precision/recall
- [ ] Test on real-world data
- [ ] Validate processing speed
- [ ] Collect user feedback
**Governance:**
- [ ] Configure DLP policies
- [ ] Review data residency
- [ ] Document model performance metrics
---
## Risk Assessment Template
### Risk Identification Matrix
| Risk Category | Risk Description | Likelihood (1-5) | Impact (1-5) | Risk Score (L×I) | Mitigation Plan |
|---------------|------------------|------------------|--------------|------------------|-----------------|
| **Technical** | _[Example: Model accuracy below target]_ | _[X]_ | _[X]_ | _[X]_ | _[Retrain with more data, fine-tune prompts]_ |
| **Data** | _[Example: Insufficient training data]_ | _[X]_ | _[X]_ | _[X]_ | _[Synthetic data generation, expand data sources]_ |
| **Security** | _[Example: PII leakage in outputs]_ | _[X]_ | _[X]_ | _[X]_ | _[Implement PII detection, anonymization]_ |
| **Compliance** | _[Example: GDPR violation]_ | _[X]_ | _[X]_ | _[X]_ | _[Legal review, data residency controls]_ |
| **Organizational** | _[Example: Lack of user adoption]_ | _[X]_ | _[X]_ | _[X]_ | _[Change management, training, communication]_ |
| **Budget** | _[Example: Cost overruns]_ | _[X]_ | _[X]_ | _[X]_ | _[Monitor spending, set cost alerts]_ |
| **Responsible AI** | _[Example: Bias in model outputs]_ | _[X]_ | _[X]_ | _[X]_ | _[Fairness testing, diverse training data]_ |
**Risk Scoring:**
- 1-5: Low risk (monitor)
- 6-10: Medium risk (active mitigation required)
- 11-25: High risk (escalate, consider showstopper)
---
### Common AI POC Risks & Mitigations
#### Technical Risks
**Risk:** Model decay over time (accuracy degrades)
- **Mitigation:** Implement continuous monitoring, plan for retraining cadence
**Risk:** Latency exceeds user expectations
- **Mitigation:** Optimize prompt length, use faster models, implement caching
**Risk:** Integration failures with existing systems
- **Mitigation:** Early integration testing, API contract validation
---
#### Data Risks
**Risk:** Data quality issues (missing, incomplete, biased data)
- **Mitigation:** Data profiling upfront, data cleaning pipelines, diverse data sources
**Risk:** Insufficient data volume for training
- **Mitigation:** Synthetic data generation, transfer learning, start with simpler models
**Risk:** Data access blocked by governance/compliance
- **Mitigation:** Early stakeholder engagement, privacy-preserving techniques (anonymization)
---
#### Security & Compliance Risks
**Risk:** Prompt injection attacks
- **Mitigation:** Input validation, content filtering, prompt engineering defenses
**Risk:** Data residency violations
- **Mitigation:** Use compliant Azure regions, review data flow architecture
**Risk:** Unauthorized data access
- **Mitigation:** RBAC, private endpoints, encryption at rest/in transit
---
#### Organizational Risks
**Risk:** User resistance to AI adoption
- **Mitigation:** Involve users early, transparent communication about AI capabilities/limitations
**Risk:** Insufficient team skills
- **Mitigation:** Training programs, external consultants, phased learning approach
**Risk:** Unclear ownership and accountability
- **Mitigation:** Define RACI matrix, establish AI governance board
---
## Timeline Templates
### 1-Week Sprint POC
**Anbefalt for:** Simple use cases (Copilot Studio med pre-built connectors, basic chatbot)
| Day | Activities | Deliverables |
|-----|------------|--------------|
| **Day 1** | Kickoff, scope definition, environment setup | Approved scope, dev environment ready |
| **Day 2-3** | Build prototype, integrate data sources | Working prototype |
| **Day 4** | Testing (functional, UAT) | Test results, feedback collected |
| **Day 5** | Document findings, prepare recommendation | POC report, go/no-go decision |
**Total Effort:** ~40 person-hours
---
### 2-Week Standard POC
**Anbefalt for:** Moderate complexity (Microsoft Foundry RAG, Copilot Studio med custom topics)
| Week | Activities | Deliverables |
|------|------------|--------------|
| **Week 1** | - Kickoff & planning (Day 1-2)<br>- Data preparation (Day 2-3)<br>- Environment setup (Day 3-4)<br>- Initial prototype build (Day 4-5) | Data ready, dev environment, initial prototype |
| **Week 2** | - Iterate on prototype (Day 1-2)<br>- Testing & validation (Day 3-4)<br>- Documentation & presentation (Day 5) | Validated POC, test results, final report, go/no-go decision |
**Total Effort:** ~80-120 person-hours
---
### 4-Week Extended POC
**Anbefalt for:** Complex use cases (Azure ML model training, multi-agent systems, advanced RAG)
| Week | Phase | Activities | Deliverables |
|------|-------|------------|--------------|
| **Week 1** | **Prepare** | - Kickoff, detailed planning<br>- Data collection & preparation<br>- Infrastructure setup<br>- Team onboarding | Data pipeline ready, infra provisioned, team aligned |
| **Week 2** | **Build** | - Develop core functionality<br>- Model training/fine-tuning<br>- Integration with systems | Working prototype (alpha) |
| **Week 3** | **Evaluate** | - Functional testing<br>- Performance testing<br>- Responsible AI evaluation<br>- User acceptance testing<br>- Iterate based on feedback | Validated prototype (beta), test reports |
| **Week 4** | **Decide** | - Final validation<br>- Documentation (lessons learned, architecture)<br>- Stakeholder presentation<br>- Go/no-go decision | POC final report, production roadmap, decision |
**Total Effort:** ~200-320 person-hours
---
### Timeline Adjustment Factors
Legg til ekstra tid hvis:
- [ ] **Data ikke er klar:** +1-2 uker for data cleaning, labeling
- [ ] **Komplekse integrasjoner:** +1 uke per critical integration
- [ ] **Compliance reviews:** +1-2 uker for legal/security approvals
- [ ] **New team to AI:** +1 uke for onboarding/training
- [ ] **Custom model training:** +2-4 uker for ML model development
**Anbefaling:** Legg til 20-30% buffer for uforutsette utfordringer.
---
## Stakeholder Communication Template
### POC Kickoff Email
**Subject:** [POC Name] - Kickoff & Plan
**To:** Project team, stakeholders
**Body:**
> Vi starter POC for [use case name] med mål om å [brief objective]. POC vil løpe fra [start date] til [end date] ([X uker]).
>
> **Mål:**
> - [Goal 1]
> - [Goal 2]
>
> **Team:**
> - Project Lead: [Name]
> - Solution Architect: [Name]
> - Developer: [Name]
>
> **Neste steg:**
> - [Action 1]
> - [Action 2]
>
> **Beslutningspunkt:** [Date for go/no-go decision]
>
> Spørsmål? Kontakt [Lead].
---
### Weekly Status Update Template
**Subject:** [POC Name] - Week [X] Status
**Progress This Week:**
- [Completed item 1]
- [Completed item 2]
**Blockers/Risks:**
- [Risk 1 + mitigation plan]
**Next Week:**
- [Planned item 1]
- [Planned item 2]
**On Track?** [Yes / No / At Risk]
---
### Final POC Report Template
**Executive Summary:**
- **Objective:** [What we set out to prove]
- **Outcome:** [What we learned/achieved]
- **Recommendation:** [GO / NO-GO / CONDITIONAL GO]
**Technical Results:**
- Accuracy: [X%] (Target: [Y%])
- Performance: [X seconds] (Target: [Y seconds])
- [Other KPIs]
**Business Value:**
- Efficiency gains: [X hours/week saved]
- User satisfaction: [X% CSAT]
- Estimated ROI: [X]
**Responsible AI:**
- Fairness: [Pass/Fail + details]
- Safety: [Pass/Fail + details]
- Privacy: [Pass/Fail + details]
**Risks & Mitigations:**
- [Risk 1 + status]
- [Risk 2 + status]
**Next Steps:**
- If GO: [Production roadmap, timeline, budget]
- If NO-GO: [Reasons, alternative approaches]
**Attachments:**
- Test results
- User feedback summary
- Cost analysis
---
## Go/No-Go Decision Framework
### Decision Criteria Scorecard
| Category | Weight | Pass Threshold | Actual Score | Weighted | Pass? |
|----------|--------|---------------|--------------|----------|-------|
| **Technical Feasibility** | 30% | >75% | _[X%]_ | _[X]_ | _[Y/N]_ |
| **Business Value** | 30% | >70% | _[X%]_ | _[X]_ | _[Y/N]_ |
| **Responsible AI** | 20% | >80% | _[X%]_ | _[X]_ | _[Y/N]_ |
| **User Acceptance** | 10% | >70% | _[X%]_ | _[X]_ | _[Y/N]_ |
| **Risk Management** | 10% | No critical risks | _[X/5 risks mitigated]_ | _[X]_ | _[Y/N]_ |
| **TOTAL** | **100%** | **>75%** | | **_[X%]_** | **_[Y/N]_** |
---
### Decision Paths
#### GO Decision
**Criteria:**
- Overall score >75%
- All critical dimensions pass threshold
- No unmitigated high risks (score >15)
- Stakeholder approval obtained
**Next Steps:**
1. Finalize production architecture
2. Secure production budget
3. Define production roadmap (6-12 months)
4. Establish MLOps/GenAIOps processes
5. Plan change management/training
**Timeline to Production:** _[X weeks/months]_
---
#### CONDITIONAL GO Decision
**Criteria:**
- Overall score 60-75%
- Some dimensions below threshold
- High risks present but mitigatable
**Conditions to Meet:**
- _[Condition 1: e.g., Improve accuracy to 85% before production]_
- _[Condition 2: e.g., Complete security audit]_
- _[Condition 3: e.g., Obtain legal approval for data usage]_
**Re-evaluation Date:** _[Date]_
---
#### NO-GO Decision
**Criteria:**
- Overall score <60%
- Critical dimension failures
- Unmitigatable high risks
- Stakeholder concerns unresolved
**Reasons:**
- _[Reason 1]_
- _[Reason 2]_
**Alternatives:**
1. **Pivot:** Change approach (different platform, simpler use case)
2. **Delay:** Address blockers, re-run POC in [X months]
3. **Cancel:** Not viable, explore non-AI solutions
---
## Example POC Plan
### POC: Customer Support Chatbot (Copilot Studio)
**Executive Summary:**
- **Hensikt:** Automatisere repetitive kundehenvendninger (ordrestatus, returer, leveringsspørsmål) via chatbot i Teams og på web.
- **Varighet:** 2 uker
- **Ressurser:** 3 personer (1 solution architect, 1 developer, 1 SME)
- **Beslutningsdato:** 2025-02-14
---
**Business Case:**
**Problem:** Kundestøtte bruker 40% av tiden (16 timer/uke) på repetitive spørsmål. Gjennomsnittlig responstid er 15 minutter.
**Målsetting:**
- Automatisere 60% av repetitive henvendelser
- Redusere responstid til <2 minutter
- Frigjøre 10 timer/uke for støtteteamet
**Strategic Value:**
- Business Impact: 4/5 (betydelig effektivisering)
- User Value: 5/5 (raskere svar for kunder)
- Innovation: 3/5 (standard chatbot-løsning)
- Strategic Alignment: 4/5 (aligner med AI-strategi)
---
**Technical Scope:**
**AI Maturity:** Level 2 (har litt erfaring med Power Platform, basic AI-forståelse)
**Chosen Solution:** Copilot Studio
- Hvorfor: Low-code, rask utvikling, godt integrert med Teams/Dataverse, møter compliance-krav
**Data Sources:**
1. **Dataverse:** Ordredata (Order Status, Tracking Numbers)
2. **SharePoint:** FAQ-dokumenter, return policies
3. **Customer Service API:** Live order lookup
**Infrastructure:**
- Copilot Studio capacity: 1000 conversations/month
- Azure AI Search: For FAQ grounding
- Dataverse: For order data
- Content Safety: Azure AI Content Safety filters
---
**Success Criteria:**
**Technical:**
- Intent recognition accuracy: >85%
- Response time: <3 seconds
- Availability: >99%
- Content Safety: 100% pass rate
**Business:**
- Automation rate: >60% of test queries handled without human
- User satisfaction: >70% CSAT
- Cost per conversation: <5 NOK
**Responsible AI:**
- No bias in responses across customer demographics
- All PII handled securely
- Transparent AI disclosure to users
---
**Implementation Plan (2 weeks):**
**Week 1:**
- Day 1-2: Setup Copilot Studio environment, define topics (Order Status, Returns, Shipping)
- Day 3-4: Integrate Dataverse + SharePoint, configure gen AI orchestration
- Day 5: Build initial conversation flows
**Week 2:**
- Day 1-2: Test with internal users, iterate on prompts
- Day 3-4: User acceptance testing (10 customer service reps), collect feedback
- Day 5: Document results, prepare go/no-go recommendation
---
**Team:**
- **Project Lead:** Kari Nordmann (10 timer/uke)
- **Solution Architect:** Ola Hansen (15 timer/uke)
- **Developer (Copilot Studio):** Emma Larsen (20 timer/uke)
- **SME (Customer Service):** Per Johansen (5 timer/uke)
---
**Testing Plan:**
**Functional Tests:**
- Test all 3 main topics (Order Status, Returns, Shipping)
- Test escalation to human agent
**Performance:**
- Load test: 50 concurrent conversations
- Latency: Measure p50, p95, p99
**Responsible AI:**
- Test 20 adversarial prompts (jailbreak attempts)
- Validate content filters active
**UAT:**
- 10 customer service reps test for 2 days
- Survey: CSAT, ease of use, accuracy
---
**Risks:**
| Risk | Likelihood | Impact | Score | Mitigation |
|------|-----------|--------|-------|------------|
| Low intent recognition accuracy | 3 | 4 | 12 | Add more training phrases, use gen AI fallback |
| Dataverse integration delays | 2 | 3 | 6 | Start integration early, have mock data ready |
| User resistance (prefer human support) | 2 | 2 | 4 | Change management, involve users early |
---
**Budget:**
| Item | Cost |
|------|------|
| Copilot Studio license (1 month) | 5,000 NOK |
| Azure AI Search (dev tier) | 500 NOK |
| Personnel (80 hours × 1000 NOK/hr) | 80,000 NOK |
| **TOTAL** | **85,500 NOK** |
---
**Go/No-Go Criteria:**
- [ ] Intent accuracy >85%
- [ ] Response time <3s
- [ ] User satisfaction >70%
- [ ] No critical safety issues
- [ ] Budget for production <50,000 NOK/year
**Expected Outcome:** GO (90% confidence based on similar implementations)
---
## Vedlegg: Nyttige Ressurser
### Microsoft Documentation
- [AI Adoption Framework (CAF)](https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/ai/)
- [Copilot Studio Implementation Guidance](https://learn.microsoft.com/microsoft-copilot-studio/guidance/overview)
- [Microsoft Foundry Evaluation](https://learn.microsoft.com/azure/foundry/how-to/evaluate-generative-ai-app)
- [Responsible AI Standard](https://www.microsoft.com/ai/responsible-ai)
### Tools
- **Microsoft Foundry:** Model evaluation, deployment
- **Copilot Studio:** Agent development, testing
- **Azure AI Content Safety:** Content moderation
- **Responsible AI Dashboard:** Fairness, bias detection (Azure ML)
### Templates
- [AI Impact Assessment Template](https://www.microsoft.com/ai/tools-practices)
- [Responsible AI Maturity Model](https://www.microsoft.com/research/publication/responsible-ai-maturity-model/)
---
**Sist oppdatert:** 2026-06-24
**Versjon:** 1.0
**Eier:** AI Architect Plugin
Reference in a new issue View git blame Copy permalink