open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity

feat(ms-ai-architect): idx-33b-klassen adjudisert videre — 2 medlemmer ble 8, og limiter-hypotesen holdt ikke [skip-docs]

Browse source 
Ordre 20260818T190638Z-221629388: stege-2-adjudisering av resten av
idx-33b-klassen mot levende Learn-kilder. Ingen KB-fil roert (#29-porten
staar).

Nettene fra 18.08 var borte (scratchpad). Bygget opp igjen fra
vokabularreglene i idx-33c, og denne gangen SPORET:
scripts/kb-eval/data/idx-33-net-2026-08-20.json. Kjent-positiv-kontroll
kjoert FOER noen flagg ble lest — begge kjente medlemmer overlever
populasjonen. Reproduserer ikke 18.08s boetter og skal ikke: samme 283
stempler / 89 filer, men 223 paastands-passasjer (mot 226) og 32/19
container/limiter (mot 16/26). Vokabular, ikke korpus.

17 passasjer adjudisert mot levende kilder (11 container + 6 limiter),
haandplukket mot enumerasjoner og limitere — baerer INGEN rate.

6 nye medlemmer av under-paastandsklassen + 1 av annen klasse:
- idx-33e security-copilot-integration.md:126 — «M365 E5» der kilden sier
  «E5 and E7»; samme fil skriver E5/E7 tre steder fra samme kilde
- idx-33f data-leakage-prevention-ai.md:131 — gjengir 1 av kildens 2
  filtype-lister, dropper den andre + MP4
- idx-33g agent-365:43 — «Kun E7» der kilden sier «based on licensed
  subscription» og lister Agent 365 som egen lisens
- idx-33h agent-365:62 — «Kun Agent Builder» der kilden sier «Agent
  Builder and Copilot Studio»
- idx-33i ai-prompt-shield-network.md:304 — «Kun engelsk» der kilden
  lister aatte spraak
- idx-33j ai-prompt-shield-network.md:237 — 2 av 3 endepunkt-kategorier;
  «Self-hosted models and endpoints» mangler
- idx-33k alerting-strategies-escalation.md:76+84 — ANNEN KLASSE: sier
  Azure Function ikke stoetter managed identity, kilden sier Yes

Klassen har 8 bekreftede medlemmer, ikke 2. idx-33a-presedensen som
#29-spoersmaalet hviler paa beskriver ikke lenger denne klassen.
Limiter-hypotesen fra 18.08 er falsifisert som generell paastand: 3 av 6
adjudiserte limitere er smalere enn eller uten dekning i kilden.

Fortsatt UMAALT (ikke rent): 21 av 32 container- og 13 av 19
limiter-kandidater er aldri tatt til levende kilde.

check-g7-queue 0 drift. Suite 1052/1052.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-20 23:04:39 +02:00
commit f83824ff4d
2 changed files with 899 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

89
scripts/kb-eval/data/g7-review-queue.json
View file

File diff suppressed because one or more lines are too long

811
scripts/kb-eval/data/idx-33-net-2026-08-20.json Normal file
View file

@ -0,0 +1,811 @@
{
"_meta": {
"what": "Rebuilt idx-33b nets + the stage-2 adjudication verdicts, run 2026-08-20 under order 20260818T190638Z-221629388.",
"why_tracked": "The 2026-08-13 and 2026-08-18 nets both lived in a scratchpad and were both gone by the next session, so the class had to be re-measured from scratch twice. This file is the third attempt made durable. It is DATA, not a gate: nothing validates it and no test reads it.",
"not_reproducible_from_18_08": "Same corpus (283 stamps / 89 files) but this run's subtraction rules and vocabulary differ from 2026-08-18: 42 source-section + 18 link-entry subtractions -> 223 claim passages (18.08 measured 37 + 20 -> 226), and 32 container / 19 limiter (18.08: 16 / 26). Treat 18.08's numbers as unreproducible, not wrong.",
"known_positive_control": "Run BEFORE any flag was read. ai-threat-modeling-stride.md:210 -> container (\"across Azure subscriptions\"); digital-accessibility-action-plan.md:160 -> container (\"for K-12 laerere\"). Both PASS. A net that fails this control proves nothing.",
"sample_is_not_random": "The 17 stage-2 adjudications were hand-picked toward enumerations and explicit limiters - the stratum where narrowing is even possible. They carry NO rate. 21 of 32 container and 13 of 19 limiter candidates were never taken to a live source and are UNMEASURED, not clean.",
"population_rule": "For each line containing \"Verified MCP\": the passage is that line plus following lines until a blank line, a heading, or the next stamp. A passage is excluded if its nearest preceding heading matches /kilde|source|referanse|reference|further reading|videre lesning/i, or if the stamped line is itself a markdown link list entry."
},
"counts": {
"stamps": 283,
"files": 89,
"in_source_section": 42,
"link_entries": 18,
"claim_passages": 223,
"buckets": {
"container": 32,
"none": 170,
"limiter": 19,
"product-collision": 2
}
},
"vocabulary": {
"prepositions": [
"across",
"within",
"in",
"inside",
"for",
"per",
"to",
"under",
"på tvers av",
"innenfor",
"innen",
"i",
"for",
"til",
"per",
"hos",
"på"
],
"containers": [
"subscription",
"subscriptions",
"abonnement",
"tenant",
"tenants",
"tenanter",
"region",
"regions",
"regioner",
"environment",
"environments",
"miljø",
"miljøer",
"workspace",
"workspaces",
"arbeidsområde",
"resource group",
"ressursgruppe",
"management group",
"directory",
"katalog",
"geography",
"geografi",
"cloud",
"clouds",
"sky",
"skyer",
"platform",
"platforms",
"plattform",
"plattformer",
"service",
"services",
"tjeneste",
"tjenester",
"product",
"products",
"produkt",
"produkter",
"license",
"licenses",
"lisens",
"lisenser",
"sku",
"plan",
"plans",
"portal",
"portaler",
"app",
"apps",
"applikasjon",
"channel",
"channels",
"kanal",
"kanaler",
"model",
"models",
"modell",
"modeller",
"language",
"languages",
"språk",
"role",
"roles",
"rolle",
"roller",
"user",
"users",
"bruker",
"brukere",
"lærere",
"elever",
"kunde",
"kunder",
"market",
"markets",
"marked",
"country",
"countries",
"land",
"datasenter",
"data center",
"endpoint",
"endpoints",
"endepunkt",
"scenario",
"scenarios",
"scenarier",
"ledelse",
"organisasjon"
],
"limiters": [
"kun",
"bare",
"only",
"eneste",
"utelukkende",
"begrenset til",
"limited to",
"exclusively",
"solely",
"ikke tilgjengelig for",
"ikke støttet i"
]
},
"candidates": [
{
"file": "skills/ms-ai-advisor/references/architecture/cost-models.md",
"stamp_line": 601,
"end_line": 603,
"bucket": "container",
"scope_hits": [
"til kanonisk register. Scenario"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-advisor/references/prompt-engineering/error-handling-and-fallback-prompting.md",
"stamp_line": 377,
"end_line": 385,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-advisor/references/prompt-engineering/multimodal-prompt-design.md",
"stamp_line": 190,
"end_line": 198,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-engineering/references/agent-orchestration/agent-365-governance-and-deployment.md",
"stamp_line": 43,
"end_line": 43,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"Kun"
],
"stage2": {
"stratum": "limiter",
"verdict": "member",
"entry": "idx-33g",
"note": "\"Kun ... E7\" where source says \"based on licensed subscription\" and lists Agent 365 as its own license"
}
},
{
"file": "skills/ms-ai-engineering/references/agent-orchestration/agent-365-governance-and-deployment.md",
"stamp_line": 56,
"end_line": 61,
"bucket": "container",
"scope_hits": [
"per tenant"
],
"limiter_hits": [
"kun"
],
"stage2": {
"stratum": "container",
"verdict": "not-adjudicable",
"note": "source states both \"up to three agents ... in the organization\" and \"three slots ... for each user\"; \"per tenant\" cannot be falsified from it"
}
},
{
"file": "skills/ms-ai-engineering/references/agent-orchestration/agent-365-governance-and-deployment.md",
"stamp_line": 62,
"end_line": 63,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"Kun"
],
"stage2": {
"stratum": "limiter",
"verdict": "member",
"entry": "idx-33h",
"note": "\"Kun Agent Builder\" where source says \"Agent Builder and Copilot Studio agents\""
}
},
{
"file": "skills/ms-ai-engineering/references/agent-orchestration/agent-365-governance-and-deployment.md",
"stamp_line": 215,
"end_line": 217,
"bucket": "container",
"scope_hits": [
"for hele tenant",
"i Admin Center for hele tenant"
],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-engineering/references/agent-orchestration/computer-using-agents-cua.md",
"stamp_line": 130,
"end_line": 130,
"bucket": "container",
"scope_hits": [
"til eksterne modeller",
"for miljø"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "faithful",
"note": "faithful to the computer-use page it cites; the second required admin action lives one link deeper on allow-llm-generative-responses"
}
},
{
"file": "skills/ms-ai-engineering/references/data-engineering/microsoft-purview-governance.md",
"stamp_line": 248,
"end_line": 248,
"bucket": "container",
"scope_hits": [
"til data products"
],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-engineering/references/mlops-genaiops/genaiops-llm-specific-practices.md",
"stamp_line": 210,
"end_line": 212,
"bucket": "container",
"scope_hits": [
"til App Service, Container Apps"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-versioning-registry-management.md",
"stamp_line": 323,
"end_line": 328,
"bucket": "container",
"scope_hits": [
"For production online endpoint",
"i AI Builder\n- Eksporter modell",
"til Azure ML online endpoint"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-engineering/references/mlops-genaiops/responsible-ai-mlops-integration.md",
"stamp_line": 65,
"end_line": 65,
"bucket": "container",
"scope_hits": [
"i public preview. Bruker"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/alerting-strategies-escalation.md",
"stamp_line": 41,
"end_line": 43,
"bucket": "container",
"scope_hits": [
"under model"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/alerting-strategies-escalation.md",
"stamp_line": 84,
"end_line": 84,
"bucket": "container",
"scope_hits": [
"For Automation Runbooks, Logic Apps",
"i stedet for service",
"til role"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "member-other-class",
"entry": "idx-33k",
"note": "contradicts source: Azure Function managed identity is Yes, file says No in table (76) and prose (84)"
}
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/custom-dashboards-ai-operations.md",
"stamp_line": 290,
"end_line": 294,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/data-residency-audit-monitoring.md",
"stamp_line": 73,
"end_line": 75,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/data-residency-audit-monitoring.md",
"stamp_line": 530,
"end_line": 530,
"bucket": "container",
"scope_hits": [
"for nye tenants"
],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/data-residency-audit-monitoring.md",
"stamp_line": 558,
"end_line": 558,
"bucket": "container",
"scope_hits": [
"for EU, Microsoft 365 services"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/distributed-tracing-ai-pipelines.md",
"stamp_line": 171,
"end_line": 175,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun",
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/endpoint-health-and-capacity-planning.md",
"stamp_line": 629,
"end_line": 629,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/endpoint-health-and-capacity-planning.md",
"stamp_line": 633,
"end_line": 633,
"bucket": "container",
"scope_hits": [
"for custom endpoints"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/real-time-streaming-monitoring.md",
"stamp_line": 547,
"end_line": 547,
"bucket": "container",
"scope_hits": [
"in Foundry Models"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/security-and-audit-logging-ai.md",
"stamp_line": 408,
"end_line": 408,
"bucket": "container",
"scope_hits": [
"for Defender for Cloud"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/monitoring-observability/sla-monitoring-ai-services.md",
"stamp_line": 177,
"end_line": 179,
"bucket": "container",
"scope_hits": [
"for Azure Service",
"i samme workspace"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/digital-accessibility-action-plan.md",
"stamp_line": 160,
"end_line": 165,
"bucket": "container",
"scope_hits": [
"to create an inclusive learning environment",
"for K-12 lærere",
"for brukere"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/norwegian-public-sector-governance/norge-ai-strategy-government.md",
"stamp_line": 147,
"end_line": 154,
"bucket": "container",
"scope_hits": [
"i Defender for Cloud"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/responsible-ai/algorithmic-accountability-auditability.md",
"stamp_line": 76,
"end_line": 76,
"bucket": "container",
"scope_hits": [
"i Defender for Cloud Apps"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md",
"stamp_line": 46,
"end_line": 56,
"bucket": "container",
"scope_hits": [
"for å knytte bruker"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-incident-response-procedures.md",
"stamp_line": 46,
"end_line": 50,
"bucket": "container",
"scope_hits": [
"for AI Services",
"for deteksjon av anomalous model"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-prompt-shield-network.md",
"stamp_line": 237,
"end_line": 237,
"bucket": "container",
"scope_hits": [
"for å styre modeller"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "member",
"entry": "idx-33j",
"note": "names 2 of the source's 3 top-level endpoint categories; \"Self-hosted models and endpoints\" dropped"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-prompt-shield-network.md",
"stamp_line": 254,
"end_line": 265,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"bare"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-prompt-shield-network.md",
"stamp_line": 304,
"end_line": 304,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"Kun"
],
"stage2": {
"stratum": "limiter",
"verdict": "member",
"entry": "idx-33i",
"note": "\"Kun engelsk\" where source names eight trained-and-tested languages"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-threat-modeling-stride.md",
"stamp_line": 210,
"end_line": 213,
"bucket": "container",
"scope_hits": [
"across Azure subscriptions"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/data-leakage-prevention-ai.md",
"stamp_line": 47,
"end_line": 47,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": {
"stratum": "limiter",
"verdict": "faithful",
"note": "\"kun avsender-metadata evalueres\" is the source's own sentence"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/data-leakage-prevention-ai.md",
"stamp_line": 69,
"end_line": 73,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": {
"stratum": "limiter",
"verdict": "faithful",
"note": "\"only available in the Custom policy template\" verbatim"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/data-leakage-prevention-ai.md",
"stamp_line": 79,
"end_line": 79,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": {
"stratum": "limiter",
"verdict": "faithful",
"note": "\"DLP only checks the text you type into the prompt itself\" verbatim"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/data-leakage-prevention-ai.md",
"stamp_line": 131,
"end_line": 135,
"bucket": "container",
"scope_hits": [
"for Business\n- Labels med bruker"
],
"limiter_hits": [
"Kun",
"kun"
],
"stage2": {
"stratum": "container",
"verdict": "member",
"entry": "idx-33f",
"note": "reproduces one of the source's two supported-file-type lists, drops the other plus MP4"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/entra-agent-id-zero-trust.md",
"stamp_line": 136,
"end_line": 136,
"bucket": "container",
"scope_hits": [
"for menneskelige brukere"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "faithful",
"note": "paraphrase supported by conditional-access/agent-id and howto-target-agent-identities"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/entra-agent-id-zero-trust.md",
"stamp_line": 138,
"end_line": 138,
"bucket": "container",
"scope_hits": [
"per bruker"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "faithful",
"note": "near-verbatim against conditional-access/agent-id"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/entra-agent-id-zero-trust.md",
"stamp_line": 305,
"end_line": 305,
"bucket": "container",
"scope_hits": [
"for hver Power Platform-connector agenten bruker"
],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/entra-agent-id-zero-trust.md",
"stamp_line": 454,
"end_line": 454,
"bucket": "container",
"scope_hits": [
"per bruker"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "faithful",
"note": "near-verbatim; duplicate of the 138 claim in the same file"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/jailbreak-prevention-production.md",
"stamp_line": 351,
"end_line": 357,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/pii-detection-norwegian-context.md",
"stamp_line": 212,
"end_line": 212,
"bucket": "container",
"scope_hits": [
"For opprettelse av Language"
],
"limiter_hits": [],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/security-copilot-integration.md",
"stamp_line": 104,
"end_line": 106,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/security-copilot-integration.md",
"stamp_line": 122,
"end_line": 122,
"bucket": "container",
"scope_hits": [
"på tvers av alle brukere"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "faithful",
"note": "abbreviation of a six-property bullet list; the two dropped properties (default workspace, UI cost values informational only) are not scope-bearing"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/security-copilot-integration.md",
"stamp_line": 124,
"end_line": 128,
"bucket": "container",
"scope_hits": [
"for M365 E5-kunder som også bruker"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "member",
"entry": "idx-33e",
"note": "E5 where source says E5 and E7; file carries E7 at 112/114/452 from the same source"
}
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/security-copilot-integration.md",
"stamp_line": 262,
"end_line": 267,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"begrenset til"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/ai-security-engineering/zero-trust-ai-services.md",
"stamp_line": 130,
"end_line": 130,
"bucket": "container",
"scope_hits": [
"for Azure AI Services"
],
"limiter_hits": [],
"stage2": {
"stratum": "container",
"verdict": "flagged-not-adjudicated",
"note": "\"Bruk alltid DefaultAzureCredential\" sits against Microsoft best practice to move from DefaultAzureCredential to ManagedIdentityCredential in production; the page this passage rests on was not established"
}
},
{
"file": "skills/ms-ai-security/references/cost-optimization/observability-cost-reduction.md",
"stamp_line": 193,
"end_line": 193,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/cost-optimization/observability-cost-reduction.md",
"stamp_line": 194,
"end_line": 194,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"Kun"
],
"stage2": null
},
{
"file": "skills/ms-ai-security/references/cost-optimization/rag-query-cost-reduction.md",
"stamp_line": 161,
"end_line": 161,
"bucket": "limiter",
"scope_hits": [],
"limiter_hits": [
"kun"
],
"stage2": null
}
]
}