open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity

feat(ms-ai-architect): Sesjon 2 lag 0 - konsolider taksonomi til en sannhetskilde

Browse source 
Ny data/domain-taxonomy.json (tracket via gitignore-negasjon; generert
registry/rapporter forblir ignorert) konsoliderer de 4 divergerende
taksonomiene: (a) sitemap-prefikser, (b) discover INCLUDE/EXCLUDE,
(c) category->skill, (d) getFilePriority. lib/taxonomy.mjs laster + kompilerer.

Refaktor: poll-sitemaps, discover-new-urls, report-changes LESER taksonomien
(ingen embeddede kopier). Kriterium grep TARGET_PREFIXES=[ == 0 oppfylt.

Konsolideringsfunn:
- category-skill-map.json stale i 4 entries (copilot-extensibility,
  monitoring-observability, performance-scalability, prompt-engineering ->
  sa engineering; disk sier advisor/governance/security/advisor). DISK er
  kanon. Map-en er ukonsumert av kode (kun README) -> flagget, ikke rort.
- discover fikk poll-paritet (12 -> 18 sitemap-prefikser).
- skill utledes na fra kanonisk category_skill -> ingen divergens (invariant-test).

Adferdsbevarende: 0 mismatch old vs ny prioritetslogikk pa 51 endrede filer;
discover sine hardkodede skill-verdier matchet allerede disk.
Tester: validate 239, kb-update 49 (+7 taksonomi), kb-eval 13 - alle gronne.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REiKFhP4w6xGXXqWKpPCJJ
This commit is contained in:
Kjell Tore Guttormsen 2026-06-19 21:02:59 +02:00
commit 8bea3a224a
7 changed files with 323 additions and 95 deletions
Download patch file Download diff file Expand all files Collapse all files

69
scripts/kb-update/discover-new-urls.mjs
View file

@ -8,74 +8,19 @@ import { fileURLToPath } from 'node:url';
import { normalizeUrl } from './lib/url-normalize.mjs';
import { loadRegistry, saveReport } from './lib/registry-io.mjs';
import { streamSitemap, fetchSitemapIndex } from './lib/sitemap-stream.mjs';
import { loadTaxonomy, getSitemapPrefixes, makeClassifier } from './lib/taxonomy.mjs';
const __dirname = dirname(fileURLToPath(import.meta.url));
const DATA_DIR = join(__dirname, 'data');
const limitArg = process.argv.indexOf('--limit');
const limit = limitArg !== -1 ? parseInt(process.argv[limitArg + 1], 10) : Infinity;
// Relevance patterns — URL paths that indicate content this plugin should cover
const INCLUDE = [
{ pattern: /\/azure\/ai-foundry\//, skill: 'ms-ai-engineering', category: 'azure-ai-services' },
{ pattern: /\/azure\/ai-services\//, skill: 'ms-ai-engineering', category: 'azure-ai-services' },
{ pattern: /\/azure\/machine-learning\//, skill: 'ms-ai-engineering', category: 'mlops-genaiops' },
{ pattern: /\/azure\/search\//, skill: 'ms-ai-engineering', category: 'rag-architecture' },
{ pattern: /\/azure\/api-management\//, skill: 'ms-ai-engineering', category: 'api-management' },
{ pattern: /\/azure\/azure-monitor\//, skill: 'ms-ai-governance', category: 'monitoring-observability' },
{ pattern: /\/azure\/well-architected\//, skill: 'ms-ai-advisor', category: 'architecture' },
{ pattern: /\/microsoft-copilot-studio\//, skill: 'ms-ai-advisor', category: 'copilot-extensibility' },
{ pattern: /\/copilot\/microsoft-365\//, skill: 'ms-ai-advisor', category: 'copilot-extensibility' },
{ pattern: /\/microsoft-365-copilot\//, skill: 'ms-ai-advisor', category: 'copilot-extensibility' },
{ pattern: /\/security\/(?:ai|benchmark|engineering)\//, skill: 'ms-ai-security', category: 'ai-security-engineering' },
{ pattern: /\/azure\/defender-for-cloud\//, skill: 'ms-ai-security', category: 'ai-security-engineering' },
{ pattern: /\/purview\//, skill: 'ms-ai-governance', category: 'responsible-ai' },
{ pattern: /\/semantic-kernel\//, skill: 'ms-ai-engineering', category: 'agent-orchestration' },
{ pattern: /\/agent-framework\//, skill: 'ms-ai-engineering', category: 'agent-orchestration' },
{ pattern: /\/fabric\/(?:data-engineering|data-science|real-time-intelligence)\//, skill: 'ms-ai-engineering', category: 'data-engineering' },
{ pattern: /\/azure\/cosmos-db\//, skill: 'ms-ai-engineering', category: 'data-engineering' },
{ pattern: /\/azure\/databricks\//, skill: 'ms-ai-engineering', category: 'data-engineering' },
{ pattern: /\/entra\/(?:identity|agent)\//, skill: 'ms-ai-security', category: 'ai-security-engineering' },
];
// Exclude patterns — skip even if they match INCLUDE
const EXCLUDE = [
/\/training\//,
/\/credentials\//,
/\/legal\//,
/\/previous-versions\//,
/\/archive\//,
/\/samples\//,
/\/release-notes?\//,
/\/whats-new/,
/\/changelog/,
/\/migrate\//,
];
function classifyUrl(url) {
if (EXCLUDE.some(p => p.test(url))) return null;
for (const rule of INCLUDE) {
if (rule.pattern.test(url)) {
return { skill: rule.skill, category: rule.category };
}
}
return null;
}
// Target sitemaps for discovery — same as poller, minus dotnet
const TARGET_PREFIXES = [
'azure_en-us_',
'microsoft-copilot-studio_en-us_',
'security_en-us_',
'fabric_en-us_',
'power-platform_en-us_',
'ai_en-us_',
'copilot_en-us_',
'compliance_en-us_',
'agent-framework_en-us_',
'semantic-kernel_en-us_',
'entra_en-us_',
'purview_en-us_',
];
// Relevance + sitemap targets come from the consolidated taxonomy (lag 0).
// classifyUrl: exclude wins, first include match, skill derived from the
// canonical category->skill map — no embedded skill assignments here.
const taxonomy = loadTaxonomy(DATA_DIR);
const classifyUrl = makeClassifier(taxonomy);
const TARGET_PREFIXES = getSitemapPrefixes(taxonomy);
function extractChildName(loc) {
const match = loc.match(/\/_sitemaps\/([^/]+)\.xml$/);