feat(ms-ai-architect): Spor 3 Port 2 — create-time-guard (born-verified contract) [skip-docs]

Type-aware create-guard for reference-filer (Spor 3 Port 1/Port 2):

- kb-headers.mjs: parseTypeHeader/parseVerifiedHeader/parseVerifiedByHeader
  (samme top-500-byte bold-label-skann som parseSourceHeader).
- transform.mjs: buildKbHeader er type-aware (reference krever
  source+verified+verified_by; non-reference kaster pa MS-Learn-source) +
  emitterer Type/Verified/Verified by; validateKbFile type-aware;
  stampVerifiedMeta = fodt-verifisert-gate (stempler KUN ved bestatt
  judge-verdikt, ellers kaster).
- validate-kb-file.mjs (ny): kjorbar gate generatorer kaller for commit
  (exit != 0 ved kontraktbrudd).
- generate-skills.md + kb-update.md + transform-prompt.md: wiret til
  fodt-verifisert kontrakt (header-felt + judge-steg + create-guard-gate).

Suite 586/586 (33 nye + 1 invariant). Plugin-validering 239/0/0.

Utenfor scope (flagget): generate-skills.sh legacy (sonnet/Cosmo/no-source),
Cosmo-persona i generatorene (S-Cosmo), korpus-migrering av 306 filer (Spor 1).
This commit is contained in:
Kjell Tore Guttormsen 2026-06-29 10:36:21 +02:00
commit 5a0ba1fc9e
10 changed files with 621 additions and 54 deletions

View file

@ -86,9 +86,13 @@ Format (STRENGT — alle seksjoner påkrevd):
# {SKILL_TITLE}
**Last updated:** 2026-02
**Last updated:** {dagens måned, YYYY-MM}
**Status:** [GA | Preview | Announced]
**Category:** {CATEGORY_NAME}
**Type:** reference
**Source:** {den autoritative Microsoft Learn-URLen fra research — den faktiske doc-siden påstandene hviler på, ikke et søketreff}
**Verified:** {SETTES i Steg 2.5 etter judge — IKKE her}
**Verified by:** {SETTES i Steg 2.5 etter judge — IKKE her}
---
@ -127,6 +131,23 @@ Format (STRENGT — alle seksjoner påkrevd):
- Confidence markers: "Verified" (fra MCP), "Baseline" (modellkunnskap)
- Konkret og balansert — vis fordeler OG ulemper
## Steg 2.5: Født-verifisert (judge FØR du returnerer)
Filen skal være *født verifisert*: ingen reference-fil forlater agenten uten at de
maskin-verifiserbare påstandene er bekreftet mot `Source`-URLen.
1. Kjør den gjeldende claim-judgen (`scripts/kb-eval/judge-claim-prompt-v2.md`) over
filens maskin-verifiserbare påstander (sku/version/tpm/status/region/taxonomy) mot
`Source`-URLen du hentet i Steg 1.
2. **Hvis alle påstander er `grounded`:** sett i headeren
`**Verified:** {dagens dato, YYYY-MM-DD}` og `**Verified by:** judge-v2`.
3. **Hvis noen påstand IKKE er grounded:** IKKE stempl `Verified`. Rett påstanden mot
kilden og kjør judgen på nytt, eller — hvis verdien er maskin-uverifiserbar (pris/
JS-rendret) — merk den eksplisitt og la filen gå til menneske-gaten (den blir IKKE
committet; se batch-gaten under).
Aldri stempl `Verified` spekulativt. Stempelet betyr «judgen bekreftet mot kilde».
## Steg 3: Returner kvittering
Returner KUN dette (ingenting annet):
@ -159,11 +180,19 @@ Task(general-purpose, sonnet): "Research + write skill: Multi-Index..."
1. **Parse kvitteringer** fra agentene
2. **Verifiser filer finnes** med Glob
3. **Oppdater state.json:**
- Legg til ferdige skill-IDer i `completed`
- Legg til eventuelle feilede i `failed`
3. **Create-guard (OBLIGATORISK — Spor 3 Port 2):** kjør gaten over batchens nye filer:
```bash
node scripts/kb-update/validate-kb-file.mjs <fil1.md> <fil2.md> ...
```
Gaten exit-er ≠0 hvis en fil mangler `Source`, ikke er født-verifisert (mangler
`Verified`/`Verified by`), eller er en stor fil uten TOC. **En fil som FAIL-er
committes IKKE** — logg den i `state.failed` og send den til menneske-gaten. Dette er
det som hindrer generatoren i å re-introdusere drift (avhengighet før Spor 1).
4. **Oppdater state.json:**
- Legg til ferdige (gate-PASS) skill-IDer i `completed`
- Legg til feilede/gate-FAIL i `failed`
- Oppdater `stats.total_generated` og `stats.total_bytes`
4. **Neste batch** eller avslutt
5. **Neste batch** eller avslutt
## Etter hele sesjonen
@ -180,13 +209,17 @@ Task(general-purpose, sonnet): "Research + write skill: Multi-Index..."
Gjenstår: N skills
```
2. **Commit:**
2. **Create-guard FØR commit (OBLIGATORISK):** kjør gaten over ALLE filer du er i ferd
med å committe. Bare gate-PASS-filer skal med:
```bash
node scripts/kb-update/validate-kb-file.mjs $(git diff --name-only --diff-filter=AM -- 'skills/ms-ai-*/references/**/*.md')
git add skills/ms-ai-*/references/<dirs>/ scripts/skill-gen/state.json
git commit -m "docs(architect): generate N knowledge skills (category-names)"
```
Hvis gaten exit-er ≠0: IKKE commit de feilende filene — fjern dem fra staging og send
til menneske-gaten.
3. **Oppdater REMEMBER.md** med ny status
3. **Oppdater STATE.md** med ny status
## Feilhåndtering
@ -263,8 +296,16 @@ Bruk Edit-verktøyet (IKKE Write) for å:
- Oppdatere "Last updated" til gjeldende måned
- Oppdatere utdaterte fakta, priser, datoer
- Oppdatere Microsoft Learn-URLer
- Markere oppdatert innhold med "Verified (MCP {måned})"
- Beholde eksisterende struktur og seksjoner
- **Backfill kontrakt-headeren** (Spor 3 Port 1) hvis legacy-fila mangler den: legg til
`**Type:** reference` og `**Source:** {autoritativ Learn-URL}` i header-blokken (en
berørt fil skal forlate oppdateringen kontrakt-kompatibel — inkrementell Spor 1).
## Steg 2.5: Født-verifisert (judge FØR du returnerer)
Kjør claim-judgen (`scripts/kb-eval/judge-claim-prompt-v2.md`) over de oppdaterte maskin-
verifiserbare påstandene mot `Source`. Alle `grounded` ⇒ sett `**Verified:** {dagens dato}`
+ `**Verified by:** judge-v2`. Ikke grounded ⇒ IKKE stempl; rett mot kilde og kjør på nytt,
eller send til menneske-gaten (fila committes ikke; se batch-gaten). Aldri stempl spekulativt.
## Steg 3: Returner kvittering
SKILL_UPDATED
@ -275,7 +316,8 @@ status: success|no_changes|failed
```
4. Track in `state.json` under a new `"updated"` array
5. After each batch, verify files still pass `validate-plugin.sh`
5. **Create-guard (OBLIGATORISK):** `node scripts/kb-update/validate-kb-file.mjs <oppdaterte filer>` — gate-FAIL committes IKKE
6. After each batch, verify files still pass `validate-plugin.sh`
**Key difference from generation:** Update uses Edit (preserves structure), generation uses Write (creates from scratch).

View file

@ -127,11 +127,11 @@ saveDecisions(led);
d. **For `approved`:** registrér URLen i `url-registry` (gated) så den fanges av polling heretter — via `lib/registry-io.mjs` `saveRegistry`. Bruk `suggested_skill`/`suggested_category` fra kandidaten. Deretter kjør **transformasjonslaget (lag 4)** for å lage KB-fila:
1. `microsoft_docs_fetch` på den godkjente URLen → kildedokument.
2. Destillér via `scripts/kb-update/transform-prompt.md` (doc→KB-fil; status-påstander holdes eksplisitte). Multi-agent parallell fan-out foreslås i produksjon (roadmap §71), speilet på `generate-skills`-mønsteret.
3. `content = composeKbFile({title, status, category, source: <godkjent URL>, lastUpdated: <YYYY-MM>}, body)` (`lib/transform.mjs`) — bygger header + (store filer >100 linjer) en deterministisk `## Innhold`-TOC + brødtekst. **`Status` + `Source` er obligatoriske** — `Source` i header-blokka (øverste 500 bytes) er det som lar lag 3 backfille `authority_source`. TOC-en fødes inn så regenerering ikke stripper den (Fase 1c).
4. `validateKbFile(content)` MÅ være `valid: true` (title + Last updated + Status + Source; **store filer også TOC**) før noe gates videre (ellers be modellen fylle manglende felt).
3. **Født-verifisert (Spor 3 Port 2):** kjør claim-judgen (`scripts/kb-eval/judge-claim-prompt-v2.md`) over brødtekstens maskin-verifiserbare påstander mot den godkjente URLen → `verdict = {pass}`. Så `meta = stampVerifiedMeta({title, status, category, source: <godkjent URL>, lastUpdated: <YYYY-MM>}, verdict, <i dag>)` — stempler `type='reference'` + `verified` + `verified_by='judge-v2'` **kun** ved `pass`; ellers KASTER (ingen fil, flagg for menneske). Deretter `content = composeKbFile(meta, body)` (`lib/transform.mjs`) — header + (store filer >100 linjer) en deterministisk `## Innhold`-TOC + brødtekst. **`Status` + `Source` + `Verified` + `Verified by` er obligatoriske** — `Source` i header-blokka (øverste 500 bytes) er det som lar lag 3 backfille `authority_source`. TOC-en fødes inn så regenerering ikke stripper den (Fase 1c).
4. `validateKbFile(content)` MÅ være `valid: true` (title + Last updated + Status + Source + **Verified + Verified by**; **store filer også TOC**) før noe gates videre (ellers be modellen fylle manglende felt).
5. For hver status-/load-bearing-påstand: `buildChange({...})`**lag 5** `classifyChange(...)` (samme gate som §4 c2). Status-påstander er alltid `flagged`.
6. `resolveTargetPath(tax, category, filename)` → eierskill-sti via taksonomien (`null` = ukjent kategori → flagg for operatør, ikke skriv).
7. Først etter operatør-gate: atomisk skriving (`lib/atomic-write.mjs` + `lib/backup.mjs`). **`transform.mjs` skriver aldri selv** — verifisert av `tests/kb-update/test-transform.test.mjs` (import-invariant). Kriterium verifisert av `tests/kb-eval/test-transform-criterion.test.mjs` (regenerer 1 fil → eval ≥ baseline).
7. **Create-guard FØR skriving:** `node scripts/kb-update/validate-kb-file.mjs <sti>` (exit ≠0 ⇒ ikke skriv). Først etter operatør-gate + grønn create-guard: atomisk skriving (`lib/atomic-write.mjs` + `lib/backup.mjs`). **`transform.mjs` skriver aldri selv** — verifisert av `tests/kb-update/test-transform.test.mjs` (import-invariant). Kriterium verifisert av `tests/kb-eval/test-transform-criterion.test.mjs` (regenerer 1 fil → eval ≥ baseline).
e. **Invariant:** `discover-new-urls.mjs` (deteksjon) skriver **aldri** ledgeren — den kun leser. Bare denne gaten skriver. Verifisert av `tests/kb-update/test-discover-invariant.test.mjs`.

View file

@ -10,6 +10,21 @@ const SOURCE_PATTERNS = [
/\*\*Primary source:\*\*\s*(\S+)/i,
];
// Spor 3 Port 1 frontmatter contract: these live in the SAME top-500-byte bold-label
// region as **Source:**, so build-registry and the create-guard read one header format.
// `Verified:` and `Verified by:` are distinct labels — the date pattern below is
// anchored on the colon right after "Verified" so it never swallows the "by:" line.
const TYPE_PATTERN = /\*\*Type:\*\*\s*(\S+)/i;
const VERIFIED_PATTERN = /\*\*Verified:\*\*\s*(\S+)/i;
const VERIFIED_BY_PATTERN = /\*\*Verified by:\*\*\s*(\S+)/i;
/** Scan only the header region for the first capture of `pattern`, trimmed. */
function scanHeader(content, pattern) {
if (!content || typeof content !== 'string') return null;
const match = content.slice(0, HEADER_REGION_BYTES).match(pattern);
return match ? match[1].trim() : null;
}
/**
* Extract the declared authority source URL from a KB reference file's header.
* @param {string} content full file content (only the first 500 bytes are scanned)
@ -24,3 +39,32 @@ export function parseSourceHeader(content) {
}
return null;
}
/**
* Extract the Port 1 `type` (reference|template|methodology|regulatory), lowercased.
* @param {string} content full file content (only the first 500 bytes are scanned)
* @returns {string|null} the declared type, or null if absent
*/
export function parseTypeHeader(content) {
const t = scanHeader(content, TYPE_PATTERN);
return t === null ? null : t.toLowerCase();
}
/**
* Extract the Port 1 `verified` date (when the file was last confirmed against source).
* Anchored on `**Verified:**` so it never matches the `**Verified by:**` line.
* @param {string} content full file content (only the first 500 bytes are scanned)
* @returns {string|null} the verified date (YYYY-MM or YYYY-MM-DD), or null if absent
*/
export function parseVerifiedHeader(content) {
return scanHeader(content, VERIFIED_PATTERN);
}
/**
* Extract the Port 1 `verified_by` field (`judge-vN` / `human`).
* @param {string} content full file content (only the first 500 bytes are scanned)
* @returns {string|null} the verifier, or null if absent
*/
export function parseVerifiedByHeader(content) {
return scanHeader(content, VERIFIED_BY_PATTERN);
}

View file

@ -15,12 +15,38 @@
// validateKbFile uses the SAME source detection as build-registry — one source of
// truth for "what counts as a Source header" (the top-500-byte region).
import { parseSourceHeader } from './kb-headers.mjs';
import {
parseSourceHeader,
parseTypeHeader,
parseVerifiedHeader,
parseVerifiedByHeader,
} from './kb-headers.mjs';
import { getCategorySkill } from './taxonomy.mjs';
// Mandatory header fields. Status is obligatory per spec ("status-felt obligatorisk");
// Source is what lets lag 3 backfill authority_source and lag-5 rule 3 eventually fire.
const REQUIRED_META = ['title', 'status', 'category', 'source', 'lastUpdated'];
// Base mandatory header fields for EVERY file type. Status is obligatory per spec
// ("status-felt obligatorisk").
const REQUIRED_BASE_META = ['title', 'status', 'category', 'lastUpdated'];
// Spor 3 Port 1 — the frontmatter-contract `type`. Only `reference` files are in the
// correctness scope (they carry a Microsoft source we verify against). Template /
// methodology / regulatory are out of scope and carry NO Microsoft source.
const VALID_TYPES = ['reference', 'template', 'methodology', 'regulatory'];
const DEFAULT_TYPE = 'reference';
// Extra fields a `reference` file MUST carry to be "born verified" (Spor 3 Port 2):
// `source` lets lag-5 authority checks fire; `verified`/`verified_by` record that the
// content cleared the judge before it shipped. A reference file without these is the
// drift-introducing failure mode the create-guard exists to stop.
const REQUIRED_REFERENCE_META = ['source', 'verified', 'verified_by'];
// The current GATE-PASS claim judge (judge-claim-prompt-v2). stampVerifiedMeta records
// `verified_by: judge-v<JUDGE_VERSION>` unless the caller overrides or marks `human`.
const JUDGE_VERSION = 2;
// A Microsoft Learn / docs authority URL — a non-reference file may NOT carry one.
const MS_LEARN_RE = /\b(?:learn|docs)\.microsoft\.com/i;
const DATE_RE = /^\d{4}-\d{2}(?:-\d{2})?$/;
// A reference file longer than this many lines should carry a table of contents
// (best-practices §Structure). MIRRORS eval.mjs N4_LARGE_FILE_LINES — kept in sync
@ -60,33 +86,84 @@ function slugify(heading) {
}
/**
* Build the deterministic KB-file header block. Status and Source live IN the
* header (top 500 bytes) so kb-headers.parseSourceHeader and the "Last updated:"
* scanner can see them the old bottom-of-file source section was invisible to
* those scanners, which is why authority_source coverage was 0%.
* Build the deterministic KB-file header block. Status, Source and the Port 1 contract
* fields (Type / Verified / Verified by) live IN the header (top 500 bytes) so
* kb-headers.* and the "Last updated:" scanner can see them the old bottom-of-file
* source section was invisible to those scanners, which is why authority_source
* coverage was 0%.
*
* @param {{title: string, status: string, category: string, source: string,
* lastUpdated: string}} meta
* Type-aware create-guard (Spor 3 Port 2): a `reference` file MUST carry
* source + verified + verified_by (it is "born verified"); a non-reference file
* (template/methodology/regulatory) is out of correctness scope and may NOT carry a
* Microsoft Learn source. Either violation is a hard throw the create-time gate.
*
* @param {{title: string, status: string, category: string, lastUpdated: string,
* type?: string, source?: string, verified?: string, verified_by?: string}} meta
* @returns {string} the header block, ending with the `---` rule + trailing newline
* @throws if any mandatory field is missing/empty
* @throws if the type is unknown, a mandatory field is missing, or a non-reference
* file carries a Microsoft Learn source
*/
export function buildKbHeader(meta) {
const m = meta ?? {};
const missing = REQUIRED_META.filter((k) => {
const type = m.type ?? DEFAULT_TYPE;
if (!VALID_TYPES.includes(type)) {
throw new Error(`buildKbHeader: invalid type '${type}' (expected ${VALID_TYPES.join('|')})`);
}
const isReference = type === 'reference';
const empty = (k) => {
const v = m[k];
return v === undefined || v === null || String(v).trim() === '';
});
};
const required = isReference
? [...REQUIRED_BASE_META, ...REQUIRED_REFERENCE_META]
: REQUIRED_BASE_META;
const missing = required.filter(empty);
if (missing.length) {
throw new Error(`buildKbHeader: missing mandatory field(s): ${missing.join(', ')}`);
}
return (
if (!isReference && !empty('source') && MS_LEARN_RE.test(String(m.source))) {
throw new Error(
`buildKbHeader: a ${type} file may not carry a Microsoft Learn source (out of correctness scope)`,
);
}
let header =
`# ${m.title}\n\n` +
`**Last updated:** ${m.lastUpdated}\n` +
`**Status:** ${m.status}\n` +
`**Category:** ${m.category}\n` +
`**Source:** ${m.source}\n\n` +
`---\n`
);
`**Type:** ${type}\n`;
if (!empty('source')) header += `**Source:** ${m.source}\n`;
if (!empty('verified')) header += `**Verified:** ${m.verified}\n`;
if (!empty('verified_by')) header += `**Verified by:** ${m.verified_by}\n`;
return `${header}\n---\n`;
}
/**
* Born-verified gate (Spor 3 Port 2): stamp `verified`/`verified_by` onto a reference
* meta IFF the judge verdict passed otherwise REFUSE (throw). This encodes
* "nytt/regenerert innhold kjøres gjennom judgen FØR commit født verifisert; ellers
* ingen fil". transform.mjs never calls the judge itself (pure lib); the command runs
* the v2 claim judge, aggregates per-claim results into `{pass}`, and calls this. A
* failing verdict means no file is born.
*
* @param {object} baseMeta the reference meta (title/status/category/source/lastUpdated)
* @param {{pass: boolean, judgeVersion?: number, by?: 'human'}} verdict
* @param {string} today YYYY-MM or YYYY-MM-DD, supplied by the caller (lib stays pure)
* @returns {object} baseMeta with type='reference' + verified + verified_by set
* @throws if the verdict is not passing, or today is not a valid date
*/
export function stampVerifiedMeta(baseMeta, verdict, today) {
if (!verdict || verdict.pass !== true) {
throw new Error('stampVerifiedMeta: refusing to stamp verified — judge verdict not passing');
}
if (!DATE_RE.test(String(today ?? ''))) {
throw new Error(`stampVerifiedMeta: invalid today date '${today}' (expected YYYY-MM[-DD])`);
}
const verifiedBy =
verdict.by === 'human'
? 'human'
: `judge-v${Number.isInteger(verdict.judgeVersion) ? verdict.judgeVersion : JUDGE_VERSION}`;
return { ...baseMeta, type: 'reference', verified: today, verified_by: verifiedBy };
}
/**
@ -181,10 +258,16 @@ const RE_STATUS = /\*\*Status:\*\*\s*\S/i;
/**
* Validate that a candidate KB file carries the mandatory contract: a title, a
* "Last updated:" header (so it is never dateless never the 'unverified' bucket),
* a Status field, a Source header in the scannable region, and for a large file
* (>100 lines, per N4) a table of contents. The TOC clause is the write-path
* regression gate: a regenerated large file that lost its TOC fails here and the
* caller re-prompts, so KB-update can never silently strip a TOC back out.
* a Status field, and for a large file (>100 lines, per N4) a table of contents.
* The TOC clause is the write-path regression gate: a regenerated large file that lost
* its TOC fails here and the caller re-prompts, so KB-update can never silently strip a
* TOC back out.
*
* Type-aware (Spor 3 Port 2): a `reference` file MUST additionally carry source +
* verified + verified_by (born-verified). A file with no `**Type:**` header is treated
* as a reference the create path always stamps Type, and the legacy corpus (no Type)
* is migrated under Spor 1, where this same contract is the target. Non-reference files
* (template/methodology/regulatory) are out of correctness scope and need no source.
*
* @param {string} content full file content
* @returns {{valid: boolean, missing: string[]}}
@ -195,7 +278,12 @@ export function validateKbFile(content) {
if (!RE_TITLE.test(s)) missing.push('title');
if (!RE_LAST_UPDATED.test(s)) missing.push('last_updated');
if (!RE_STATUS.test(s)) missing.push('status');
if (parseSourceHeader(s) === null) missing.push('source');
const type = parseTypeHeader(s) ?? DEFAULT_TYPE;
if (type === 'reference') {
if (parseSourceHeader(s) === null) missing.push('source');
if (parseVerifiedHeader(s) === null) missing.push('verified');
if (parseVerifiedByHeader(s) === null) missing.push('verified_by');
}
if (isLargeContent(s) && !hasToc(s)) missing.push('toc');
return { valid: missing.length === 0, missing };
}

View file

@ -23,13 +23,16 @@ regel 3 (autoritets-mismatch) faktisk fyre.
- `{{CATEGORY}}` — taksonomi-kategorinøkkel (f.eks. `rag-architecture`) — bestemmer eierskill via `resolveTargetPath`
- `{{SOURCE_URL}}` — den **utpekte autoritetskilden** (kanonisk Learn-URL dokumentet ble hentet fra)
- `{{LAST_UPDATED}}``YYYY-MM` (dagens måned, caller-injisert)
- `{{VERIFIED_DATE}}``YYYY-MM-DD` (dagen judgen bekreftet innholdet; settes av `stampVerifiedMeta` etter steg 2, ikke før)
- `{{FETCHED_DOC}}` — den hentede markdownen fra `microsoft_docs_fetch`
- `{{EXISTING_CONTEXT}}` — sammendrag av overlappende eksisterende ref-filer (unngå duplisering — K4)
## Header (deterministisk — IKKE la LLM-en skrive den)
Header-blokka bygges av `buildKbHeader({title, status, category, source, lastUpdated})`,
ikke av modellen. Den ser slik ut og MÅ stå først:
Header-blokka bygges av `buildKbHeader({title, status, category, type, source, lastUpdated,
verified, verified_by})`, ikke av modellen. En `reference`-fil er **født verifisert**
(Spor 3 Port 1/Port 2): den MÅ bære `Type` + `Source` + `Verified` + `Verified by`.
Headeren ser slik ut og MÅ stå først:
```markdown
# {{TITLE}}
@ -37,17 +40,22 @@ ikke av modellen. Den ser slik ut og MÅ stå først:
**Last updated:** {{LAST_UPDATED}}
**Status:** {{STATUS}}
**Category:** {{CATEGORY}}
**Type:** reference
**Source:** {{SOURCE_URL}}
**Verified:** {{VERIFIED_DATE}}
**Verified by:** judge-v2
---
```
Modellen skriver **kun brødteksten under `---`**. `composeKbFile({...}, body)` setter så
sammen hele fila — header + (for **store filer >100 linjer**) en deterministisk
`## Innhold`-TOC fra brødtekstens `##`-seksjoner + brødteksten. Etterpå kjører
`validateKbFile(content)` og MÅ returnere `valid: true` (title + Last updated + Status +
Source til stede; **for store filer også TOC**) før noe gates videre. TOC-en fødes inn så
en senere regenerering ikke stripper den (write-path-regresjon — Fase 1c).
`Verified`/`Verified by` settes IKKE før innholdet har bestått judgen (se «Etter
destillasjon» steg 2). Modellen skriver **kun brødteksten under `---`**.
`composeKbFile(stampVerifiedMeta({...}, verdict, today), body)` setter så sammen hele fila
— header + (for **store filer >100 linjer**) en deterministisk `## Innhold`-TOC fra
brødtekstens `##`-seksjoner + brødteksten. Etterpå kjører `validateKbFile(content)` og MÅ
returnere `valid: true` (title + Last updated + Status + Source + **Verified + Verified by**
til stede; **for store filer også TOC**) før noe gates videre. TOC-en fødes inn så en
senere regenerering ikke stripper den (write-path-regresjon — Fase 1c).
## Prompt (brødtekst)
@ -90,10 +98,19 @@ Ingen innledning eller avslutningskommentar — kun markdown-brødteksten.
## Etter destillasjon (caller, gated)
1. `content = composeKbFile({title, status, category, source, lastUpdated}, body)` — header + (store filer) `## Innhold`-TOC + brødtekst. Erstatter den bare `buildKbHeader(...) + body`-konkateneringen.
2. `validateKbFile(content)` → må være `valid: true` (title + Last updated + Status + Source; **store filer også TOC**) (ellers: be modellen fikse manglende felt).
3. For hver status-/load-bearing-påstand: `buildChange({field, oldValue, newValue, sourceUrl, authoritySource})`
1. **Født-verifisert judge (FØR commit):** kjør claim-judgen (`scripts/kb-eval/judge-claim-prompt-v2.md`)
over brødtekstens maskin-verifiserbare påstander mot `{{SOURCE_URL}}`. Aggreger til `verdict = {pass}`
(alle maskin-verifiserbare påstander `grounded``pass: true`).
2. `meta = stampVerifiedMeta({title, status, category, source, lastUpdated}, verdict, today)`
stempler `type='reference'` + `verified=today` + `verified_by='judge-v2'` **kun** ved `pass: true`;
ellers KASTER den (ingen fil fødes). Deretter `content = composeKbFile(meta, body)`
header + (store filer) `## Innhold`-TOC + brødtekst.
3. `validateKbFile(content)` → må være `valid: true` (title + Last updated + Status + Source +
**Verified + Verified by**; **store filer også TOC**) (ellers: be modellen fikse manglende felt).
4. For hver status-/load-bearing-påstand: `buildChange({field, oldValue, newValue, sourceUrl, authoritySource})`
`classifyChange(...)` (lag 5). `flagged` → operatør avgjør; status-påstander er alltid `flagged`.
4. `resolveTargetPath(tax, category, filename)` → eierskill-sti (null = ukjent kategori → gate, ikke skriv).
5. Først etter operatør-gate: atomisk skriving (`lib/atomic-write.mjs` + `lib/backup.mjs`). Transform-libben skriver aldri selv.
5. `resolveTargetPath(tax, category, filename)` → eierskill-sti (null = ukjent kategori → gate, ikke skriv).
6. **Create-guard FØR skriving:** `node scripts/kb-update/validate-kb-file.mjs <sti>` (exit ≠0 = ikke skriv).
Først etter operatør-gate + grønn create-guard: atomisk skriving (`lib/atomic-write.mjs` +
`lib/backup.mjs`). Transform-libben skriver aldri selv.
```

View file

@ -0,0 +1,69 @@
#!/usr/bin/env node
// validate-kb-file.mjs — the executable create-time-guard (Spor 3 Port 2).
//
// The gate any generator (generate-skills, kb-update) calls AFTER writing a candidate
// KB file and BEFORE committing it. It wraps transform.validateKbFile over one or more
// paths and exits NON-ZERO if any file violates the contract — so a regenerated file
// that lost its Source, is not born-verified, or dropped its TOC can never ship. This
// is what makes "create-guard MUST be in place before Spor 1 corpus-fixing" enforceable:
// the generator physically cannot re-introduce the drift the corpus pass is removing.
//
// Usage:
// node scripts/kb-update/validate-kb-file.mjs <file.md> [<file2.md> ...]
//
// Exit code: 0 = all files valid; 1 = at least one violation (or usage error).
import { readFileSync, realpathSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { validateKbFile } from './lib/transform.mjs';
/**
* Validate a batch of candidate KB files. Pure + dependency-injected (readFile) so it is
* unit-testable without disk. A path that cannot be read is reported as invalid (with a
* `read-error` marker) rather than throwing one unreadable file must not crash the gate.
*
* @param {string[]} paths
* @param {(p: string) => string} [readFile] reader (defaults to readFileSync utf8)
* @returns {{ok: boolean, results: Array<{path: string, valid: boolean, missing: string[]}>}}
*/
export function validatePaths(paths, readFile = (p) => readFileSync(p, 'utf8')) {
const results = (paths ?? []).map((path) => {
let content;
try {
content = readFile(path);
} catch (err) {
return { path, valid: false, missing: [`read-error: ${err.message}`] };
}
const { valid, missing } = validateKbFile(content);
return { path, valid, missing };
});
return { ok: results.every((r) => r.valid), results };
}
function main(argv) {
const paths = argv.slice(2);
if (paths.length === 0) {
process.stderr.write('usage: validate-kb-file.mjs <file.md> [<file2.md> ...]\n');
process.exit(1);
}
const { ok, results } = validatePaths(paths);
for (const r of results) {
if (r.valid) {
process.stdout.write(`OK ${r.path}\n`);
} else {
process.stdout.write(`FAIL ${r.path} — missing: ${r.missing.join(', ')}\n`);
}
}
process.exit(ok ? 0 : 1);
}
// Robust direct-invocation check (mirrors detect-courses.mjs): only run main when this
// file is the entrypoint, not when imported by a test.
const isMain = (() => {
try {
return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url));
} catch {
return false;
}
})();
if (isMain) main(process.argv);

View file

@ -19,7 +19,7 @@ import { readFileSync } from 'node:fs';
import { dirname, join, basename } from 'node:path';
import { fileURLToPath } from 'node:url';
import { evalSkill } from '../../scripts/kb-eval/eval.mjs';
import { composeKbFile, validateKbFile, resolveTargetPath } from '../../scripts/kb-update/lib/transform.mjs';
import { composeKbFile, validateKbFile, resolveTargetPath, stampVerifiedMeta } from '../../scripts/kb-update/lib/transform.mjs';
import { loadTaxonomy } from '../../scripts/kb-update/lib/taxonomy.mjs';
const __dirname = dirname(fileURLToPath(import.meta.url));
@ -71,16 +71,20 @@ test('regenerate 1 real file → valid, dated, source-anchored, body preserved,
const category = parts[3];
const filename = basename(refRel);
const regenerated = composeKbFile({
// Born-verified (Spor 3 Port 2): the regenerated file is stamped only after a passing
// judge verdict. Here the verdict is supplied directly (no LLM in this test) — the
// point is the regenerated file carries the full Port 1 contract (source + verified).
const stamped = stampVerifiedMeta({
title: 'AI Foundry Disaster Recovery Planning',
status: 'GA',
category,
source: 'https://learn.microsoft.com/azure/ai-foundry/concepts/disaster-recovery',
lastUpdated: '2026-06',
}, body);
}, { pass: true }, '2026-06-29');
const regenerated = composeKbFile(stamped, body);
// Strictly better than the original: now valid (Source present + TOC for the large
// body), still dated. validateKbFile requires a TOC on large files, so valid:true
// Strictly better than the original: now valid (Source + Verified present + TOC for the
// large body), still dated. validateKbFile requires a TOC on large files, so valid:true
// here is also the Fase 1c anti-regression proof.
const v = validateKbFile(regenerated);
assert.equal(v.valid, true, `regenerated file invalid: ${v.missing}`);

View file

@ -6,7 +6,12 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { parseSourceHeader } from '../../scripts/kb-update/lib/kb-headers.mjs';
import {
parseSourceHeader,
parseTypeHeader,
parseVerifiedHeader,
parseVerifiedByHeader,
} from '../../scripts/kb-update/lib/kb-headers.mjs';
test('parseSourceHeader — **Source:** header', () => {
const content = '# Title\n\n**Source:** https://learn.microsoft.com/azure/foundry/concepts/x\n**Status:** GA\n';
@ -50,3 +55,62 @@ test('parseSourceHeader — trims trailing whitespace from URL', () => {
const content = '**Source:** https://learn.microsoft.com/azure/trim \n';
assert.equal(parseSourceHeader(content), 'https://learn.microsoft.com/azure/trim');
});
// --- parseTypeHeader: the Port 1 frontmatter-contract `type` field --------------
// Spor 3 Port 1: every file carries type (reference|template|methodology|regulatory).
// Same top-500-byte bold-label scan as parseSourceHeader (one source of truth).
test('parseTypeHeader — **Type:** header', () => {
const content = '# T\n\n**Status:** GA\n**Type:** reference\n**Source:** https://learn.microsoft.com/x\n';
assert.equal(parseTypeHeader(content), 'reference');
});
test('parseTypeHeader — case-insensitive label, lowercased value', () => {
assert.equal(parseTypeHeader('**type:** Methodology\n'), 'methodology');
});
test('parseTypeHeader — no header returns null', () => {
assert.equal(parseTypeHeader('# T\n\n**Status:** GA\n'), null);
assert.equal(parseTypeHeader(''), null);
assert.equal(parseTypeHeader(null), null);
});
test('parseTypeHeader — only scans the top of the file (header region)', () => {
const buried = '# T\n' + 'x'.repeat(800) + '\n**Type:** template\n';
assert.equal(parseTypeHeader(buried), null);
});
// --- parseVerifiedHeader: the Port 1 `verified` date (born-verified stamp) ------
test('parseVerifiedHeader — full date', () => {
assert.equal(parseVerifiedHeader('**Verified:** 2026-06-29\n'), '2026-06-29');
});
test('parseVerifiedHeader — YYYY-MM date', () => {
assert.equal(parseVerifiedHeader('**Verified:** 2026-06\n'), '2026-06');
});
test('parseVerifiedHeader — does NOT match the "**Verified by:**" line', () => {
// "Verified by:" is a distinct label; the date parser must not swallow it.
assert.equal(parseVerifiedHeader('**Verified by:** judge-v2\n'), null);
});
test('parseVerifiedHeader — no header / garbage returns null', () => {
assert.equal(parseVerifiedHeader('**Status:** GA\n'), null);
assert.equal(parseVerifiedHeader(null), null);
});
// --- parseVerifiedByHeader: the Port 1 `verified_by` field (judge-vN | human) ---
test('parseVerifiedByHeader — judge version', () => {
assert.equal(parseVerifiedByHeader('**Verified by:** judge-v2\n'), 'judge-v2');
});
test('parseVerifiedByHeader — human', () => {
assert.equal(parseVerifiedByHeader('**Verified by:** human\n'), 'human');
});
test('parseVerifiedByHeader — no header returns null', () => {
assert.equal(parseVerifiedByHeader('**Verified:** 2026-06-29\n'), null);
assert.equal(parseVerifiedByHeader(''), null);
});

View file

@ -24,8 +24,14 @@ import {
buildToc,
composeKbFile,
insertToc,
stampVerifiedMeta,
} from '../../scripts/kb-update/lib/transform.mjs';
import { parseSourceHeader } from '../../scripts/kb-update/lib/kb-headers.mjs';
import {
parseSourceHeader,
parseTypeHeader,
parseVerifiedHeader,
parseVerifiedByHeader,
} from '../../scripts/kb-update/lib/kb-headers.mjs';
import { classifyChange } from '../../scripts/kb-update/lib/verify-out.mjs';
import { loadTaxonomy } from '../../scripts/kb-update/lib/taxonomy.mjs';
// The N4 (TOC-in-large-files) check is the cross-subsystem best-practice contract.
@ -44,12 +50,17 @@ const SHORT_BODY = '## Introduksjon\n\nKort.\n';
const __dirname = dirname(fileURLToPath(import.meta.url));
// A born-verified reference meta: Spor 3 Port 1 contract adds type/verified/verified_by
// (reference files are "born verified" — stamped only after the v2 judge clears them).
const META = {
title: 'Azure AI Search - Hybrid Retrieval',
status: 'GA',
category: 'rag-architecture',
type: 'reference',
source: 'https://learn.microsoft.com/azure/search/hybrid-search-overview',
lastUpdated: '2026-06',
verified: '2026-06-29',
verified_by: 'judge-v2',
};
// --- buildKbHeader: mandatory fields, parseable by the real header scanners ----
@ -76,6 +87,154 @@ test('buildKbHeader throws when a mandatory field is missing (status-felt obliga
assert.throws(() => buildKbHeader({ ...META, lastUpdated: undefined }), /last.?updated/i);
});
// --- Spor 3 Port 1/Port 2: type-aware create-guard contract --------------------
// Every reference file is "born verified": carries Type + Verified + Verified by in
// the header, all within the top-500-byte region the scanners read.
test('buildKbHeader emits Type, Verified, Verified by in the header (Port 1 contract)', () => {
const h = buildKbHeader(META);
assert.match(h, /\*\*Type:\*\*\s*reference/);
assert.match(h, /\*\*Verified:\*\*\s*2026-06-29/);
assert.match(h, /\*\*Verified by:\*\*\s*judge-v2/);
});
test('buildKbHeader: the new fields are parseable in the top-500-byte region', () => {
const h = buildKbHeader(META);
assert.equal(parseTypeHeader(h), 'reference');
assert.equal(parseVerifiedHeader(h), '2026-06-29');
assert.equal(parseVerifiedByHeader(h), 'judge-v2');
});
test('buildKbHeader: Verified by (the last header line) stays inside the 500-byte scan region for a realistic long title + URL', () => {
// Verified by sits after Source (the long field) — guard that a realistic worst case
// still fits, so a valid file is never wrongly flagged as missing verified_by.
const h = buildKbHeader({
title: 'Azure AI Foundry Model Router and Deployment Strategies for Enterprise',
status: 'Public Preview',
category: 'azure-ai-services',
type: 'reference',
source: 'https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/model-router-deployment-strategies-overview',
lastUpdated: '2026-06',
verified: '2026-06-29',
verified_by: 'judge-v2',
});
assert.equal(parseVerifiedByHeader(h), 'judge-v2', 'verified_by fell outside the 500-byte header region');
});
test('buildKbHeader: a reference file MUST carry verified + verified_by (born-verified guard)', () => {
assert.throws(() => buildKbHeader({ ...META, verified: undefined }), /verified/i);
assert.throws(() => buildKbHeader({ ...META, verified_by: '' }), /verified_by|verified by/i);
});
test('buildKbHeader: type defaults to reference when omitted (back-compat for callers)', () => {
const { type, ...noType } = META;
const h = buildKbHeader(noType);
assert.equal(parseTypeHeader(h), 'reference');
});
test('buildKbHeader: an unknown type is rejected', () => {
assert.throws(() => buildKbHeader({ ...META, type: 'blogpost' }), /type/i);
});
test('buildKbHeader: a non-reference file (methodology) needs NO source/verified', () => {
// Template/methodology/regulatory are out of correctness-scope — they carry no MS source.
const h = buildKbHeader({
title: 'ROS-metodikk (NS 5814)',
status: 'Stable',
category: 'norwegian-public-sector-ai-governance',
type: 'methodology',
lastUpdated: '2026-06',
});
assert.match(h, /\*\*Type:\*\*\s*methodology/);
assert.doesNotMatch(h, /\*\*Source:\*\*/);
assert.doesNotMatch(h, /\*\*Verified:\*\*/);
});
test('buildKbHeader: a non-reference file may NOT carry a Microsoft Learn source (contract)', () => {
assert.throws(
() => buildKbHeader({
title: 'X', status: 'Stable', category: 'c', type: 'template',
lastUpdated: '2026-06', source: 'https://learn.microsoft.com/azure/x',
}),
/source/i,
);
});
// --- validateKbFile: type-aware enforcement of the full contract ---------------
test('validateKbFile: a reference file missing verified/verified_by is invalid', () => {
// Header has Source but no Verified / Verified by → not born-verified → rejected.
const file =
'# T\n\n**Last updated:** 2026-06\n**Status:** GA\n**Type:** reference\n' +
'**Source:** https://learn.microsoft.com/x\n\n---\n\n## A\n\ntekst\n';
const r = validateKbFile(file);
assert.equal(r.valid, false);
assert.ok(r.missing.includes('verified'), `expected 'verified' in ${r.missing}`);
assert.ok(r.missing.includes('verified_by'), `expected 'verified_by' in ${r.missing}`);
});
test('validateKbFile: a complete born-verified reference file is valid', () => {
const file = buildKbHeader(META) + '\n## Introduksjon\n\nNoe innhold.\n';
assert.equal(validateKbFile(file).valid, true);
});
test('validateKbFile: a methodology file without source/verified is valid (out of scope)', () => {
const file =
'# ROS-metodikk\n\n**Last updated:** 2026-06\n**Status:** Stable\n**Type:** methodology\n\n' +
'---\n\n## Metode\n\nNS 5814.\n';
const r = validateKbFile(file);
assert.equal(r.valid, true, `unexpected missing: ${r.missing}`);
});
// --- stampVerifiedMeta: the born-verified gate (judge clears → stamp; else refuse) --
// transform.mjs never calls the judge (pure lib). The command runs the v2 claim judge,
// aggregates per-claim results into {pass}, and calls this. A failing verdict throws —
// "nytt innhold gjennom judgen FØR commit; ellers ingen fil." (§4 Port 2.)
test('stampVerifiedMeta stamps verified=today + verified_by=judge-vN on a passing verdict', () => {
const base = { title: 'T', status: 'GA', category: 'c',
source: 'https://learn.microsoft.com/x', lastUpdated: '2026-06' };
const m = stampVerifiedMeta(base, { pass: true }, '2026-06-29');
assert.equal(m.type, 'reference');
assert.equal(m.verified, '2026-06-29');
assert.equal(m.verified_by, 'judge-v2'); // current GATE-PASS judge
});
test('stampVerifiedMeta honours an explicit judgeVersion', () => {
const base = { title: 'T', status: 'GA', category: 'c',
source: 'https://learn.microsoft.com/x', lastUpdated: '2026-06' };
assert.equal(stampVerifiedMeta(base, { pass: true, judgeVersion: 3 }, '2026-06-29').verified_by, 'judge-v3');
});
test('stampVerifiedMeta supports a human verification path', () => {
const base = { title: 'T', status: 'GA', category: 'c',
source: 'https://learn.microsoft.com/x', lastUpdated: '2026-06' };
assert.equal(stampVerifiedMeta(base, { pass: true, by: 'human' }, '2026-06-29').verified_by, 'human');
});
test('stampVerifiedMeta REFUSES to stamp when the judge verdict is not passing', () => {
const base = { title: 'T', status: 'GA', category: 'c',
source: 'https://learn.microsoft.com/x', lastUpdated: '2026-06' };
assert.throws(() => stampVerifiedMeta(base, { pass: false }, '2026-06-29'), /judge|verdict|verified/i);
assert.throws(() => stampVerifiedMeta(base, null, '2026-06-29'), /judge|verdict|verified/i);
});
test('stampVerifiedMeta rejects an invalid today date', () => {
const base = { title: 'T', status: 'GA', category: 'c',
source: 'https://learn.microsoft.com/x', lastUpdated: '2026-06' };
assert.throws(() => stampVerifiedMeta(base, { pass: true }, 'i går'), /date/i);
});
test('stampVerifiedMeta → composeKbFile is a complete born-verified file (end-to-end)', () => {
const base = { title: 'Azure AI Search', status: 'GA', category: 'rag-architecture',
source: 'https://learn.microsoft.com/azure/search/x', lastUpdated: '2026-06' };
const stamped = stampVerifiedMeta(base, { pass: true }, '2026-06-29');
const file = composeKbFile(stamped, '## Introduksjon\n\nKort.\n');
assert.equal(validateKbFile(file).valid, true);
assert.equal(parseVerifiedHeader(file), '2026-06-29');
assert.equal(parseVerifiedByHeader(file), 'judge-v2');
});
// --- validateKbFile: catches files missing the mandatory contract -------------
test('validateKbFile accepts a header-conformant file', () => {

View file

@ -0,0 +1,80 @@
// tests/kb-update/test-validate-kb-file.test.mjs
// The executable create-guard (Spor 3 Port 2): scripts/kb-update/validate-kb-file.mjs
// is the gate any generator calls AFTER writing a candidate file and BEFORE committing.
// It wraps transform.validateKbFile over one or more paths and exits non-zero if any
// file violates the contract (no source / not born-verified / large file without a TOC).
//
// validatePaths is a pure function with an injectable reader, so these tests never
// touch disk — they feed synthetic content and assert the verdict + ok flag.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { validatePaths } from '../../scripts/kb-update/validate-kb-file.mjs';
import { buildKbHeader } from '../../scripts/kb-update/lib/transform.mjs';
const GOOD = buildKbHeader({
title: 'Azure AI Search - Hybrid Retrieval',
status: 'GA',
category: 'rag-architecture',
type: 'reference',
source: 'https://learn.microsoft.com/azure/search/hybrid-search-overview',
lastUpdated: '2026-06',
verified: '2026-06-29',
verified_by: 'judge-v2',
}) + '\n## Introduksjon\n\nNoe innhold.\n';
// A reference file with a Source but NOT born-verified (no Verified / Verified by).
const NOT_VERIFIED =
'# T\n\n**Last updated:** 2026-06\n**Status:** GA\n**Type:** reference\n' +
'**Source:** https://learn.microsoft.com/x\n\n---\n\n## A\n\ntekst\n';
// A reference file with no Source header at all (the 0%-coverage failure mode).
const NO_SOURCE = '# T\n\n**Last updated:** 2026-06\n**Status:** GA\n\n---\n\n## A\n\ntekst\n';
function reader(map) {
return (p) => {
if (!(p in map)) throw new Error(`unexpected read: ${p}`);
return map[p];
};
}
test('validatePaths — all files valid → ok:true', () => {
const r = validatePaths(['a.md', 'b.md'], reader({ 'a.md': GOOD, 'b.md': GOOD }));
assert.equal(r.ok, true);
assert.equal(r.results.length, 2);
assert.ok(r.results.every((x) => x.valid));
});
test('validatePaths — a file missing source fails the gate', () => {
const r = validatePaths(['bad.md'], reader({ 'bad.md': NO_SOURCE }));
assert.equal(r.ok, false);
assert.equal(r.results[0].valid, false);
assert.ok(r.results[0].missing.includes('source'));
});
test('validatePaths — a reference file that is not born-verified fails the gate', () => {
const r = validatePaths(['nv.md'], reader({ 'nv.md': NOT_VERIFIED }));
assert.equal(r.ok, false);
assert.ok(r.results[0].missing.includes('verified'));
assert.ok(r.results[0].missing.includes('verified_by'));
});
test('validatePaths — one bad file in a batch fails the whole gate', () => {
const r = validatePaths(['ok.md', 'bad.md'], reader({ 'ok.md': GOOD, 'bad.md': NO_SOURCE }));
assert.equal(r.ok, false);
assert.equal(r.results.find((x) => x.path === 'ok.md').valid, true);
assert.equal(r.results.find((x) => x.path === 'bad.md').valid, false);
});
test('validatePaths — a path that cannot be read is reported as invalid, not thrown', () => {
const r = validatePaths(['missing.md'], () => { throw new Error('ENOENT'); });
assert.equal(r.ok, false);
assert.equal(r.results[0].valid, false);
assert.ok(r.results[0].missing.some((m) => /read|enoent/i.test(m)));
});
test('validatePaths — no paths → ok:true (nothing to gate)', () => {
const r = validatePaths([], reader({}));
assert.equal(r.ok, true);
assert.deepEqual(r.results, []);
});