feat(ms-ai-architect): G7-ankere kan baere sin egen fil — klasse-entryen for de 42 er skrevet, og #17 og #14 er begge oppfylt
Operator ratifiserte utvidelsen 2026-08-11. Beslutning #17 (én klasse-entry) og grunnen bak #14 (ingen lokator usynlig for gaten) var begge ekte og kolliderte i skjemaet. De kolliderer ikke lenger. SKJEMAET: et anker er enten en ordrett STRENG, sjekket mot entry.file som foer, eller et { file, text }-PAR sjekket mot SIN EGEN fil. Bakoverkompatibelt — alle 47 eksisterende entries er uroert. Et misformet par er en schema-feil, ikke et anker som stille matcher ingenting. TDD, og testene maatte skjerpes foer de var ekte: fem tester skrevet og kjoert FOERST. To av dem PASSERTE mot gammel kode — av feil grunn: den gamle koden stringify-er ankerobjektet inn i drift-meldingen, saa /other\.md/ traff tilfeldig. En test som ikke kan feile beviser ingenting. Begge fikk en diskriminator (meldingen skal IKKE navne entry.file) og feilet deretter. 5 feilende -> implementasjon -> 20/20 i fila, 1052/1052 i suiten. idx-26an: 42 { file, text }-ankere, ett per medlem, hvert re-verifisert ordrett OG unikt i sin egen fil med split-telling foer skriving. MEKANISMEN ER BEVIST, IKKE ANTATT: kjoert mot den ekte validatoren med ett medlems anker fjernet i en lesestubb — ok=false, ett anchor_drift, riktig entry-id, og meldingen navner agent-evaluation-testing-frameworks.md. Det er nettopp garantien #14 fantes for, naa baaret av én entry i stedet for 42. MAALT HASARD BOKFOERT I ENTRYEN: to av de 42 ankertekstene er strenge prefikser av andre medlemmers ankere («**Total MCP calls:** 6» i ai-services-cost-optimization.md, «**MCP Calls:** 3» i reserved-capacity-planning.md). Hver er unik i SIN fil, saa koen er trygg — en kryss-fil search-and-replace er det ikke. Slett per fil, og rest-soek etter hver edit. Koen: 47 -> 48 entries (28 open, 20 resolved). Bokfoeringen av #17 er dermed komplett: 1 klasse-entry + 16 individuelle = 17. Ingen korpusfil roert.
This commit is contained in:
parent
1c07afd056
commit
8ea54ec00e
3 changed files with 287 additions and 12 deletions
|
|
@ -645,6 +645,185 @@
|
||||||
"- 9 unique source URLs from Microsoft Learn",
|
"- 9 unique source URLs from Microsoft Learn",
|
||||||
"- Coverage: Prompt Shields, Security Benchmark (AI-2, AI-3), LLM Security Planning, Content Filtering"
|
"- Coverage: Prompt Shields, Security Benchmark (AI-2, AI-3), LLM Security Planning, Content Filtering"
|
||||||
]
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "idx-26an",
|
||||||
|
"file": "scripts/kb-eval/data/r11-footer-class-2026-08-11.json",
|
||||||
|
"class": "multi-locator",
|
||||||
|
"status": "open",
|
||||||
|
"raised": "2026-08-11",
|
||||||
|
"summary": "CLASS-WIDE ENTRY. 42 members in 42 files, all line-form, none flagged, none carrying an open decision - the remainder of the footer counting class after the 16 that needed their own entry (idx-26x, idx-26y, idx-26z, idx-26aa..idx-26am). Measured in docs sec. 9.16: the full class is 58 members in 58 files, dataset scripts/kb-eval/data/r11-footer-class-2026-08-11.json (tracked), buckets 30 konsistent / 11 inkonsistent / 1 tvetydig / 16 ikke sjekkbar.\nTHE FORM APPLIES TO THESE WITHOUT FURTHER JUDGEMENT. Ratified on idx-26j 2026-08-09: current provenance, one referent for the whole footer block, and the unverifiable MCP-calls line is DELETED rather than corrected to the sum. THE GROUND IS UNVERIFIABILITY, NOT ARITHMETIC - no artifact in the repo records MCP calls per file, so the number is unadjudicable under both readings of the referent. The consistency bucket therefore governs BOOKKEEPING ONLY and never whether a line is deleted; a konsistent member goes for the same reason an inkonsistent one does.\nWHY ONE ENTRY AND NOT 42. Operator decision #17, 2026-08-11, overriding decision #14 for exactly this subset: #14 (\"one entry per file\") was taken when the class was believed to be 14 files, and 42 near-identical entries are the mechanism sec. 9.15 showed lets a mismeasurement hide behind its own copies - seven entries there cited each other. #14 also had a REAL reason, re-measured and honoured here rather than discarded: the queue schema carried one `file` per entry, so a class-wide entry would have left these files invisible to check-g7-queue.mjs - the 795d494 failure, a locator alive only in prose while both gates go green. Measured empirically against the live validator before this entry was written: a class entry with 42 string anchors returned ok=false with 41 of 42 reported as anchor_drift. Both reasons were true and they collided.\nRESOLVED BY EXTENDING THE SCHEMA, operator-ratified 2026-08-11: an anchor may now be a { file, text } pair checked against ITS OWN file, alongside the existing string form checked against entry.file. Backwards compatible - every pre-existing entry is untouched - and it satisfies both constraints at once: one record, and all 42 locators still machine-checked. Five tests written FIRST and observed failing; two of them initially passed against the old code for the wrong reason (it serialised the anchor object into the drift message, so the file-name regex matched by accident) and were tightened until they discriminated.\nANCHORS: 42 { file, text } pairs, one per member, each re-verified verbatim AND unique within its own file by split-count before writing - the gate proves presence, never uniqueness.\nHAZARD FOR WHOEVER APPLIES THE DELETIONS, measured not assumed: two of the 42 anchor texts are STRICT PREFIXES of other members' anchors - \"**Total MCP calls:** 6\" in ai-services-cost-optimization.md is a prefix of the longer lines in multi-agent-orchestration-patterns.md and continuous-improvement-feedback-loops.md, and \"**MCP Calls:** 3\" in reserved-capacity-planning.md is a prefix of the line in prompt-testing-and-evaluation.md. Each is unique inside its own file, so this queue is safe; a cross-file search-and-replace is NOT. Delete per file, and run a rest-search in each file after its edit, because a deletion can leave the number alive elsewhere in the same file (idx-26ab is the measured instance).\nNOT MACHINE-APPLIABLE. Nothing in this queue is, by definition. apply-o2-ratified.mjs is delete-only and aborts here. Closing this entry means recording, in the resolution, which files were edited and what was verified after each edit.",
|
||||||
|
"evidence": "scripts/kb-eval/data/r11-footer-class-2026-08-11.json (the 42 are the members with form=line and flags=[]); docs sec. 9.16 (the measurement: four sweeps with different blind spots, residual of each read by hand, label vocabulary enumerated at 68 distinct labels / 141 lines); idx-26j resolution (the ratified form); idx-26q (the #14 reasoning and the 795d494 failure class); docs sec. 9.15 (duplication as the place a mismeasurement hides); commit 1c07afd (the 16 individual entries)",
|
||||||
|
"anchors": [
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/copilot-extensibility/adaptive-cards-copilot-responses.md",
|
||||||
|
"text": "**MCP calls:** 3 docs_search, 2 docs_fetch, 1 code_sample_search"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/copilot-extensibility/copilot-api-rate-limiting-resilience.md",
|
||||||
|
"text": "**MCP Calls:** 6 (3 searches, 2 fetches, 1 code sample search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/copilot-extensibility/declarative-agents-grounding-strategies.md",
|
||||||
|
"text": "**MCP-kall:** 7 (3 search, 3 fetch, 1 code sample search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/copilot-extensibility/sharepoint-copilot-agents.md",
|
||||||
|
"text": "**MCP-calls:** 5 (3 search + 2 fetch)."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/copilot-extensibility/teams-copilot-message-extensions.md",
|
||||||
|
"text": "**MCP-kall utført:** 6 (3 search, 2 fetch, 1 code sample search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/prompt-engineering/multi-turn-conversation-management.md",
|
||||||
|
"text": "**MCP calls:** 5 (search + fetch)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-advisor/references/prompt-engineering/prompt-testing-and-evaluation.md",
|
||||||
|
"text": "**MCP Calls:** 3 (microsoft_docs_search × 2, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/agent-orchestration/agent-evaluation-testing-frameworks.md",
|
||||||
|
"text": "- **MCP calls:** 3 (microsoft_docs_search) + 2 (microsoft_docs_fetch) + 1 (microsoft_code_sample_search) = 6"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/agent-orchestration/agent-memory-and-context-management.md",
|
||||||
|
"text": "**MCP calls**: 6 (3x microsoft_docs_search, 2x microsoft_docs_fetch, 1x microsoft_code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/agent-orchestration/foundry-workflows-visual-orchestration.md",
|
||||||
|
"text": "**MCP calls**: 4 (2x docs_search, 2x docs_fetch)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/agent-orchestration/multi-agent-orchestration-patterns.md",
|
||||||
|
"text": "**Total MCP calls:** 6 (3 microsoft_docs_search + 2 microsoft_docs_fetch + 1 microsoft_code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-api-best-practices.md",
|
||||||
|
"text": "**MCP call summary:** 7 microsoft_docs_search + 4 microsoft_docs_fetch + 1 microsoft_code_sample_search = 12 total MCP calls"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-cost-optimization.md",
|
||||||
|
"text": "**Total MCP calls:** 6"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-governance-compliance.md",
|
||||||
|
"text": "**Total antall MCP-kall:** 8 (4 docs_search + 4 docs_fetch)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/ai-services-networking-security.md",
|
||||||
|
"text": "**MCP calls:** 7 (microsoft_docs_search, microsoft_docs_fetch, microsoft_code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/azure-ai-vision-image-analysis.md",
|
||||||
|
"text": "**MCP-kall totalt:** 4 (3 docs_search + 1 code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/document-intelligence-prebuilt-models.md",
|
||||||
|
"text": "**Totalt MCP-kall:** 5 (3× search, 2× fetch, 1× code samples)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/speech-services-text-to-speech.md",
|
||||||
|
"text": "**Totalt antall MCP-kall:** 7 (4 × docs_search, 3 × docs_fetch, 1 × code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/azure-ai-services/translator-document-translation.md",
|
||||||
|
"text": "**Total MCP calls:** 4 (docs_search) + 3 (docs_fetch) = **7**"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/mlops-genaiops/data-drift-monitoring-detection.md",
|
||||||
|
"text": "**MCP Calls:** 5 (3 × microsoft_docs_search, 1 × microsoft_docs_fetch, 1 × microsoft_code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/mlops-genaiops/genaiops-llm-specific-practices.md",
|
||||||
|
"text": "**Totalt:** 18 kilder, 8 MCP-kall."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-security-access-control.md",
|
||||||
|
"text": "**MCP Calls:** 8 (microsoft-learn docs search + fetch, code samples)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/mlops-genaiops/mlops-teams-collaboration-tools.md",
|
||||||
|
"text": "**Totalt antall MCP-kall:** 6 (3x search, 2x fetch, 1x code samples)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/mlops-genaiops/model-deployment-strategies-azure.md",
|
||||||
|
"text": "**MCP-kall utført:** 8 (microsoft_docs_search × 5, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/mlops-genaiops/responsible-ai-mlops-integration.md",
|
||||||
|
"text": "**MCP-calls brukt:** 6 (microsoft_docs_search x 3, microsoft_docs_fetch x 2, microsoft_code_sample_search x 1)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-engineering/references/rag-architecture/rag-cost-optimization.md",
|
||||||
|
"text": "**MCP calls:** 3 (search) + 2 (fetch) = 5 total"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-governance/references/monitoring-observability/real-time-streaming-monitoring.md",
|
||||||
|
"text": "**MCP calls:** 6 (3 × search, 2 × fetch, 1 × code search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-governance/references/monitoring-observability/response-quality-metrics-rag.md",
|
||||||
|
"text": "**MCP research calls:** 3 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-governance/references/responsible-ai/continuous-improvement-feedback-loops.md",
|
||||||
|
"text": "**Total MCP calls:** 6 (3 searches + 2 fetches + 1 code sample search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-governance/references/responsible-ai/human-in-the-loop-oversight.md",
|
||||||
|
"text": "**MCP Calls:** 6 (3 searches + 2 fetches + 1 code sample search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-governance/references/responsible-ai/responsible-ai-policy-development.md",
|
||||||
|
"text": "**MCP Calls:** 4 (microsoft_docs_search x3, microsoft_docs_fetch x2)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/ai-security-engineering/ai-security-scoring-framework.md",
|
||||||
|
"text": "**MCP calls:** 5 (3 søk + 2 fetch)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/ai-security-engineering/content-safety-filter-calibration.md",
|
||||||
|
"text": "**MCP-kall:** 6 (3x microsoft_docs_search, 2x microsoft_docs_fetch, 1x microsoft_code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/ai-security-engineering/norwegian-content-safety.md",
|
||||||
|
"text": "**MCP-kall:** 6 (microsoft_docs_search x6)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/ai-security-engineering/output-validation-grounding-verification.md",
|
||||||
|
"text": "**MCP-kall utført:** 4 (2x docs_search, 1x code_sample_search, 2x docs_fetch)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/azure-ai-foundry-cost-governance.md",
|
||||||
|
"text": "**Total MCP Calls:** 4 (3x microsoft_docs_search, 1x microsoft_docs_fetch, 1x microsoft_code_sample_search)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/budget-forecasting-ai-projects.md",
|
||||||
|
"text": "**Total MCP calls:** 3 (docs_search) + 2 (docs_fetch) + 1 (code_sample_search) = 6"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/inference-endpoint-cost-optimization.md",
|
||||||
|
"text": "**Totalt MCP-kall:** 3 (microsoft_docs_search) + 2 (microsoft_docs_fetch) + 1 (microsoft_code_sample_search) = 6"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/model-selection-price-performance.md",
|
||||||
|
"text": "**MCP-kall brukt:** 6 (4x docs_search, 2x docs_fetch)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/reserved-capacity-planning.md",
|
||||||
|
"text": "**MCP Calls:** 3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/small-language-models-economics.md",
|
||||||
|
"text": "**Total MCP-kall:** 4 (3x search, 2x fetch, 1x code samples)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"file": "skills/ms-ai-security/references/cost-optimization/token-counting-optimization.md",
|
||||||
|
"text": "**MCP Calls:** 4 (microsoft_docs_search × 3, microsoft_docs_fetch × 2, microsoft_code_sample_search × 1)"
|
||||||
|
}
|
||||||
|
]
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -23,6 +23,9 @@ export const ENTRY_STATES = new Set(['open', 'resolved']);
|
||||||
|
|
||||||
const REQUIRED = ['id', 'file', 'class', 'status', 'raised', 'summary', 'evidence', 'anchors'];
|
const REQUIRED = ['id', 'file', 'class', 'status', 'raised', 'summary', 'evidence', 'anchors'];
|
||||||
|
|
||||||
|
const isAnchor = (a) =>
|
||||||
|
typeof a === 'string' || (a !== null && typeof a === 'object' && typeof a.file === 'string' && typeof a.text === 'string');
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @param {unknown} entries queue entries
|
* @param {unknown} entries queue entries
|
||||||
* @param {(path: string) => string} readFile
|
* @param {(path: string) => string} readFile
|
||||||
|
|
@ -78,21 +81,50 @@ export function validateQueue(entries, readFile) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
let text;
|
// An anchor is either a verbatim string, checked against entry.file, or a
|
||||||
try {
|
// { file, text } pair checked against ITS OWN file. The pair form exists for
|
||||||
text = readFile(entry.file);
|
// class-wide entries — one record for a defect that lives in many files.
|
||||||
} catch {
|
// Measured 2026-08-11: booking such a class as a single-file entry hid 41 of
|
||||||
findings.push({ id, kind: 'file_unreadable', message: `cannot read ${entry.file}` });
|
// 42 locators from this check, which is the 795d494 failure, a locator alive
|
||||||
|
// only in prose while both gates go green. A malformed pair is a schema fault
|
||||||
|
// rather than an anchor that quietly matches nothing.
|
||||||
|
const malformed = entry.anchors.filter((a) => !isAnchor(a));
|
||||||
|
if (malformed.length > 0) {
|
||||||
|
findings.push({
|
||||||
|
id,
|
||||||
|
kind: 'schema',
|
||||||
|
message: `anchor must be a verbatim string or a { file, text } pair: ${malformed
|
||||||
|
.map((m) => JSON.stringify(m))
|
||||||
|
.join(', ')}`,
|
||||||
|
});
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
const gone = entry.anchors.filter((a) => !text.includes(a));
|
const byFile = new Map();
|
||||||
if (gone.length > 0) {
|
for (const a of entry.anchors) {
|
||||||
findings.push({
|
const target = typeof a === 'string' ? entry.file : a.file;
|
||||||
id,
|
const text = typeof a === 'string' ? a : a.text;
|
||||||
kind: 'anchor_drift',
|
if (!byFile.has(target)) byFile.set(target, []);
|
||||||
message: `anchor no longer occurs in ${entry.file}: ${gone.map((g) => JSON.stringify(g)).join(', ')}`,
|
byFile.get(target).push(text);
|
||||||
});
|
}
|
||||||
|
|
||||||
|
for (const [target, texts] of byFile) {
|
||||||
|
let text;
|
||||||
|
try {
|
||||||
|
text = readFile(target);
|
||||||
|
} catch {
|
||||||
|
findings.push({ id, kind: 'file_unreadable', message: `cannot read ${target}` });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const gone = texts.filter((t) => !text.includes(t));
|
||||||
|
if (gone.length > 0) {
|
||||||
|
findings.push({
|
||||||
|
id,
|
||||||
|
kind: 'anchor_drift',
|
||||||
|
message: `anchor no longer occurs in ${target}: ${gone.map((g) => JSON.stringify(g)).join(', ')}`,
|
||||||
|
});
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -119,6 +119,70 @@ test('a non-array queue is rejected without throwing', () => {
|
||||||
assert.match(r.findings[0].message, /array/i);
|
assert.match(r.findings[0].message, /array/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// A class-wide entry books ONE finding that lives in many files. Booking it as a
|
||||||
|
// single-file entry was measured to hide 41 of 42 locators from the gate — the
|
||||||
|
// 795d494 failure, a locator alive only in prose while both gates go green. So an
|
||||||
|
// anchor may also be a { file, text } pair, checked against ITS OWN file.
|
||||||
|
test('a { file, text } anchor is checked against its own file, not entry.file', () => {
|
||||||
|
const e = { ...okEntry(), anchors: [{ file: 'skills/x/references/other.md', text: 'over here' }] };
|
||||||
|
const r = validateQueue(
|
||||||
|
[e],
|
||||||
|
stub({ 'skills/x/references/y.md': 'nothing matching', 'skills/x/references/other.md': 'aaa over here bbb' }),
|
||||||
|
);
|
||||||
|
assert.equal(r.ok, true);
|
||||||
|
assert.deepEqual(r.findings, []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a { file, text } anchor absent from its own file DRIFTS, and the finding names that file', () => {
|
||||||
|
const e = { ...okEntry(), anchors: [{ file: 'skills/x/references/other.md', text: 'over here' }] };
|
||||||
|
const r = validateQueue(
|
||||||
|
[e],
|
||||||
|
stub({ 'skills/x/references/y.md': 'over here', 'skills/x/references/other.md': 'empty' }),
|
||||||
|
);
|
||||||
|
assert.equal(r.ok, false);
|
||||||
|
assert.equal(r.findings[0].kind, 'anchor_drift');
|
||||||
|
assert.match(r.findings[0].message, /other\.md/);
|
||||||
|
// Discriminating: entry.file DOES contain the text, so a message blaming y.md
|
||||||
|
// would mean the anchor was looked up in the wrong file. Without this line the
|
||||||
|
// test passes against the old code, which serialises the object into the
|
||||||
|
// message and matches /other\.md/ by accident.
|
||||||
|
assert.doesNotMatch(r.findings[0].message, /y\.md/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('string and { file, text } anchors mix in one entry, and BOTH are checked', () => {
|
||||||
|
const e = { ...okEntry(), anchors: ['the exact text', { file: 'skills/x/references/other.md', text: 'gone' }] };
|
||||||
|
const r = validateQueue(
|
||||||
|
[e],
|
||||||
|
stub({ 'skills/x/references/y.md': 'the exact text', 'skills/x/references/other.md': 'not it' }),
|
||||||
|
);
|
||||||
|
assert.equal(r.ok, false);
|
||||||
|
assert.equal(r.findings.length, 1);
|
||||||
|
assert.match(r.findings[0].message, /gone/);
|
||||||
|
// Discriminating, for the same reason: the surviving string anchor is present
|
||||||
|
// in entry.file, so the drift must be reported against other.md.
|
||||||
|
assert.match(r.findings[0].message, /other\.md/);
|
||||||
|
assert.doesNotMatch(r.findings[0].message, /y\.md/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an unreadable per-anchor file is reported, not thrown, and names that file', () => {
|
||||||
|
const e = { ...okEntry(), anchors: [{ file: 'skills/x/references/missing.md', text: 'whatever' }] };
|
||||||
|
const r = validateQueue([e], stub({ 'skills/x/references/y.md': 'the exact text' }));
|
||||||
|
assert.equal(r.ok, false);
|
||||||
|
assert.equal(r.findings[0].kind, 'file_unreadable');
|
||||||
|
assert.match(r.findings[0].message, /missing\.md/);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Without this, a typo'd anchor object would be checked against nothing and the
|
||||||
|
// entry would pass — a defect leaving the programme unnoticed, which is the one
|
||||||
|
// outcome the queue exists to prevent.
|
||||||
|
test('a malformed anchor object is a schema fault, never a silent pass', () => {
|
||||||
|
const e = { ...okEntry(), anchors: [{ file: 'skills/x/references/other.md' }] };
|
||||||
|
const r = validateQueue([e], stub({ 'skills/x/references/y.md': 'x', 'skills/x/references/other.md': 'x' }));
|
||||||
|
assert.equal(r.ok, false);
|
||||||
|
assert.equal(r.findings[0].kind, 'schema');
|
||||||
|
assert.match(r.findings[0].message, /anchor/i);
|
||||||
|
});
|
||||||
|
|
||||||
test('the real queue file validates against the live corpus', async () => {
|
test('the real queue file validates against the live corpus', async () => {
|
||||||
const { readFileSync } = await import('node:fs');
|
const { readFileSync } = await import('node:fs');
|
||||||
const queue = JSON.parse(readFileSync('scripts/kb-eval/data/g7-review-queue.json', 'utf8'));
|
const queue = JSON.parse(readFileSync('scripts/kb-eval/data/g7-review-queue.json', 'utf8'));
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue