open/ms-ai-architect
1
0
Fork 0
Code Issues Releases 1 Activity

feat(ms-ai-architect): R11 §4b implementert — status-synonymtabellen målt, 8 forslag hånd-dømt [skip-docs]

Browse source 
Implementerer den ratifiserte §4b-tabellen i lib/fix-op.mjs (19 nye tester,
suite 996/996). Alle tre skrankene har egne tester: tabellen er LUKKET, fil-
tokenet må være en KOMPLETT livssyklus-etikett, og verdien som skrives er den
korpus-side ekvivalenten med filas egen markup bevart.

To implementasjonsvalg den ratifiserte teksten lot stå åpne, begge løst mot
fail-closed: status-lokatoren er LINJE-scopet (livssyklus-vokabular gjentas
nedover hver kolonne i en statustabell, så et blokkvindu er tvetydig ved
konstruksjon), og et sitat som hevder to rader aborterer.

MÅLT: 15 pilot / 54 korpus-brede flagg -> 5 og 8 provbare. Alle 8 hånd-dømt
mot kilden (r11-pilot-results.md appendiks B): 5 korrekte, 1 ubevist, 2 GALE.

De tre defektene er én familie: §4b binder tabellen, etikettens fullstendighet
og verdien som skrives — og INGENTING om hvorvidt kilde-frasen refererer til
radens eget subjekt. Samme proveniens-uten-referent-defekt som falsifiserte §4.
Klassen er derfor REVIEW-grade, ikke apply-grade: `status` står bevisst utenfor
o1_recommended, ingen driver applikerer den.

To kandidatvilkår er kostnadsberegnet over de åtte (begge dreper gale forslag
og null korrekte) men IKKE implementert — å utvide en tabell operatøren
ratifiserte som lukket er en operatørbeslutning, slik vilkår 5 var i §4a.

Rettet samtidig 2 NUL-bytes i testfila (pre-eksisterende, fra en tidligere
økt) som gjorde at git behandlet hele fila som binær og blokkerte diff-
gjennomgang før commit.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 17:12:47 +02:00
commit e4925c6b28
5 changed files with 330 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

36
scripts/kb-eval/classify-fix-ops.mjs
View file

@ -86,7 +86,21 @@ const r8 = items.filter((i) => i.rule === 'R8');
const LOCATOR_CODES = new Set([ABORT_CODES.LOCATOR_MISS, ABORT_CODES.LOCATOR_AMBIGUOUS]);
const locatorAborts = o3.filter((i) => LOCATOR_CODES.has(i.code)).length;
const s4O1 = s4Only.filter((v) => v.op === 'O1').length;
// §4-as-written is a NUMERIC-path measurement: it is the baseline the context
// condition (§4a) was added against. §4b status swaps do not run through the
// context condition at all, so counting them here would silently inflate the
// baseline and break the comparison with the pilot's hand-verified 6.
const isStatus = (v) => v.op === 'O1' && v.proposal.type === 'status';
const s4O1 = s4Only.filter((v) => v.op === 'O1' && !isStatus(v)).length;
const o1Numeric = o1.filter((i) => i.proposal.type !== 'status').length;
// §4b (ratified 2026-08-03): the STATUS_SYNONYM class, split into what the closed
// synonym table proves and why the remainder still aborts. The abort REASON
// sub-distribution is the actionable part — NO_COMPLETE_FILE_LABEL is a corpus
// shape, SOURCE_STATUS_AMBIGUOUS is a quote shape, FILE_ALREADY_MATCHES means the
// flag was never a status mismatch in the first place.
const statusProposals = items.filter((i) => i.op === 'O1' && i.proposal.type === 'status');
const statusAborts = o3.filter((i) => i.code === ABORT_CODES.STATUS_SYNONYM);
// O1 precision is NOT uniform across token types, and this split is the pilot's
// operational conclusion. Hand-verified over the whole not_grounded population:
@ -117,7 +131,19 @@ const report = {
s4_as_written: {
O1: s4O1,
note:
'What §4 exactly as written would admit. NOT a source of proposals: on the >=7 pilot all 6 were hand-verified and 4 were wrong edits (unit crossing, metric crossing, two mutilated identifiers) — measured precision 2/6. Runs at other thresholds carry no hand-verification.',
'What §4 exactly as written would admit on the NUMERIC path (§4b status swaps excluded — they never run through the context condition). NOT a source of proposals: on the >=7 pilot all 6 were hand-verified and 4 were wrong edits (unit crossing, metric crossing, two mutilated identifiers) — measured precision 2/6. Runs at other thresholds carry no hand-verification.',
},
status_synonym: {
contract: '§4b — the closed synonym table, ratified 2026-08-03',
class_total: statusProposals.length + statusAborts.length,
proven: statusProposals.length,
aborts: tally(statusAborts, (i) => (i.detail && i.detail.reason) || '(unspecified)'),
hand_verified:
THRESHOLD === 7
? 'All 5 hand-judged 2026-08-03 (docs/r11-pilot-results.md appendix B). Four carry the source phrasing on the row\'s OWN subject and are correct. One (security-copilot-integration.md:94) harvests a "(Preview)" marker that belongs to a DIFFERENT agent in an enumerated quote — the same provenance-without-referent defect that falsified §4. Its outcome is plausibly right; its proof is not.'
: 'hand-verification was done on the >=7 pilot only',
applicability:
'REVIEW-GRADE, NOT APPLY-GRADE. status is deliberately absent from o1_recommended: §4b binds the table, the completeness of the file label and the written value, and nothing about whether the source phrasing refers to the row\'s subject. A referent condition is an open operator decision.',
},
o1_by_type: byType,
o1_recommended: {
@ -145,7 +171,11 @@ const handNote =
THRESHOLD === 7
? ' — all 6 hand-verified: 4 are wrong edits (unit crossing, metric crossing, two mutilated identifiers)'
: ' (hand-verification was done on the >=7 pilot only)';
console.log(`\n§4 as written would admit ${s4O1}${handNote}. Context condition removes ${s4O1 - o1.length}.\n`);
console.log(`\n§4 as written would admit ${s4O1} on the numeric path${handNote}. Context condition removes ${s4O1 - o1Numeric}.`);
console.log(
`§4b status class: ${report.status_synonym.class_total} flags -> ${report.status_synonym.proven} proven, ` +
`${JSON.stringify(report.status_synonym.aborts)} — REVIEW-grade, not applied by any driver.\n`,
);
console.log('abort codes:');
for (const [code, n] of Object.entries(byCode).sort((a, b) => b[1] - a[1])) {
console.log(` ${code.padEnd(20)} ${String(n).padStart(4)} ${pct(n)}`);