feat(ms-ai-architect): R11 §4b implementert — status-synonymtabellen målt, 8 forslag hånd-dømt [skip-docs]
Implementerer den ratifiserte §4b-tabellen i lib/fix-op.mjs (19 nye tester, suite 996/996). Alle tre skrankene har egne tester: tabellen er LUKKET, fil- tokenet må være en KOMPLETT livssyklus-etikett, og verdien som skrives er den korpus-side ekvivalenten med filas egen markup bevart. To implementasjonsvalg den ratifiserte teksten lot stå åpne, begge løst mot fail-closed: status-lokatoren er LINJE-scopet (livssyklus-vokabular gjentas nedover hver kolonne i en statustabell, så et blokkvindu er tvetydig ved konstruksjon), og et sitat som hevder to rader aborterer. MÅLT: 15 pilot / 54 korpus-brede flagg -> 5 og 8 provbare. Alle 8 hånd-dømt mot kilden (r11-pilot-results.md appendiks B): 5 korrekte, 1 ubevist, 2 GALE. De tre defektene er én familie: §4b binder tabellen, etikettens fullstendighet og verdien som skrives — og INGENTING om hvorvidt kilde-frasen refererer til radens eget subjekt. Samme proveniens-uten-referent-defekt som falsifiserte §4. Klassen er derfor REVIEW-grade, ikke apply-grade: `status` står bevisst utenfor o1_recommended, ingen driver applikerer den. To kandidatvilkår er kostnadsberegnet over de åtte (begge dreper gale forslag og null korrekte) men IKKE implementert — å utvide en tabell operatøren ratifiserte som lukket er en operatørbeslutning, slik vilkår 5 var i §4a. Rettet samtidig 2 NUL-bytes i testfila (pre-eksisterende, fra en tidligere økt) som gjorde at git behandlet hele fila som binær og blokkerte diff- gjennomgang før commit.
This commit is contained in:
parent
e86948a71a
commit
e4925c6b28
5 changed files with 330 additions and 10 deletions
|
|
@ -86,7 +86,21 @@ const r8 = items.filter((i) => i.rule === 'R8');
|
|||
const LOCATOR_CODES = new Set([ABORT_CODES.LOCATOR_MISS, ABORT_CODES.LOCATOR_AMBIGUOUS]);
|
||||
const locatorAborts = o3.filter((i) => LOCATOR_CODES.has(i.code)).length;
|
||||
|
||||
const s4O1 = s4Only.filter((v) => v.op === 'O1').length;
|
||||
// §4-as-written is a NUMERIC-path measurement: it is the baseline the context
|
||||
// condition (§4a) was added against. §4b status swaps do not run through the
|
||||
// context condition at all, so counting them here would silently inflate the
|
||||
// baseline and break the comparison with the pilot's hand-verified 6.
|
||||
const isStatus = (v) => v.op === 'O1' && v.proposal.type === 'status';
|
||||
const s4O1 = s4Only.filter((v) => v.op === 'O1' && !isStatus(v)).length;
|
||||
const o1Numeric = o1.filter((i) => i.proposal.type !== 'status').length;
|
||||
|
||||
// §4b (ratified 2026-08-03): the STATUS_SYNONYM class, split into what the closed
|
||||
// synonym table proves and why the remainder still aborts. The abort REASON
|
||||
// sub-distribution is the actionable part — NO_COMPLETE_FILE_LABEL is a corpus
|
||||
// shape, SOURCE_STATUS_AMBIGUOUS is a quote shape, FILE_ALREADY_MATCHES means the
|
||||
// flag was never a status mismatch in the first place.
|
||||
const statusProposals = items.filter((i) => i.op === 'O1' && i.proposal.type === 'status');
|
||||
const statusAborts = o3.filter((i) => i.code === ABORT_CODES.STATUS_SYNONYM);
|
||||
|
||||
// O1 precision is NOT uniform across token types, and this split is the pilot's
|
||||
// operational conclusion. Hand-verified over the whole not_grounded population:
|
||||
|
|
@ -117,7 +131,19 @@ const report = {
|
|||
s4_as_written: {
|
||||
O1: s4O1,
|
||||
note:
|
||||
'What §4 exactly as written would admit. NOT a source of proposals: on the >=7 pilot all 6 were hand-verified and 4 were wrong edits (unit crossing, metric crossing, two mutilated identifiers) — measured precision 2/6. Runs at other thresholds carry no hand-verification.',
|
||||
'What §4 exactly as written would admit on the NUMERIC path (§4b status swaps excluded — they never run through the context condition). NOT a source of proposals: on the >=7 pilot all 6 were hand-verified and 4 were wrong edits (unit crossing, metric crossing, two mutilated identifiers) — measured precision 2/6. Runs at other thresholds carry no hand-verification.',
|
||||
},
|
||||
status_synonym: {
|
||||
contract: '§4b — the closed synonym table, ratified 2026-08-03',
|
||||
class_total: statusProposals.length + statusAborts.length,
|
||||
proven: statusProposals.length,
|
||||
aborts: tally(statusAborts, (i) => (i.detail && i.detail.reason) || '(unspecified)'),
|
||||
hand_verified:
|
||||
THRESHOLD === 7
|
||||
? 'All 5 hand-judged 2026-08-03 (docs/r11-pilot-results.md appendix B). Four carry the source phrasing on the row\'s OWN subject and are correct. One (security-copilot-integration.md:94) harvests a "(Preview)" marker that belongs to a DIFFERENT agent in an enumerated quote — the same provenance-without-referent defect that falsified §4. Its outcome is plausibly right; its proof is not.'
|
||||
: 'hand-verification was done on the >=7 pilot only',
|
||||
applicability:
|
||||
'REVIEW-GRADE, NOT APPLY-GRADE. status is deliberately absent from o1_recommended: §4b binds the table, the completeness of the file label and the written value, and nothing about whether the source phrasing refers to the row\'s subject. A referent condition is an open operator decision.',
|
||||
},
|
||||
o1_by_type: byType,
|
||||
o1_recommended: {
|
||||
|
|
@ -145,7 +171,11 @@ const handNote =
|
|||
THRESHOLD === 7
|
||||
? ' — all 6 hand-verified: 4 are wrong edits (unit crossing, metric crossing, two mutilated identifiers)'
|
||||
: ' (hand-verification was done on the >=7 pilot only)';
|
||||
console.log(`\n§4 as written would admit ${s4O1}${handNote}. Context condition removes ${s4O1 - o1.length}.\n`);
|
||||
console.log(`\n§4 as written would admit ${s4O1} on the numeric path${handNote}. Context condition removes ${s4O1 - o1Numeric}.`);
|
||||
console.log(
|
||||
`§4b status class: ${report.status_synonym.class_total} flags -> ${report.status_synonym.proven} proven, ` +
|
||||
`${JSON.stringify(report.status_synonym.aborts)} — REVIEW-grade, not applied by any driver.\n`,
|
||||
);
|
||||
console.log('abort codes:');
|
||||
for (const [code, n] of Object.entries(byCode).sort((a, b) => b[1] - a[1])) {
|
||||
console.log(` ${code.padEnd(20)} ${String(n).padStart(4)} ${pct(n)}`);
|
||||
|
|
|
|||
|
|
@ -55,12 +55,195 @@ const TOKEN_PATTERNS = [
|
|||
|
||||
// Lifecycle vocabulary. Present in a claim without any numeric token, this is the
|
||||
// GA/Preview class: the corpus writes `**Preview**` / `**GA**` while the cited
|
||||
// source writes "generally available". A swap would satisfy §4 literally while
|
||||
// pasting English prose into a Norwegian table, so the class aborts and is put to
|
||||
// the operator as a design question (a ratified synonym table, or permanent O3).
|
||||
// source writes "generally available". A swap satisfies §4 literally while
|
||||
// pasting English prose into a Norwegian table, which is why the class was put to
|
||||
// the operator as a design question — answered by the §4b table below.
|
||||
const STATUS_RE =
|
||||
/\b(?:GA|generally available|allment tilgjengelig|public preview|private preview|preview|deprecated|utfaset|retired|avviklet)\b/i;
|
||||
|
||||
// ------------------------------------------------------- §4b synonym table
|
||||
//
|
||||
// RATIFIED 2026-08-03 (docs/r11-tiered-fix-design.md §4b). This is the one place
|
||||
// where the value written into the file does NOT appear verbatim in the quote —
|
||||
// §4 condition 1 is structurally unsatisfiable for it, because the corpus writes
|
||||
// a label and the source writes a phrase. The table is what the operator ratified
|
||||
// in its place, and it is CLOSED: a pair not listed here aborts, and nothing
|
||||
// extends it at run time.
|
||||
//
|
||||
// `corpus[0]` is the canonical value written back. The order is the table's own:
|
||||
// the row's least specific label wins, so a source that says only "preview" can
|
||||
// never produce the more specific "Public Preview" — that would assert something
|
||||
// the source does not.
|
||||
export const STATUS_TABLE = [
|
||||
{ row: 'GA', corpus: ['GA'], source: ['generally available', 'general availability'] },
|
||||
{ row: 'PREVIEW', corpus: ['Preview', 'Public Preview'], source: ['public preview', 'preview'] },
|
||||
{ row: 'PRIVATE_PREVIEW', corpus: ['Private Preview'], source: ['private preview'] },
|
||||
{ row: 'DEPRECATED', corpus: ['Deprecated', 'Utfaset'], source: ['deprecated', 'retired'] },
|
||||
];
|
||||
|
||||
// Longest first, so "private preview" is consumed as its own row before the
|
||||
// "preview" row can claim the tail of it.
|
||||
const SOURCE_PHRASES = STATUS_TABLE.flatMap((r) => r.source.map((p) => ({ row: r.row, phrase: p }))).sort(
|
||||
(a, b) => b.phrase.length - a.phrase.length,
|
||||
);
|
||||
|
||||
const CORPUS_LABELS = new Map(
|
||||
STATUS_TABLE.flatMap((r) => r.corpus.map((label) => [label.toLowerCase(), r])),
|
||||
);
|
||||
|
||||
// A hyphen counts as a word character HERE, unlike WORD elsewhere in this module:
|
||||
// `2025-11-15-preview` is an api-version identifier, not a statement that the
|
||||
// feature is in preview. Treating `-` as a boundary would harvest lifecycle rows
|
||||
// out of URLs and code samples.
|
||||
const isPhraseChar = (ch) => ch !== undefined && /[A-Za-z0-9-]/.test(ch);
|
||||
|
||||
/**
|
||||
* Which lifecycle rows does the cited quote assert, per the closed §4b table?
|
||||
* Matching is case-insensitive, non-overlapping, longest phrase first.
|
||||
* @returns {string[]} distinct row keys, in table order
|
||||
*/
|
||||
export function sourceStatusRows(quote) {
|
||||
const text = (quote || '').toLowerCase();
|
||||
const taken = [];
|
||||
const rows = new Set();
|
||||
for (const { row, phrase } of SOURCE_PHRASES) {
|
||||
let from = 0;
|
||||
for (;;) {
|
||||
const at = text.indexOf(phrase, from);
|
||||
if (at === -1) break;
|
||||
const end = at + phrase.length;
|
||||
from = end;
|
||||
if (taken.some(([s, e]) => at < e && end > s)) continue;
|
||||
if (isPhraseChar(text[at - 1]) || isPhraseChar(text[end])) continue;
|
||||
taken.push([at, end]);
|
||||
rows.add(row);
|
||||
}
|
||||
}
|
||||
return STATUS_TABLE.map((r) => r.row).filter((r) => rows.has(r));
|
||||
}
|
||||
|
||||
// Markup wrappers the corpus actually uses around a lifecycle label. The label is
|
||||
// replaced INSIDE the wrapper, which is how §4b constraint 3 ("the file's own
|
||||
// markup preserved") is satisfied without any markup handling at write time.
|
||||
const WRAPPERS = ['**', '__', '*', '_', '`'];
|
||||
|
||||
/**
|
||||
* Is this segment exactly a lifecycle label, once whitespace and one or more
|
||||
* markup wrappers are peeled off? Returns the label's span in the ORIGINAL line.
|
||||
*/
|
||||
function labelInSegment(segment, segStart) {
|
||||
let text = segment;
|
||||
let off = 0;
|
||||
const trim = () => {
|
||||
const lead = text.length - text.trimStart().length;
|
||||
off += lead;
|
||||
text = text.trim();
|
||||
};
|
||||
trim();
|
||||
for (;;) {
|
||||
const w = WRAPPERS.find((x) => text.length > 2 * x.length && text.startsWith(x) && text.endsWith(x));
|
||||
if (!w) break;
|
||||
off += w.length;
|
||||
text = text.slice(w.length, text.length - w.length);
|
||||
trim();
|
||||
}
|
||||
const rec = CORPUS_LABELS.get(text.toLowerCase());
|
||||
return rec ? { row: rec.row, label: text, index: segStart + off, length: text.length } : null;
|
||||
}
|
||||
|
||||
// An emphasised run anywhere on the line. `|` is excluded from the inner text so a
|
||||
// run can never span two table cells.
|
||||
const EMPHASIS_RE = /(\*\*|__|\*|_|`)([^*_`|]+?)\1/g;
|
||||
|
||||
/**
|
||||
* The single complete lifecycle label on `line`, or null when there is none or
|
||||
* more than one (§4b constraint 2: a whole table cell or an emphasised token,
|
||||
* never a substring of a longer sentence).
|
||||
*
|
||||
* Deliberately LINE-scoped rather than block-scoped, unlike the numeric locator.
|
||||
* Lifecycle vocabulary repeats down every column of a status table, so a block
|
||||
* window is ambiguous by construction — and all 15 pilot flags in this class
|
||||
* point at the row that carries the claim, not at the table header.
|
||||
*
|
||||
* @returns {{row: string, label: string, index: number, length: number}|null}
|
||||
*/
|
||||
export function fileStatusLabel(line) {
|
||||
const text = line || '';
|
||||
const hits = [];
|
||||
if (text.includes('|')) {
|
||||
let at = 0;
|
||||
for (const cell of text.split('|')) {
|
||||
const hit = labelInSegment(cell, at);
|
||||
if (hit) hits.push(hit);
|
||||
at += cell.length + 1;
|
||||
}
|
||||
}
|
||||
EMPHASIS_RE.lastIndex = 0;
|
||||
let m;
|
||||
while ((m = EMPHASIS_RE.exec(text)) !== null) {
|
||||
const hit = labelInSegment(m[0], m.index);
|
||||
if (hit && !hits.some((h) => h.index === hit.index)) hits.push(hit);
|
||||
}
|
||||
return hits.length === 1 ? hits[0] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* §4b: classify a status-vocabulary flag against the ratified table.
|
||||
*
|
||||
* Every abort keeps the STATUS_SYNONYM code and names its cause in `detail.reason`
|
||||
* — the top-level abort taxonomy is the pilot's measurement #3 and stays
|
||||
* comparable across the implementation, with the reasons reported as a
|
||||
* sub-distribution.
|
||||
*/
|
||||
function classifyStatusSynonym(flag, lines) {
|
||||
const detail = (reason, extra = {}) => abort(ABORT_CODES.STATUS_SYNONYM, { detail: { reason, ...extra } });
|
||||
|
||||
if (flag.line < 1 || flag.line > lines.length) return detail('LINE_OUT_OF_RANGE', { line: flag.line });
|
||||
|
||||
const rows = sourceStatusRows(flag.evidence_quote || '');
|
||||
if (rows.length === 0) return detail('NO_SOURCE_STATUS');
|
||||
if (rows.length > 1) return detail('SOURCE_STATUS_AMBIGUOUS', { rows });
|
||||
|
||||
const before = lines[flag.line - 1];
|
||||
const hit = fileStatusLabel(before);
|
||||
if (!hit) return detail('NO_COMPLETE_FILE_LABEL');
|
||||
if (hit.row === rows[0]) return detail('FILE_ALREADY_MATCHES', { row: hit.row });
|
||||
|
||||
const target = STATUS_TABLE.find((r) => r.row === rows[0]);
|
||||
const replacement = target.corpus[0];
|
||||
const after = before.slice(0, hit.index) + replacement + before.slice(hit.index + hit.length);
|
||||
|
||||
// §4 conditions 2 and 3 still bind. Condition 1 is replaced by the table: what
|
||||
// must appear verbatim in the quote is the SOURCE phrasing, not the written value.
|
||||
const rest =
|
||||
before.slice(0, hit.index) === after.slice(0, hit.index) &&
|
||||
before.slice(hit.index + hit.length) === after.slice(hit.index + replacement.length);
|
||||
const rebuilt = lines.slice();
|
||||
rebuilt[flag.line - 1] = after;
|
||||
const changedLines = rebuilt.reduce((n, l, i) => n + (l === lines[i] ? 0 : 1), 0);
|
||||
if (!rest || changedLines !== 1) {
|
||||
return abort(ABORT_CODES.INVARIANT_FAIL, { detail: { restIdentical: rest, changedLines } });
|
||||
}
|
||||
|
||||
return {
|
||||
op: 'O1',
|
||||
code: PROVEN,
|
||||
proposal: {
|
||||
file: flag.file,
|
||||
line: flag.line,
|
||||
token: hit.label,
|
||||
replacement,
|
||||
type: 'status',
|
||||
status_row_from: hit.row,
|
||||
status_row_to: rows[0],
|
||||
before,
|
||||
after,
|
||||
evidence_url: flag.evidence_url,
|
||||
evidence_quote: flag.evidence_quote,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract swappable value tokens, non-overlapping and priority ordered.
|
||||
* Status words are NOT value tokens — see STATUS_RE.
|
||||
|
|
@ -206,7 +389,8 @@ export function classifyFlag(flag, fileText, opts = {}) {
|
|||
|
||||
const tokens = distinct(extractValueTokens(claim));
|
||||
if (tokens.length === 0) {
|
||||
return abort(hasStatusWord(claim) ? ABORT_CODES.STATUS_SYNONYM : ABORT_CODES.NO_VALUE_TOKEN);
|
||||
if (!hasStatusWord(claim)) return abort(ABORT_CODES.NO_VALUE_TOKEN);
|
||||
return classifyStatusSynonym(flag, lines); // §4b — the ratified table, not a swap of prose
|
||||
}
|
||||
if (tokens.length > 1) {
|
||||
return abort(ABORT_CODES.MULTI_VALUE_TOKEN, { detail: { candidates: tokens.map((t) => t.value) } });
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue