docs(ms-ai-architect): R11 — de tre operatørbeslutningene ratifisert og ført inn i kontrakten [skip-docs]
Operatøren fulgte alle tre anbefalingene (2026-08-03). Beslutningene styrer all nedstrøms fiksing av et offentlig distribuert korpus, så de føres i den SPOREDE kontrakten, ikke i en økt-transkripsjon. §5 — O2 RATIFISERT, med rest-sjekk (ikke blankt). Tre betingelser, alle menneske-bekreftet: (1) setningen påstår strengt mindre, (2) resten bærer ingen falsk eller misvisende stående implikasjon — lest som en LESER leser den, ikke som en logiker, (3) ingenting kilden BEKREFTER fjernes. Betingelse 2 og 3 krever at et menneske leser resten, så O2 er billigere enn O3 (ingen fakta-innhenting) men ikke mekanisk. §10s throughput-antakelse bør re-måles mot det. §4a — kontekst-korrespondanse som betingelse 4, og betingelse 5: den APPLISERBARE klassen er `iso_date` alene. Hånd-verifisert 9/9 mot 0/6 for number/version. Klassifikatoren fortsetter å rapportere alle typer — det er målingen — og merker den appliserbare mengden som `o1_recommended`. §4b — status-synonymtabellen RATIFISERT, snever og LUKKET. Fire rader. Tre skranker, fordi dette er det ENESTE stedet der verdien som skrives inn i fila ikke selv står ordrett i quoten: tabellen utvides aldri ved slutning i run time, fil-tokenet må være en komplett livssyklus-etikett, og verdien som skrives er den KORPUS-side ekvivalenten med filas egen markup bevart (`**Preview**` → `**GA**`), aldri den engelske frasen limt inn. §4s påstand om at invarianten er «deliberately stronger than human review at scale» er strøket og merket falsifisert, med peker til målingen. INGEN AV DE TRE ER IMPLEMENTERT. Klassifikatoren aborterer fortsatt STATUS_SYNONYM og ruter fortsatt alt ikke-O1 til O3. Begge dokumentene sier det eksplisitt, slik at en senere økt ikke antar at koden allerede følger kontrakten.
This commit is contained in:
parent
064078b8c2
commit
e86948a71a
2 changed files with 101 additions and 20 deletions
|
|
@ -93,18 +93,86 @@ routes the item to O3. This is the same discipline already proven in the
|
|||
header-backfill drivers (frozen manifest, hard per-file invariant, abort before
|
||||
write, idempotent re-run, `atomicWriteSync`) — see `scripts/kb-update/backfill-*.mjs`.
|
||||
|
||||
This invariant is deliberately stronger than human review at scale. "The new
|
||||
value occurs verbatim in the quote the judge cited" is a checkable property; an
|
||||
operator eyeballing edit number 600 is not. Human attention is therefore spent
|
||||
where it cannot be replaced (O3), not spread thin across all of them.
|
||||
~~This invariant is deliberately stronger than human review at scale.~~
|
||||
**FALSIFIED 2026-08-03 — see `docs/r11-pilot-results.md` §2.** Run exactly as
|
||||
written it admitted 6 swaps of which **4 were wrong edits**. Conditions 1–3
|
||||
constrain where the value *came from* and what the edit *looks like*, and nothing
|
||||
about whether the two tokens denote the same quantity. Conditions 1–3 are
|
||||
necessary; they are not sufficient.
|
||||
|
||||
## 5. The O2 policy (requires operator ratification before use)
|
||||
### 4a. Condition 4 — context correspondence (added 2026-08-03)
|
||||
|
||||
4. the token must sit under **the same label, or the same trailing unit, on both
|
||||
sides** (`contextCorresponds()` in `scripts/kb-eval/lib/fix-op.mjs`).
|
||||
|
||||
Deliberately lexical, with no translation table beyond §4b: a swap is therefore
|
||||
provable essentially only where the context is language-neutral — a URL, a code
|
||||
sample, a parameter key.
|
||||
|
||||
**Condition 5 — the applied class is `iso_date` only** (operator decision,
|
||||
2026-08-03). Condition 4 is still not sufficient: a matching identifier *prefix*
|
||||
(`AI-`, `gpt-`, `Agent `) satisfies it while the digit is part of a **name**
|
||||
rather than a quantity, which produced `AI-900` → `AI-901`, `gpt-4o` →
|
||||
`gpt-5.1o` (twice) and a Java-agent downgrade. Hand-verification over the whole
|
||||
population: **`iso_date` 9/9 correct, `number` and `version` 0/6**. A driver may
|
||||
apply `iso_date` proposals and **must never apply `number` or `version` ones**.
|
||||
The classifier keeps reporting all admitted types — that is the measurement — and
|
||||
marks the applicable set as `o1_recommended`.
|
||||
|
||||
### 4b. The ratified status-synonym table (operator decision, 2026-08-03)
|
||||
|
||||
`STATUS_SYNONYM` — 15 pilot flags, **54 corpus-wide** — is the class where the
|
||||
corpus writes `**Preview**` / `**GA**` while the source writes *"generally
|
||||
available"*. A narrow, **closed** equivalence table is ratified:
|
||||
|
||||
| Corpus-side label | Source-side phrasing (must appear verbatim in `evidence_quote`) |
|
||||
|---|---|
|
||||
| `GA` | `generally available`, `general availability` |
|
||||
| `Preview`, `Public Preview` | `public preview`, `preview` |
|
||||
| `Private Preview` | `private preview` |
|
||||
| `Deprecated`, `Utfaset` | `deprecated`, `retired` |
|
||||
|
||||
Three constraints, because this is the **one** place where the value written into
|
||||
the file does not itself appear verbatim in the quote:
|
||||
|
||||
- The table is **closed**. Any pair not listed aborts to O3; it is never extended
|
||||
by inference at run time.
|
||||
- The file-side token must be a **complete lifecycle label** (a whole table cell
|
||||
or emphasised token), never a substring of a longer sentence.
|
||||
- The written value is the **corpus-side** equivalent with the file's own markup
|
||||
preserved (`**Preview**` → `**GA**`), never the English phrase pasted in.
|
||||
|
||||
**Not yet implemented.** The classifier still aborts these as `STATUS_SYNONYM`;
|
||||
the ratification above is the contract a later session builds against.
|
||||
|
||||
## 5. The O2 policy — RATIFIED 2026-08-03, with a remainder check
|
||||
|
||||
O2 fixes by **subtraction**: the unsupported specificity is removed or
|
||||
generalised rather than replaced with a researched value.
|
||||
|
||||
- It **cannot introduce a new error**, because it asserts strictly less than the
|
||||
file did before.
|
||||
**Ratified by the operator on 2026-08-03, with one condition: the remainder
|
||||
check.** Subtraction is not admitted as a blanket rule, because the pilot found
|
||||
two ways it fails (`docs/r11-pilot-results.md` §5):
|
||||
|
||||
- It can leave a **misleading remainder**. Removing `er GA (juni 2025)` from a
|
||||
claim about a tool the source calls *deprecated* leaves that tool standing in a
|
||||
list of available ones. Strictly less asserted, still misleading.
|
||||
- It can **destroy true information**. Dropping `prebuilt-check` from a model list
|
||||
removes a model that exists — its ID is `prebuilt-check.us`, so the correct fix
|
||||
is a swap.
|
||||
|
||||
**An O2 proposal is valid only if all three hold, and a human confirms them:**
|
||||
|
||||
1. the edited sentence asserts **strictly less** than before;
|
||||
2. the **remainder carries no false or misleading standing implication** — read as
|
||||
a reader would read it, not as a logician would;
|
||||
3. nothing the source **confirms** is removed. Where the source supports a
|
||||
corrected value, the fix is O1 or O3, never subtraction.
|
||||
|
||||
Conditions 2 and 3 require a human to read the remainder. O2 is therefore
|
||||
**cheaper than O3 — no fact-finding — but not mechanical**, and the §10
|
||||
throughput assumption should be re-measured against that.
|
||||
|
||||
- It requires **no new fact-finding**, which is what makes it cheap.
|
||||
- It **reduces information density**. That is the real cost, and it is an
|
||||
operator decision, not an engineering one.
|
||||
|
|
@ -115,8 +183,11 @@ publicly distributed; an incorrect specific number is a worse failure than an
|
|||
honest general statement.
|
||||
|
||||
Ratifying O2 also resolves the standing `source_silent` question as **one class
|
||||
decision** instead of 190 individual ones. Until it is ratified, every O2
|
||||
candidate falls to O3.
|
||||
decision** instead of 190 individual ones. ~~Until it is ratified, every O2
|
||||
candidate falls to O3.~~ Ratified — O2 is in use, subject to the remainder check
|
||||
above. **Not yet implemented:** the classifier still routes every non-O1 item to
|
||||
O3, because O2 candidacy turns on the judge's prose `reason` and needs prose
|
||||
classification (`docs/r11-pilot-results.md` §4.2).
|
||||
|
||||
## 6. What stays human, permanently
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue