docs(ms-ai-architect): R11 — de tre operatørbeslutningene ratifisert og ført inn i kontrakten [skip-docs]
Operatøren fulgte alle tre anbefalingene (2026-08-03). Beslutningene styrer all nedstrøms fiksing av et offentlig distribuert korpus, så de føres i den SPOREDE kontrakten, ikke i en økt-transkripsjon. §5 — O2 RATIFISERT, med rest-sjekk (ikke blankt). Tre betingelser, alle menneske-bekreftet: (1) setningen påstår strengt mindre, (2) resten bærer ingen falsk eller misvisende stående implikasjon — lest som en LESER leser den, ikke som en logiker, (3) ingenting kilden BEKREFTER fjernes. Betingelse 2 og 3 krever at et menneske leser resten, så O2 er billigere enn O3 (ingen fakta-innhenting) men ikke mekanisk. §10s throughput-antakelse bør re-måles mot det. §4a — kontekst-korrespondanse som betingelse 4, og betingelse 5: den APPLISERBARE klassen er `iso_date` alene. Hånd-verifisert 9/9 mot 0/6 for number/version. Klassifikatoren fortsetter å rapportere alle typer — det er målingen — og merker den appliserbare mengden som `o1_recommended`. §4b — status-synonymtabellen RATIFISERT, snever og LUKKET. Fire rader. Tre skranker, fordi dette er det ENESTE stedet der verdien som skrives inn i fila ikke selv står ordrett i quoten: tabellen utvides aldri ved slutning i run time, fil-tokenet må være en komplett livssyklus-etikett, og verdien som skrives er den KORPUS-side ekvivalenten med filas egen markup bevart (`**Preview**` → `**GA**`), aldri den engelske frasen limt inn. §4s påstand om at invarianten er «deliberately stronger than human review at scale» er strøket og merket falsifisert, med peker til målingen. INGEN AV DE TRE ER IMPLEMENTERT. Klassifikatoren aborterer fortsatt STATUS_SYNONYM og ruter fortsatt alt ikke-O1 til O3. Begge dokumentene sier det eksplisitt, slik at en senere økt ikke antar at koden allerede følger kontrakten.
This commit is contained in:
parent
064078b8c2
commit
e86948a71a
2 changed files with 101 additions and 20 deletions
|
|
@ -145,17 +145,27 @@ run with it. For table claims, `line` points at the **header**, not the value.
|
|||
This is why the locator exists at all, and why it searches the enclosing block
|
||||
rather than the line.
|
||||
|
||||
## 6. Open operator decisions
|
||||
## 6. Operator decisions — ALL THREE RATIFIED 2026-08-03
|
||||
|
||||
1. **Ratify O2 (§5)?** Until then O2 does not exist and everything is O3. F1/F2
|
||||
above argue for ratifying it **with a remainder check** rather than as a blanket
|
||||
rule.
|
||||
2. **Amend §4 with a ratified synonym table?** 15 pilot flags (54 corpus-wide) are
|
||||
`STATUS_SYNONYM`: the corpus writes `**Preview**` / `**GA**`, the source writes
|
||||
*"generally available"*. Either a small ratified equivalence table admits them,
|
||||
or they are permanently O3.
|
||||
3. **Is O1 worth building at all?** Nine edits corpus-wide, all `api-version`
|
||||
bumps. The alternative is to drop the O1 driver and treat those nine as O3.
|
||||
All three were put to the operator with the recommendations below and **all three
|
||||
were accepted as recommended**. The contract text now lives in
|
||||
`docs/r11-tiered-fix-design.md` §4a/§4b/§5; this section records what was asked
|
||||
and what the answer was.
|
||||
|
||||
**None of the three is implemented yet.** The classifier still aborts
|
||||
`STATUS_SYNONYM` and still routes every non-O1 item to O3. A later session builds
|
||||
against the ratified contract — it must not assume the code already honours it.
|
||||
|
||||
1. **Ratify O2 (§5)?** → **RATIFIED, with the remainder check** (not as a blanket
|
||||
rule), exactly as F1/F2 above argued. Contract: design doc §5, three
|
||||
conditions, human-confirmed.
|
||||
2. **Amend §4 with a ratified synonym table?** → **RATIFIED, narrow and closed.**
|
||||
Contract: design doc §4b — four label rows, closed table, complete-label-only,
|
||||
corpus-side value with the file's own markup preserved. Unlocks up to 54
|
||||
corpus-wide flags.
|
||||
3. **Is O1 worth building at all?** → **KEPT, locked to `iso_date`.** Contract:
|
||||
design doc §4a condition 5 — a driver may apply `iso_date` proposals and must
|
||||
never apply `number` or `version` ones. Nine edits corpus-wide.
|
||||
|
||||
## 7. What this does not change
|
||||
|
||||
|
|
|
|||
|
|
@ -93,18 +93,86 @@ routes the item to O3. This is the same discipline already proven in the
|
|||
header-backfill drivers (frozen manifest, hard per-file invariant, abort before
|
||||
write, idempotent re-run, `atomicWriteSync`) — see `scripts/kb-update/backfill-*.mjs`.
|
||||
|
||||
This invariant is deliberately stronger than human review at scale. "The new
|
||||
value occurs verbatim in the quote the judge cited" is a checkable property; an
|
||||
operator eyeballing edit number 600 is not. Human attention is therefore spent
|
||||
where it cannot be replaced (O3), not spread thin across all of them.
|
||||
~~This invariant is deliberately stronger than human review at scale.~~
|
||||
**FALSIFIED 2026-08-03 — see `docs/r11-pilot-results.md` §2.** Run exactly as
|
||||
written it admitted 6 swaps of which **4 were wrong edits**. Conditions 1–3
|
||||
constrain where the value *came from* and what the edit *looks like*, and nothing
|
||||
about whether the two tokens denote the same quantity. Conditions 1–3 are
|
||||
necessary; they are not sufficient.
|
||||
|
||||
## 5. The O2 policy (requires operator ratification before use)
|
||||
### 4a. Condition 4 — context correspondence (added 2026-08-03)
|
||||
|
||||
4. the token must sit under **the same label, or the same trailing unit, on both
|
||||
sides** (`contextCorresponds()` in `scripts/kb-eval/lib/fix-op.mjs`).
|
||||
|
||||
Deliberately lexical, with no translation table beyond §4b: a swap is therefore
|
||||
provable essentially only where the context is language-neutral — a URL, a code
|
||||
sample, a parameter key.
|
||||
|
||||
**Condition 5 — the applied class is `iso_date` only** (operator decision,
|
||||
2026-08-03). Condition 4 is still not sufficient: a matching identifier *prefix*
|
||||
(`AI-`, `gpt-`, `Agent `) satisfies it while the digit is part of a **name**
|
||||
rather than a quantity, which produced `AI-900` → `AI-901`, `gpt-4o` →
|
||||
`gpt-5.1o` (twice) and a Java-agent downgrade. Hand-verification over the whole
|
||||
population: **`iso_date` 9/9 correct, `number` and `version` 0/6**. A driver may
|
||||
apply `iso_date` proposals and **must never apply `number` or `version` ones**.
|
||||
The classifier keeps reporting all admitted types — that is the measurement — and
|
||||
marks the applicable set as `o1_recommended`.
|
||||
|
||||
### 4b. The ratified status-synonym table (operator decision, 2026-08-03)
|
||||
|
||||
`STATUS_SYNONYM` — 15 pilot flags, **54 corpus-wide** — is the class where the
|
||||
corpus writes `**Preview**` / `**GA**` while the source writes *"generally
|
||||
available"*. A narrow, **closed** equivalence table is ratified:
|
||||
|
||||
| Corpus-side label | Source-side phrasing (must appear verbatim in `evidence_quote`) |
|
||||
|---|---|
|
||||
| `GA` | `generally available`, `general availability` |
|
||||
| `Preview`, `Public Preview` | `public preview`, `preview` |
|
||||
| `Private Preview` | `private preview` |
|
||||
| `Deprecated`, `Utfaset` | `deprecated`, `retired` |
|
||||
|
||||
Three constraints, because this is the **one** place where the value written into
|
||||
the file does not itself appear verbatim in the quote:
|
||||
|
||||
- The table is **closed**. Any pair not listed aborts to O3; it is never extended
|
||||
by inference at run time.
|
||||
- The file-side token must be a **complete lifecycle label** (a whole table cell
|
||||
or emphasised token), never a substring of a longer sentence.
|
||||
- The written value is the **corpus-side** equivalent with the file's own markup
|
||||
preserved (`**Preview**` → `**GA**`), never the English phrase pasted in.
|
||||
|
||||
**Not yet implemented.** The classifier still aborts these as `STATUS_SYNONYM`;
|
||||
the ratification above is the contract a later session builds against.
|
||||
|
||||
## 5. The O2 policy — RATIFIED 2026-08-03, with a remainder check
|
||||
|
||||
O2 fixes by **subtraction**: the unsupported specificity is removed or
|
||||
generalised rather than replaced with a researched value.
|
||||
|
||||
- It **cannot introduce a new error**, because it asserts strictly less than the
|
||||
file did before.
|
||||
**Ratified by the operator on 2026-08-03, with one condition: the remainder
|
||||
check.** Subtraction is not admitted as a blanket rule, because the pilot found
|
||||
two ways it fails (`docs/r11-pilot-results.md` §5):
|
||||
|
||||
- It can leave a **misleading remainder**. Removing `er GA (juni 2025)` from a
|
||||
claim about a tool the source calls *deprecated* leaves that tool standing in a
|
||||
list of available ones. Strictly less asserted, still misleading.
|
||||
- It can **destroy true information**. Dropping `prebuilt-check` from a model list
|
||||
removes a model that exists — its ID is `prebuilt-check.us`, so the correct fix
|
||||
is a swap.
|
||||
|
||||
**An O2 proposal is valid only if all three hold, and a human confirms them:**
|
||||
|
||||
1. the edited sentence asserts **strictly less** than before;
|
||||
2. the **remainder carries no false or misleading standing implication** — read as
|
||||
a reader would read it, not as a logician would;
|
||||
3. nothing the source **confirms** is removed. Where the source supports a
|
||||
corrected value, the fix is O1 or O3, never subtraction.
|
||||
|
||||
Conditions 2 and 3 require a human to read the remainder. O2 is therefore
|
||||
**cheaper than O3 — no fact-finding — but not mechanical**, and the §10
|
||||
throughput assumption should be re-measured against that.
|
||||
|
||||
- It requires **no new fact-finding**, which is what makes it cheap.
|
||||
- It **reduces information density**. That is the real cost, and it is an
|
||||
operator decision, not an engineering one.
|
||||
|
|
@ -115,8 +183,11 @@ publicly distributed; an incorrect specific number is a worse failure than an
|
|||
honest general statement.
|
||||
|
||||
Ratifying O2 also resolves the standing `source_silent` question as **one class
|
||||
decision** instead of 190 individual ones. Until it is ratified, every O2
|
||||
candidate falls to O3.
|
||||
decision** instead of 190 individual ones. ~~Until it is ratified, every O2
|
||||
candidate falls to O3.~~ Ratified — O2 is in use, subject to the remainder check
|
||||
above. **Not yet implemented:** the classifier still routes every non-O1 item to
|
||||
O3, because O2 candidacy turns on the judge's prose `reason` and needs prose
|
||||
classification (`docs/r11-pilot-results.md` §4.2).
|
||||
|
||||
## 6. What stays human, permanently
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue