docs(ms-ai-architect): R11 §9.4 — ratifikasjonspakke: fire editer forberedt + maskin-attestert, ingen anvendt; G7 reist [skip-docs]
This commit is contained in:
parent
43f0a5e4e9
commit
4042d0b94a
2 changed files with 99 additions and 0 deletions
|
|
@ -560,6 +560,103 @@ never asked the classifier to act on it. That is a prompt gap, not a model
|
|||
failure, and it is cheap to close in a later wave: name internal consistency as a
|
||||
cond-2 dimension and require the citation.
|
||||
|
||||
### 9.4 The ratification packet — four edits prepared, none applied
|
||||
|
||||
Prepared 2026-08-03. **No KB file was edited.** This section exists so the
|
||||
ratifier decides from attested strings rather than from prose.
|
||||
|
||||
**First, an ambiguity in §9.3 that had to be resolved before anything could be
|
||||
written.** The three reduction bullets above use the form *"drop X, keep Y"*.
|
||||
That reads two ways — drop X *from the subtraction*, or drop X *from the file* —
|
||||
and the two readings produce **opposite edits**. The prose does not settle it.
|
||||
What settles it is the contradiction each reduction exists to avoid: the member
|
||||
the file continues to assert elsewhere must **survive in the file**, so it is the
|
||||
*other* member that stays in the subtraction. Applied to each, and verified
|
||||
against the live file this session:
|
||||
|
||||
| idx | the file asserts elsewhere | therefore survives | subtraction reduces to |
|
||||
|---|---|---|---|
|
||||
| 14 | 566 `Reviewed documents automatisk tilgjengelige …` | `Automatically` | `/ SharePoint` only |
|
||||
| 26 | 300 `Responsible AI Scorecard: Error analysis, fairness assessment` | item 4, Error analysis | item 5, Counterfactual, only |
|
||||
| 27 | 216 `**Copilot Studio**: "Powered by AI" disclosure i chat interface` | the Chat-interface row | the Plugin-actions row only |
|
||||
|
||||
§9.3's own parenthetical corroborates this independently: it records the surviving
|
||||
numbering for idx 26 as `1,2,3,4,6,7`, which is what deleting item 5 produces.
|
||||
The reading also matches STATE's summary. The `drop/keep` phrasing above should be
|
||||
read in the subtraction sense throughout.
|
||||
|
||||
**The four prepared edits.** Each reduced remainder was derived by string surgery
|
||||
on the original `file_text_verbatim` — never transcribed by hand — and re-run
|
||||
through `checkRow` (V1/V2/V2b/V3):
|
||||
|
||||
| candidate | file:real_line | machine | cond 1 | cond 2 (whole file) | cond 3 |
|
||||
|---|---|---|---|---|---|
|
||||
| **17** as attested | `rag-caching-optimization.md:254` | clean | yes | yes (§9.2) | yes |
|
||||
| **14** reduced | `feedback-loops-continuous-improvement.md:555` | clean | yes | **yes (verified here)** | yes |
|
||||
| **26** reduced | `transparency-documentation-standards.md:117` | clean | yes | no contradiction, **but see below** | yes |
|
||||
| **27** reduced | `transparency-documentation-standards.md:426` | clean | yes | **yes (verified here)** | `human_must_confirm` |
|
||||
|
||||
The cond-2 evidence gathered this session, since a reduced remainder is a new
|
||||
remainder and inherits no clearance from the one V2b originally attested:
|
||||
|
||||
- **idx 14** — `sharepoint` occurs in the file **only** inside the verbatim block
|
||||
(line 555). Nothing else asserts SharePoint as feedback storage; line 563's
|
||||
"AI Builder feedback loop storage" is generic and consistent with Dataverse.
|
||||
The classifier's cond-2 doubt was line 566's automaticity — and the reduction
|
||||
deletes the automaticity change, so that defeater no longer applies to this
|
||||
subtraction at all.
|
||||
- **idx 26** — counterfactuals appear at 324, 475 and 693. Line 324 is a row in
|
||||
the table *Azure Machine Learning — Built-in transparency tools*, attaching
|
||||
`counterfactual what-if` to **Model interpretability**, not to the Scorecard;
|
||||
475 and 693 are GDPR right-to-explanation and Azure ML explanations. None
|
||||
asserts Counterfactual analysis as a scorecard segment, so the reduction
|
||||
contradicts nothing.
|
||||
- **idx 27** — `grep -niE "confirmation|plugin|sensitive action"` and the
|
||||
Norwegian forms (`bekreft|godkjenn|samtykke`) return line 428 alone, i.e. only
|
||||
the row being deleted. Nothing else in the file carries the claim.
|
||||
|
||||
**Nothing here promotes itself, and that is deliberate.** Two of the four still
|
||||
carry an unresolved human condition, and the resolution is the ratifier's:
|
||||
|
||||
- **idx 26** — the classifier's cond-2 `human_must_confirm` was never about a
|
||||
contradiction. It was the **renumbering artifact**: delete-only cannot renumber,
|
||||
so the list reads `1,2,3,4,6,7`. The whole-file check does not touch that, and
|
||||
the artifact survives the reduction. Accepting it is a judgement about the file.
|
||||
- **idx 27** — cond 3 stands at `human_must_confirm` and the sweep settles only
|
||||
cond 2.
|
||||
|
||||
**The verified score therefore remains 1 of 46 (idx 17)** until a ratifier acts.
|
||||
Moving idx 14 to affirmative is defensible on the record — its only stated doubt
|
||||
is deleted along with the automaticity change — but that is a ratification, and
|
||||
§9.3's asymmetry rule applies: this pass may confirm doubt, never clear it on its
|
||||
own authority.
|
||||
|
||||
**Idx 17: recommended as attested, both residues left standing.** The amended
|
||||
variant with the trailing colon dropped was also run through `checkRow` and is
|
||||
**also machine-clean**, so the choice is free on machine grounds — which means it
|
||||
must be made on other grounds. Two argue for leaving it: the colon is not false,
|
||||
and the attested string is the one the measurement was taken on. The `**Verified**`
|
||||
stamp at line 258 is *outside* the verbatim block; editing it would be a second
|
||||
locator, excluded by the same single-locator rule that put idx 36's companion edit
|
||||
at 310 out of envelope. Both residues should be recorded as consciously left, not
|
||||
overlooked.
|
||||
|
||||
**One coupling checked before any write, because it fails silently.** Applying
|
||||
idx 17 makes its `file_text_verbatim` no longer occur in the file, so **V1 fails
|
||||
for that row permanently** and "46/46 pass V1" stops being true. The test suite is
|
||||
unaffected — `tests/kb-eval/test-o2-return-check.test.mjs` drives `checkRow` with
|
||||
a synthetic `readFile` stub (`skills/x/references/y.md`) and never reads the live
|
||||
corpus. But `check-o2-returns.mjs`, the CLI, *does* read live, and will report the
|
||||
failure on every future run. The returns directory is evidence of a pre-edit
|
||||
state and must be read as such; the CLI's V1 tally is only meaningful against an
|
||||
unedited corpus. Recorded rather than worked around.
|
||||
|
||||
**A second-order note for whoever applies these.** Idx 26 and 27 are in the same
|
||||
file. Applying either shifts the line numbers the other cites (216, 300, 324),
|
||||
so re-derive references after the first write and anchor on
|
||||
`file_text_verbatim` — never on the line number, which already differs from
|
||||
`real_line` in 9 of 17 records.
|
||||
|
||||
## Appendix A — the 15 admitted proposals, hand-verified
|
||||
|
||||
Every proposal the classifier (§4 + context condition) admitted over the whole
|
||||
|
|
|
|||
|
|
@ -106,6 +106,8 @@ Status-nøkkel: 🔴 ikke startet · 🟡 pågår · 🟢 lukket.
|
|||
|
||||
| **G6** | Ingen sikkerhetsgate på ingestion-kjeden (hentet eksternt innhold → korpus): llm-security-pluginen er **deaktivert globalt** (verifisert 2026-07-03 i `~/.claude/settings.json`), så `post-mcp-verify`-hooken (injection-skann på all tool-output, inkl. `microsoft_docs_fetch`) fyrer ikke; commit-gaten dekker kun secrets (gitleaks), ikke injeksjon/steganografi i `.md`-innhold. Tillit til MS Learn dekker faktisk korrekthet — ikke adversarielt innhold i kanalen eller i kodeeksempler/lokalisert stoff | Indirekte prompt-injeksjon/steganografi persistert i offentlig distribuert KB: references-filene blir instruksjonsnær kontekst i fremtidige agent-sesjoner, én forgiftet fil re-serveres til alle brukere (hele Norge) | **R6-briefen designer gaten, to lag** (`docs/ingestion-security-brief-2026-07.md`, committet 2026-07-04): (a) llm-security AKTIV + verifisert fyrende i enhver fetch-økt (kb-update, research, generate-skills, judge-pass); headless-caveat GH #36071 → foreground eller kompenserende skann; (b) deterministisk node-skann (unicode/decode/injection — de DELTE llm-security-detektorene importert in-process, ikke kopiert; operatør-valg 2026-07-04) over endrede `skills/**/*.md` før commit. Håndheves fra R7 og i kb-update-kadensen | 🟢 **Layer B (b) LUKKET 2026-07-04 (TDD).** Bærende gaten bygget: `scan-adversarial-content.mjs` (+ `lib/adversarial-scan.mjs` disposition-kjerne, `lib/adversarial-detect.mjs` llm-security-bro), wiret som sibling til `validate-kb-file.mjs` ved det eneste skrive-chokepunktet (kb-update §3b.d.7/§4/§5 + generate-skills per-batch/pre-commit). Provenance-tiered BLOCK/WARN; injection-flagg → samme menneske-i-loop som status-påstand. 30 tester (692/0). Premiss-korr.: research/research-agent skriver ingenting (kun Layer A); CLI-scan alene misset injection+base64 → importerer rene primitiver. **Baseline-adjudikering (Enhet A2) LUKKET 2026-07-18 (TDD):** korpus-baseline 55/389 flagget (83 funn) → 4 ekte defekter fikset (2 Cyrillic-homoglyph-ord, 2 filer med U+00AD) + 75 funn human-adjudikert inn i innholds-basert allowlist (`scripts/kb-update/data/layerb-allowlist.json`: class+evidence+tier+eksakt trimmet linjeinnhold per entry, med begrunnelse; flyttet linje forblir grønn, endret innhold GJENOPPSTÅR som flagg — scanneren selv usvekket, korrupt/manglende allowlist → tom = full strenghet). Korpus 389/389 OK exit 0; commit-gaten (pre-commit-scan) leser samme allowlist. **Layer A (a) LUKKET 2026-07-18 (Enhet B):** `post-mcp-verify` aktivert kirurgisk i `~/.claude/settings.json` (PostToolUse-matcher `mcp__microsoft-learn__.*`) + live-verifisert fyrende med 2 foreground `microsoft_docs_fetch` — og en **ekte hook-defekt funnet+fikset underveis** (hooken leste `tool_output`, live-protokollen sender `tool_response` → hooken skannet ingenting; fiks TDD i llm-security `44aa390` v7.8.3). Se lukke-logg. | Layer A: R7 (første judge-pass-fetch-økt) / enhver `/architect:kb-update`/`generate-skills`-fetch-økt |
|
||||
|
||||
| **G7** | **Ingen rute for korreksjoner som er RIKTIGE, men større enn O2-konvolutten.** O2 er definert som én lokator + kun-sletting. R11 §9.3/§9.4 produserte fire funn der den korrekte fiksen beviselig ligger utenfor: idx 18 (`rag-caching-optimization.md:29` — den overlevende påstanden er en hel titulert seksjon 303-318 **pluss** en `**Verified**`-rad på 510; ingen sletting begrenset til linje 29 kan reparere fila), idx 36 (`ai-threat-modeling-stride.md:38` — companion-edit på 310 kreves for at prosaen skal matche den innsnevrede severity-tabellen), idx 17 (`**Verified**`-stemplet på 258 stempler etter editen kun retnings-utsagnet), idx 33 (innholdet overlever på 357 under CAF-attribusjon, så editens gevinst er mindre enn den ser ut). Alle fire er i dag kun prosa i `r11-pilot-results.md` | **Sanne defekter som stille faller ut av programmet fordi ingen mekanisme eier dem.** O2-triagen avviser dem (utenfor konvolutt), O3 dekker dem ikke (fiksen er ikke en verdi-swap), og det menneskelige review-sporet har ingen inngangskø. Nettoeffekten er at den *vanskeligste* klassen — der fila motsier seg selv — er den eneste uten eier | **Ikke designet.** Minimum: en eksplisitt fler-lokator-klasse (O4?) med egen retur-kontrakt og egen maskin-sjekk, ELLER en navngitt kø inn i den menneskelige review-fasen. Beslutningen henger sammen med ÅPEN OPERATØRBESLUTNING #2 (lønner korpus-bred O2-klassifisering seg) — hvis svaret er «hopp til menneskelig review», er G7 samme kø | 🔴 **ikke startet** (reist 2026-08-03, R11 §9.4) | Før R11s menneskelige review-fase erklæres ferdig — ellers er de fire funnene tapt |
|
||||
|
||||
**Ikke mekanisme-gap, men sporet backlog (innhold, ikke loop):** reference-`.md`-fil-fiksene fra Spor 2b (FP1 11000+/40+, FP2 «kun», FP6 Preview/Norway-East, FN2–FN6 utdaterte tall) **+ G5b** (`adr-template.md` fjern «zero permission management»; `multi-region-azure-openai-deployment.md` bytt retired `gpt-35-turbo` → gjeldende modell; `network-resilience-patterns-ai.md` «obligatorisk» → «anbefalt»; `vector-storage-cost-optimization.md` GA-dato `2024-11-01` → `2024-07-01`) er **Spor 0/1**-innholdsarbeid — pekt per-claim i `notes`, ikke gjentakelses-mekanisme. Føres i Spor 0-manifest / Spor 1-korpus-pass, ikke her.
|
||||
|
||||
### Lukke-logg
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue