docs(ms-ai-architect): forankre ingestion-security-brief (G6 §8 / R6 punkt d — to-lags gate) [skip-docs]

docs/ingestion-security-brief-2026-07.md

@@ -0,0 +1,218 @@
+# Brief: Ingestion-pipeline security gate (G6 / R6 punkt d)
+
+**Design input for the two-layer ingestion security gate. Goal: the highest
+feasible level of adversarial-content defense on the chain that turns fetched
+Microsoft Learn content into a publicly distributed knowledge base.**
+
+Status: design brief. This sharpens and hardens the gate already decided in
+`docs/ref-kb-correctness-program-2026-06.md` §8 (G6) and scoped for
+`docs/plugin-roadmap-2026-07.md` R6 punkt (d). It does **not** invent a parallel
+mechanism — it is the concrete design for that existing item.
+
+---
+
+## 1. Why this is load-bearing for *this* plugin specifically
+
+The KB is not a private cache. Reference files under `skills/**/references/**`
+become **instruction-adjacent context in future agent sessions**, and the plugin
+ships on a public remote to the whole of Norwegian public sector. One poisoned
+reference file is re-served to every user of every downstream session (G6 §8,
+verbatim: *"references-filene blir instruksjonsnær kontekst i fremtidige
+agent-sesjoner, én forgiftet fil re-serveres til alle brukere (hele Norge)"*).
+That makes write-time adversarial-content defense a first-class control, not
+hygiene — the opposite end of the risk spectrum from a pinned, single-author
+source.
+
+## 2. The pipeline as it actually is (grounded)
+
+Verified by reading `commands/kb-update.md` and `CLAUDE.md`:
+
+- **Ingestion runs in-session, in Claude Code, via the `microsoft-learn` MCP.**
+  Apply is *"alltid manuell og kjøres in-session"* (`kb-update.md:13`); content
+  enters through `microsoft_docs_fetch` / `microsoft_docs_search` /
+  `microsoft_code_sample_search` (`kb-update.md:5,22,180`). `generate-skills`
+  (batch MCP-research) and `research` / `research-agent` follow the same
+  in-session-MCP-fetch → persist pattern (`CLAUDE.md:39,56`).
+- **The detection tier is Claude-free** — it polls sitemaps for `<lastmod>` only
+  and never feeds fetched *content* to a model (`kb-update.md:13,52-73`). The
+  adversarial-content-into-model boundary is therefore the **in-session apply**,
+  not the schedulable detection.
+- **Write and commit points are explicit.** New file: fetch → transform
+  (`transform-prompt.md`) → born-verified judge → `composeKbFile` →
+  `validateKbFile` → create-guard `validate-kb-file.mjs` → atomic write
+  (`kb-update.md:127-134`). Update: fetch → classify change → `Edit` →
+  `validateKbFile` → commit `chore(ms-ai-architect): refresh KB <fil>`
+  (`kb-update.md:180-189`); bulk commit `git add skills/` (`kb-update.md:196`).
+- **Existing defenses are substantial — but aimed at correctness, not
+  adversarial content.** Authority-source binding (`lib/authority.mjs`),
+  verify-out with an adversarial refutation panel (`lib/verify-out.mjs`; status
+  claims are *always* flagged), the born-verified judge (v3.1), the create-guard,
+  and the "never auto-fix KB — flag → human → fix" discipline all answer *"is
+  this claim true against its authority?"* None of them answers *"is this fetched
+  chunk trying to inject instructions or smuggle an invisible payload?"* G6 §8
+  states this precisely: *"Tillit til MS Learn dekker faktisk korrekthet — ikke
+  adversarielt innhold i kanalen eller i kodeeksempler/lokalisert stoff."*
+
+**That is the gap this brief closes.** It is orthogonal to the correctness
+machinery, and must not replace or weaken it.
+
+## 3. Threat model
+
+- **Vector.** Microsoft Learn is authoritative for its core docs, but it has
+  surfaces that are community-contributable or machine-ingested and *not* authored-
+  and-reviewed to the same standard: **code samples**, **localized strings**,
+  community doc contributions, and Q&A / forum-ingested material where fetched.
+  An adversary who lands a payload in one of those surfaces gets it fetched,
+  summarized, and **persisted** into a reference file.
+- **Payload classes.** Indirect prompt injection (imperative text aimed at a
+  future reading agent: spoofed `<system>` blocks, "note to the assistant…",
+  identity redefinition), invisible-character / bidi steganography, and
+  encoded (base64/hex) smuggling — especially inside fenced **code blocks**,
+  which G6 explicitly flags and which a correctness judge will happily pass as
+  "a valid code sample."
+- **Impact.** Persisted once, re-served to all downstream sessions. Because the
+  KB loads as instruction-adjacent context, a successful payload is a
+  supply-chain compromise of every consumer, not a single bad answer.
+
+## 4. Architecture reconciliation (read this before choosing a layer)
+
+The sibling repo `llm-ingestion-pipeline-security` establishes the principle:
+*defense belongs at the layer where the fetch and the LLM call actually happen.*
+Applied honestly, that principle points in **opposite directions** for two
+plugins, and the difference is the whole design:
+
+- **`claude-code-llm-wiki`** calls the Anthropic SDK from a **standalone script**.
+  Claude Code plugin hooks never fire there, so the `llm-security` plugin is the
+  *wrong layer* — its defense must live in the pipeline code.
+- **`ms-ai-architect`** fetches **in-session via MCP**. Here the `llm-security`
+  plugin's `post-mcp-verify` hook *does* fire and *does* scan MCP tool output
+  (including `microsoft_docs_fetch`). So the plugin is the **right layer for the
+  input scan** — which is exactly why G6 layer (a) is coherent.
+
+But right-layer is **not sufficient**, for two reasons:
+
+1. **Hooks are unreliable headless** (GH #36071, flagged in STATE). This is
+   partly mitigated because kb-update *apply* is in-session-only by design
+   (`kb-update.md:237`), but the R7+ judge-pass and any future automation cannot
+   depend on a hook firing.
+2. **The hook scans the input, not the artifact.** `post-mcp-verify` sees the MCP
+   response; it does **not** see what the model writes to `skills/**/*.md` after
+   transformation. The persisted file is the poisoning vector, and nothing
+   currently scans it before commit (the commit gate is gitleaks = secrets only).
+
+**Conclusion:** the deterministic, always-on **output scan before commit** is the
+load-bearing gate for *both* plugins. The plugin hook is an in-session
+early-warning bonus that happens to apply here — valuable, but not the gate.
+
+## 5. The two-layer gate, hardened
+
+Building on G6/R6, with the emphasis corrected per §4:
+
+### Layer A — input scan (in-session, advisory / early warning)
+
+- `llm-security` active and `post-mcp-verify` verified firing in every in-session
+  fetch (kb-update apply, generate-skills, research, R7+ judge-pass). This is the
+  G6 activation rule — already in effect for fetch sessions (STATE bears it).
+- **Sanitize + spotlight-fence fetched content before it reaches the transform /
+  judge model.** Strip carrier classes (invisible chars, bidi, HTML comments)
+  and wrap the fetched markdown as untrusted data in `transform-prompt.md` and
+  `judge-claim-prompt-v3.1.md`, so the *in-session* model itself is not steered.
+  (This protects the transform step; Layer B protects the artifact.)
+- Treat Layer A as **advisory**. It must never be the sole gate, because of §4.1.
+
+### Layer B — output scan before commit (deterministic, always-on, authoritative)
+
+This is the gate. A deterministic Node scan (unicode / decode / injection —
+covering prose **and fenced code blocks**) over changed `skills/**/*.md`, wired in
+at two existing chokepoints so it runs regardless of any hook:
+
+1. **Into the create-guard.** Add a security-scan sibling to
+   `validate-kb-file.mjs` (`kb-update.md:134`): a file that fails the adversarial-
+   content scan is never written — same shape as the existing "transform.mjs never
+   writes / create-guard before write" invariant.
+2. **Before commit.** Scan the staged `skills/**/*.md` set before the
+   `chore(...): refresh KB` commit (`kb-update.md:189,196`). A hit blocks the
+   commit.
+
+### Disposition — provenance-tiered, composed with existing discipline
+
+- Reuse the existing authority/`**Source:**` machinery (`lib/authority.mjs`) to
+  tag each fetched chunk's **provenance trust**: authored-doc vs code-sample vs
+  localized vs community/Q&A.
+- **High-severity injection in a low-trust chunk → hard-fail / quarantine** (never
+  written). **Any-severity in a high-trust chunk → WARN + human review.** Source
+  trust is a first-class input to disposition, not a global constant
+  (mirrors the sibling brief's principle 4.7).
+- An injection flag routes to the **same human-in-loop** as a status-claim flag:
+  quarantine, operator adjudicates, never auto-committed. No new mechanism — this
+  reuses "never auto-fix KB — flag → human → fix."
+
+## 6. No local solutions — the deterministic scanner is a shared asset
+
+Per house policy (*ingen lokale løsninger*, *showcase reusable patterns*): the
+Layer B scanner is **not** a bespoke ms-ai-architect script. Two horizons:
+
+- **Near-term:** run the existing `llm-security` deterministic scanners
+  (`unicode-scanner`, `injection-patterns` lexicon, entropy/base64) as a **CLI**
+  over the changed files — already ToS-safe (local scripts, no Claude), already
+  the plan of record in G6 layer (b).
+- **Target:** that same deterministic scanner is the core of the sibling
+  `llm-ingestion-pipeline-security` library. Both this plugin's Layer B and
+  `claude-code-llm-wiki`'s A13 output-lint should converge on **one**
+  implementation and **one** lexicon dataset, so the pattern table does not drift
+  into two copies. This brief is the second consumer that justifies extracting it.
+
+## 7. Non-goals
+
+- **Not** a query-time guardrail (downstream sessions can layer their own).
+- **Not** a replacement for the correctness / judge / authority machinery — this
+  is orthogonal, adversarial-content defense. Both run.
+- **Not** an auto-fixer — it blocks and flags; humans adjudicate.
+- **Not** a ToS change — detection stays Claude-free; the deterministic scan is
+  local; in-session hooks are unaffected.
+
+## 8. Verification (testable, TDD / Iron Law)
+
+Write the failing tests first; no production scan code without a red test.
+
+- **Seeded adversarial fixtures blocked before commit.** A synthetic "MS Learn"
+  doc carrying (a) a spoofed `<system>` / "note to the assistant" payload, (b)
+  zero-width / bidi smuggling, (c) a base64 blob inside a code sample — each is
+  rejected by the Layer B create-guard *before* write, and blocks the commit if
+  staged. Test asserts no file written and non-zero guard exit.
+- **Clean doc passes** untouched (byte-identical), guard exit 0.
+- **Provenance tiering.** A high-severity hit in a code-sample/Q&A-tier chunk
+  hard-fails; the same string in an authored-doc-tier chunk yields WARN + review,
+  not a silent block. Test both directions.
+- **Composition with correctness.** A doc that is factually correct but carries a
+  payload is still blocked (proves the gate is orthogonal to the judge).
+- **Full suite** `node --test tests/kb-update/*.test.mjs tests/kb-eval/*.test.mjs`
+  exit 0; corpus untouched by the security-code change.
+- **Layer A firing check** documented for one live fetch session
+  (`post-mcp-verify` observed firing), with the headless caveat noted.
+
+## 9. Relationship to the roadmap
+
+This brief IS the design for **R6 punkt (d)** / **G6 §8**. Sequencing is
+unchanged: gate designed at R6, enforced from R7 and in the kb-update cadence.
+The G6 **activation rule already applies immediately** to any fetch session
+(`/architect:kb-update`, `generate-skills`, research, judge-pass) — this brief
+does not change that; it specifies what the enforced gate must do.
+
+## 10. Verification log (sources; what is code-verified vs manifest-sourced)
+
+- **Code-verified (read directly this session):** the in-session-MCP architecture,
+  fetch tools, write/commit chokepoints, and existing defenses — all from
+  `commands/kb-update.md` (line refs cited inline) and `CLAUDE.md`.
+- **Decision text:** G6 register and R6 punkt (d) quoted from
+  `docs/ref-kb-correctness-program-2026-06.md` §8 and
+  `docs/plugin-roadmap-2026-07.md`.
+- **Manifest-sourced, confirm in code before wiring:** the exact write/commit
+  insertion points for `generate-skills`, `research`/`research-agent`, and the
+  R7 judge-pass follow the same pattern per `CLAUDE.md:39,56` but were not read
+  line-by-line here (an initial recon subagent was interrupted by a model usage
+  limit; this brief is grounded in the files read directly by the main session).
+  The implementing session must confirm those insertion points against the live
+  command/agent code before wiring Layer B into them.
+- **Sibling contract:** `llm-ingestion-pipeline-security/docs/BRIEF.md` (the
+  write-time ingestion contract and the source-trust disposition principle).