ms-ai-architect/docs/ingestion-security-brief-2026-07.md

# Brief: Ingestion-pipeline security gate (G6 / R6 punkt d)

**Design input for the two-layer ingestion security gate. Goal: the highest
feasible level of adversarial-content defense on the chain that turns fetched
Microsoft Learn content into a publicly distributed knowledge base.**

Status: design brief. This sharpens and hardens the gate already decided in
`docs/ref-kb-correctness-program-2026-06.md` §8 (G6) and scoped for
`docs/plugin-roadmap-2026-07.md` R6 punkt (d). It does **not** invent a parallel
mechanism — it is the concrete design for that existing item.

---

## 1. Why this is load-bearing for *this* plugin specifically

The KB is not a private cache. Reference files under `skills/**/references/**`
become **instruction-adjacent context in future agent sessions**, and the plugin
ships on a public remote to the whole of Norwegian public sector. One poisoned
reference file is re-served to every user of every downstream session (G6 §8,
verbatim: *"references-filene blir instruksjonsnær kontekst i fremtidige
agent-sesjoner, én forgiftet fil re-serveres til alle brukere (hele Norge)"*).
That makes write-time adversarial-content defense a first-class control, not
hygiene — the opposite end of the risk spectrum from a pinned, single-author
source.

## 2. The pipeline as it actually is (grounded)

Verified by reading `commands/kb-update.md` and `CLAUDE.md`:

- **Ingestion runs in-session, in Claude Code, via the `microsoft-learn` MCP.**
  Apply is *"alltid manuell og kjøres in-session"* (`kb-update.md:13`); content
  enters through `microsoft_docs_fetch` / `microsoft_docs_search` /
  `microsoft_code_sample_search` (`kb-update.md:5,22,180`). `generate-skills`
  (batch MCP-research) and `research` / `research-agent` follow the same
  in-session-MCP-fetch → persist pattern (`CLAUDE.md:39,56`).
- **The detection tier is Claude-free** — it polls sitemaps for `<lastmod>` only
  and never feeds fetched *content* to a model (`kb-update.md:13,52-73`). The
  adversarial-content-into-model boundary is therefore the **in-session apply**,
  not the schedulable detection.
- **Write and commit points are explicit.** New file: fetch → transform
  (`transform-prompt.md`) → born-verified judge → `composeKbFile` →
  `validateKbFile` → create-guard `validate-kb-file.mjs` → atomic write
  (`kb-update.md:127-134`). Update: fetch → classify change → `Edit` →
  `validateKbFile` → commit `chore(ms-ai-architect): refresh KB <fil>`
  (`kb-update.md:180-189`); bulk commit `git add skills/` (`kb-update.md:196`).
- **Existing defenses are substantial — but aimed at correctness, not
  adversarial content.** Authority-source binding (`lib/authority.mjs`),
  verify-out with an adversarial refutation panel (`lib/verify-out.mjs`; status
  claims are *always* flagged), the born-verified judge (v3.1), the create-guard,
  and the "never auto-fix KB — flag → human → fix" discipline all answer *"is
  this claim true against its authority?"* None of them answers *"is this fetched
  chunk trying to inject instructions or smuggle an invisible payload?"* G6 §8
  states this precisely: *"Tillit til MS Learn dekker faktisk korrekthet — ikke
  adversarielt innhold i kanalen eller i kodeeksempler/lokalisert stoff."*

**That is the gap this brief closes.** It is orthogonal to the correctness
machinery, and must not replace or weaken it.

## 3. Threat model

- **Vector.** Microsoft Learn is authoritative for its core docs, but it has
  surfaces that are community-contributable or machine-ingested and *not* authored-
  and-reviewed to the same standard: **code samples**, **localized strings**,
  community doc contributions, and Q&A / forum-ingested material where fetched.
  An adversary who lands a payload in one of those surfaces gets it fetched,
  summarized, and **persisted** into a reference file.
- **Payload classes.** Indirect prompt injection (imperative text aimed at a
  future reading agent: spoofed `<system>` blocks, "note to the assistant…",
  identity redefinition), invisible-character / bidi steganography, and
  encoded (base64/hex) smuggling — especially inside fenced **code blocks**,
  which G6 explicitly flags and which a correctness judge will happily pass as
  "a valid code sample."
- **Impact.** Persisted once, re-served to all downstream sessions. Because the
  KB loads as instruction-adjacent context, a successful payload is a
  supply-chain compromise of every consumer, not a single bad answer.

## 4. Architecture reconciliation (read this before choosing a layer)

The sibling repo `llm-ingestion-pipeline-security` establishes the principle:
*defense belongs at the layer where the fetch and the LLM call actually happen.*
Applied honestly, that principle points in **opposite directions** for two
plugins, and the difference is the whole design:

- **`claude-code-llm-wiki`** calls the Anthropic SDK from a **standalone script**.
  Claude Code plugin hooks never fire there, so the `llm-security` plugin is the
  *wrong layer* — its defense must live in the pipeline code.
- **`ms-ai-architect`** fetches **in-session via MCP**. Here the `llm-security`
  plugin's `post-mcp-verify` hook *does* fire and *does* scan MCP tool output
  (including `microsoft_docs_fetch`). So the plugin is the **right layer for the
  input scan** — which is exactly why G6 layer (a) is coherent.

But right-layer is **not sufficient**, for two reasons:

1. **Hooks are unreliable headless** (GH #36071, flagged in STATE). This is
   partly mitigated because kb-update *apply* is in-session-only by design
   (`kb-update.md:237`), but the R7+ judge-pass and any future automation cannot
   depend on a hook firing.
2. **The hook scans the input, not the artifact.** `post-mcp-verify` sees the MCP
   response; it does **not** see what the model writes to `skills/**/*.md` after
   transformation. The persisted file is the poisoning vector, and nothing
   currently scans it before commit (the commit gate is gitleaks = secrets only).

**Conclusion:** the deterministic, always-on **output scan before commit** is the
load-bearing gate for *both* plugins. The plugin hook is an in-session
early-warning bonus that happens to apply here — valuable, but not the gate.

## 5. The two-layer gate, hardened

Building on G6/R6, with the emphasis corrected per §4:

### Layer A — input scan (in-session, advisory / early warning)

- `llm-security` active and `post-mcp-verify` verified firing in every in-session
  fetch (kb-update apply, generate-skills, research, R7+ judge-pass). This is the
  G6 activation rule — already in effect for fetch sessions (STATE bears it).
- **Sanitize + spotlight-fence fetched content before it reaches the transform /
  judge model.** Strip carrier classes (invisible chars, bidi, HTML comments)
  and wrap the fetched markdown as untrusted data in `transform-prompt.md` and
  `judge-claim-prompt-v3.1.md`, so the *in-session* model itself is not steered.
  (This protects the transform step; Layer B protects the artifact.)
- Treat Layer A as **advisory**. It must never be the sole gate, because of §4.1.

### Layer B — output scan before commit (deterministic, always-on, authoritative)

This is the gate. A deterministic Node scan (unicode / decode / injection —
covering prose **and fenced code blocks**) over changed `skills/**/*.md`, wired in
at two existing chokepoints so it runs regardless of any hook:

1. **Into the create-guard.** Add a security-scan sibling to
   `validate-kb-file.mjs` (`kb-update.md:134`): a file that fails the adversarial-
   content scan is never written — same shape as the existing "transform.mjs never
   writes / create-guard before write" invariant.
2. **Before commit.** Scan the staged `skills/**/*.md` set before the
   `chore(...): refresh KB` commit (`kb-update.md:189,196`). A hit blocks the
   commit.

### Disposition — provenance-tiered, composed with existing discipline

- Reuse the existing authority/`**Source:**` machinery (`lib/authority.mjs`) to
  tag each fetched chunk's **provenance trust**: authored-doc vs code-sample vs
  localized vs community/Q&A.
- **High-severity injection in a low-trust chunk → hard-fail / quarantine** (never
  written). **Any-severity in a high-trust chunk → WARN + human review.** Source
  trust is a first-class input to disposition, not a global constant
  (mirrors the sibling brief's principle 4.7).
- An injection flag routes to the **same human-in-loop** as a status-claim flag:
  quarantine, operator adjudicates, never auto-committed. No new mechanism — this
  reuses "never auto-fix KB — flag → human → fix."

## 5b. Layer A-aktiveringsprotokoll (R7-gate)

R7 (og enhver senere fetch-økt) starter **ALDRI** før denne sjekklisten passerer. Den
operasjonaliserer Layer A som en hard R7-gate, ikke bare en aktiveringsregel:

1. **Aktiver hooken.** Slå på `llm-security`s `post-mcp-verify` i `~/.claude/settings.json`
   (PostToolUse på `microsoft_docs_fetch`). Dette er en konfig-endring i operatørens
   `settings.json`, ikke i plugin-repoet.
2. **Verifiser at den fyrer.** Kjør ÉN live **foreground** `microsoft_docs_fetch` og bekreft
   at `post-mcp-verify` faktisk fyrte (observer hook-output). En hook som ikke observeres fyre
   teller som IKKE aktiv.
3. **Kompenserende skann hvis den ikke fyrer.** Fyrer den ikke (headless / GH #36071), kjøres
   den kompenserende deterministiske skannen **foreground** på den komponerte artefakten FØR
   write: Layer B (`scan-adversarial-content.mjs`) + Step 3s pre-write unicode/carrier-scan
   (`detectAdversarial` uten `path` → temp-fil, R6 Step 3). Denne er alltid-på og autoritativ
   (§5 Layer B); Layer A er kun tidlig-varsling.
4. **Frossen judge → foreground er obligatorisk.** Det fetchede innholdet **prompt-fences IKKE**
   inn i judgen: v3.1-judgen er frosset (G1-adoptert), og å legge til fencing ville være en
   judge-bump (Non-goal §7, krever re-måling av P/R). Uten judge-side fencing er **foreground
   fetch obligatorisk** — det er det som lar Layer A-hooken (og operatørens øye) se innholdet
   før judgen. (§5s fencing-ambisjon gjelder transform-steget, ikke den frosne judgen.)
5. **Bekreft Layer B-substratet før R7.** `llm-security`-sibling MÅ være til stede (ev. via
   `LLM_SECURITY_ROOT`) — ellers fail-closer Layer B og **BLOCKer alt** (gaten blir en vegg).
   Bekreft at `../llm-security/scanners/…` løser før første fetch.

**R7 starter KUN når 1–5 er grønne.**

**Rollback:** aktiveringen i pkt. 1 er reverserbar — fjern `post-mcp-verify`-oppføringen fra
`~/.claude/settings.json` (og ev. `LLM_SECURITY_ROOT`) for å ta ned Layer A igjen. Layer B
(deterministisk, in-repo) er upåvirket av denne rollback-en.

## 6. No local solutions — the deterministic scanner is a shared asset

Per house policy (*ingen lokale løsninger*, *showcase reusable patterns*): the
Layer B scanner is **not** a bespoke ms-ai-architect script. Two horizons:

- **Near-term:** run the existing `llm-security` deterministic scanners
  (`unicode-scanner`, `injection-patterns` lexicon, entropy/base64) as a **CLI**
  over the changed files — already ToS-safe (local scripts, no Claude), already
  the plan of record in G6 layer (b).
- **Target:** that same deterministic scanner is the core of the sibling
  `llm-ingestion-pipeline-security` library. Both this plugin's Layer B and
  `claude-code-llm-wiki`'s A13 output-lint should converge on **one**
  implementation and **one** lexicon dataset, so the pattern table does not drift
  into two copies. This brief is the second consumer that justifies extracting it.

**Coordination (2026-07-05).** The durable cross-repo mechanism is **hub-and-spoke**:
the canonical convergence contract lives in the hub (`llm-ingestion-pipeline-security`,
the designated shared asset); ms-ai-architect's spoke pointer is roadmap **R19**. See the
outgoing brief delivered to the hub repo (2026-07-05) for the proposed contract, the
confirmed language data points (ms-ai-architect Layer B = Node), and the open
`claude-code-llm-wiki` language brick that locks the polyglot-vs-Node-primary decision.

## 7. Non-goals

- **Not** a query-time guardrail (downstream sessions can layer their own).
- **Not** a replacement for the correctness / judge / authority machinery — this
  is orthogonal, adversarial-content defense. Both run.
- **Not** an auto-fixer — it blocks and flags; humans adjudicate.
- **Not** a ToS change — detection stays Claude-free; the deterministic scan is
  local; in-session hooks are unaffected.

## 8. Verification (testable, TDD / Iron Law)

Write the failing tests first; no production scan code without a red test.

- **Seeded adversarial fixtures blocked before commit.** A synthetic "MS Learn"
  doc carrying (a) a spoofed `<system>` / "note to the assistant" payload, (b)
  zero-width / bidi smuggling, (c) a base64 blob inside a code sample — each is
  rejected by the Layer B create-guard *before* write, and blocks the commit if
  staged. Test asserts no file written and non-zero guard exit.
- **Clean doc passes** untouched (byte-identical), guard exit 0.
- **Provenance tiering.** A high-severity hit in a code-sample/Q&A-tier chunk
  hard-fails; the same string in an authored-doc-tier chunk yields WARN + review,
  not a silent block. Test both directions.
- **Composition with correctness.** A doc that is factually correct but carries a
  payload is still blocked (proves the gate is orthogonal to the judge).
- **Full suite** `node --test tests/kb-update/*.test.mjs tests/kb-eval/*.test.mjs`
  exit 0; corpus untouched by the security-code change.
- **Layer A firing check** documented for one live fetch session
  (`post-mcp-verify` observed firing), with the headless caveat noted.

## 9. Relationship to the roadmap

This brief IS the design for **R6 punkt (d)** / **G6 §8**. Sequencing is
unchanged: gate designed at R6, enforced from R7 and in the kb-update cadence.
The G6 **activation rule already applies immediately** to any fetch session
(`/architect:kb-update`, `generate-skills`, research, judge-pass) — this brief
does not change that; it specifies what the enforced gate must do.

## 10. Verification log (sources; what is code-verified vs manifest-sourced)

- **Code-verified (read directly this session):** the in-session-MCP architecture,
  fetch tools, write/commit chokepoints, and existing defenses — all from
  `commands/kb-update.md` (line refs cited inline) and `CLAUDE.md`.
- **Decision text:** G6 register and R6 punkt (d) quoted from
  `docs/ref-kb-correctness-program-2026-06.md` §8 and
  `docs/plugin-roadmap-2026-07.md`.
- **Manifest-sourced, confirm in code before wiring:** the exact write/commit
  insertion points for `generate-skills`, `research`/`research-agent`, and the
  R7 judge-pass follow the same pattern per `CLAUDE.md:39,56` but were not read
  line-by-line here (an initial recon subagent was interrupted by a model usage
  limit; this brief is grounded in the files read directly by the main session).
  The implementing session must confirm those insertion points against the live
  command/agent code before wiring Layer B into them.
- **Sibling contract:** `llm-ingestion-pipeline-security/docs/BRIEF.md` (the
  write-time ingestion contract and the source-trust disposition principle).