ms-ai-architect/docs/ingestion-security-brief-2026-07.md

Brief: Ingestion-pipeline security gate (G6 / R6 punkt d)

Design input for the two-layer ingestion security gate. Goal: the highest feasible level of adversarial-content defense on the chain that turns fetched Microsoft Learn content into a publicly distributed knowledge base.

Status: design brief. This sharpens and hardens the gate already decided in docs/ref-kb-correctness-program-2026-06.md §8 (G6) and scoped for docs/plugin-roadmap-2026-07.md R6 punkt (d). It does not invent a parallel mechanism — it is the concrete design for that existing item.

---

1. Why this is load-bearing for this plugin specifically

The KB is not a private cache. Reference files under skills/**/references/** become instruction-adjacent context in future agent sessions, and the plugin ships on a public remote to the whole of Norwegian public sector. One poisoned reference file is re-served to every user of every downstream session (G6 §8, verbatim: "references-filene blir instruksjonsnær kontekst i fremtidige agent-sesjoner, én forgiftet fil re-serveres til alle brukere (hele Norge)"). That makes write-time adversarial-content defense a first-class control, not hygiene — the opposite end of the risk spectrum from a pinned, single-author source.

2. The pipeline as it actually is (grounded)

Verified by reading commands/kb-update.md and CLAUDE.md:

• Ingestion runs in-session, in Claude Code, via the microsoft-learn MCP. Apply is "alltid manuell og kjøres in-session" (kb-update.md:13); content enters through microsoft_docs_fetch / microsoft_docs_search / microsoft_code_sample_search (kb-update.md:5,22,180). generate-skills (batch MCP-research) and research / research-agent follow the same in-session-MCP-fetch → persist pattern (CLAUDE.md:39,56).
• The detection tier is Claude-free — it polls sitemaps for <lastmod> only and never feeds fetched content to a model (kb-update.md:13,52-73). The adversarial-content-into-model boundary is therefore the in-session apply, not the schedulable detection.
• Write and commit points are explicit. New file: fetch → transform (transform-prompt.md) → born-verified judge → composeKbFile → validateKbFile → create-guard validate-kb-file.mjs → atomic write (kb-update.md:127-134). Update: fetch → classify change → Edit → validateKbFile → commit chore(ms-ai-architect): refresh KB <fil> (kb-update.md:180-189); bulk commit git add skills/ (kb-update.md:196).
• Existing defenses are substantial — but aimed at correctness, not adversarial content. Authority-source binding (lib/authority.mjs), verify-out with an adversarial refutation panel (lib/verify-out.mjs; status claims are always flagged), the born-verified judge (v3.1), the create-guard, and the "never auto-fix KB — flag → human → fix" discipline all answer "is this claim true against its authority?" None of them answers "is this fetched chunk trying to inject instructions or smuggle an invisible payload?" G6 §8 states this precisely: "Tillit til MS Learn dekker faktisk korrekthet — ikke adversarielt innhold i kanalen eller i kodeeksempler/lokalisert stoff."

That is the gap this brief closes. It is orthogonal to the correctness machinery, and must not replace or weaken it.

3. Threat model

• Vector. Microsoft Learn is authoritative for its core docs, but it has surfaces that are community-contributable or machine-ingested and not authored- and-reviewed to the same standard: code samples, localized strings, community doc contributions, and Q&A / forum-ingested material where fetched. An adversary who lands a payload in one of those surfaces gets it fetched, summarized, and persisted into a reference file.
• Payload classes. Indirect prompt injection (imperative text aimed at a future reading agent: spoofed <system> blocks, "note to the assistant…", identity redefinition), invisible-character / bidi steganography, and encoded (base64/hex) smuggling — especially inside fenced code blocks, which G6 explicitly flags and which a correctness judge will happily pass as "a valid code sample."
• Impact. Persisted once, re-served to all downstream sessions. Because the KB loads as instruction-adjacent context, a successful payload is a supply-chain compromise of every consumer, not a single bad answer.

4. Architecture reconciliation (read this before choosing a layer)

The sibling repo llm-ingestion-pipeline-security establishes the principle: defense belongs at the layer where the fetch and the LLM call actually happen. Applied honestly, that principle points in opposite directions for two plugins, and the difference is the whole design:

• claude-code-llm-wiki calls the Anthropic SDK from a standalone script. Claude Code plugin hooks never fire there, so the llm-security plugin is the wrong layer — its defense must live in the pipeline code.
• ms-ai-architect fetches in-session via MCP. Here the llm-security plugin's post-mcp-verify hook does fire and does scan MCP tool output (including microsoft_docs_fetch). So the plugin is the right layer for the input scan — which is exactly why G6 layer (a) is coherent.

But right-layer is not sufficient, for two reasons:

1. Hooks are unreliable headless (GH #36071, flagged in STATE). This is partly mitigated because kb-update apply is in-session-only by design (kb-update.md:237), but the R7+ judge-pass and any future automation cannot depend on a hook firing.
2. The hook scans the input, not the artifact. post-mcp-verify sees the MCP response; it does not see what the model writes to skills/**/*.md after transformation. The persisted file is the poisoning vector, and nothing currently scans it before commit (the commit gate is gitleaks = secrets only).

Conclusion: the deterministic, always-on output scan before commit is the load-bearing gate for both plugins. The plugin hook is an in-session early-warning bonus that happens to apply here — valuable, but not the gate.

5. The two-layer gate, hardened

Building on G6/R6, with the emphasis corrected per §4:

Layer A — input scan (in-session, advisory / early warning)

• llm-security active and post-mcp-verify verified firing in every in-session fetch (kb-update apply, generate-skills, research, R7+ judge-pass). This is the G6 activation rule — already in effect for fetch sessions (STATE bears it).
• Sanitize + spotlight-fence fetched content before it reaches the transform / judge model. Strip carrier classes (invisible chars, bidi, HTML comments) and wrap the fetched markdown as untrusted data in transform-prompt.md and judge-claim-prompt-v3.1.md, so the in-session model itself is not steered. (This protects the transform step; Layer B protects the artifact.)
• Treat Layer A as advisory. It must never be the sole gate, because of §4.1.

Layer B — output scan before commit (deterministic, always-on, authoritative)

This is the gate. A deterministic Node scan (unicode / decode / injection — covering prose and fenced code blocks) over changed skills/**/*.md, wired in at two existing chokepoints so it runs regardless of any hook:

1. Into the create-guard. Add a security-scan sibling to validate-kb-file.mjs (kb-update.md:134): a file that fails the adversarial- content scan is never written — same shape as the existing "transform.mjs never writes / create-guard before write" invariant.
2. Before commit. Scan the staged skills/**/*.md set before the chore(...): refresh KB commit (kb-update.md:189,196). A hit blocks the commit.

Disposition — provenance-tiered, composed with existing discipline

• Reuse the existing authority/**Source:** machinery (lib/authority.mjs) to tag each fetched chunk's provenance trust: authored-doc vs code-sample vs localized vs community/Q&A.
• High-severity injection in a low-trust chunk → hard-fail / quarantine (never written). Any-severity in a high-trust chunk → WARN + human review. Source trust is a first-class input to disposition, not a global constant (mirrors the sibling brief's principle 4.7).
• An injection flag routes to the same human-in-loop as a status-claim flag: quarantine, operator adjudicates, never auto-committed. No new mechanism — this reuses "never auto-fix KB — flag → human → fix."

5b. Layer A-aktiveringsprotokoll (R7-gate)

R7 (og enhver senere fetch-økt) starter ALDRI før denne sjekklisten passerer. Den operasjonaliserer Layer A som en hard R7-gate, ikke bare en aktiveringsregel:

1. Aktiver hooken. Slå på llm-securitys post-mcp-verify i ~/.claude/settings.json (PostToolUse på microsoft_docs_fetch). Dette er en konfig-endring i operatørens settings.json, ikke i plugin-repoet.
2. Verifiser at den fyrer. Kjør ÉN live foreground microsoft_docs_fetch og bekreft at post-mcp-verify faktisk fyrte (observer hook-output). En hook som ikke observeres fyre teller som IKKE aktiv.
3. Kompenserende skann hvis den ikke fyrer. Fyrer den ikke (headless / GH #36071), kjøres den kompenserende deterministiske skannen foreground på den komponerte artefakten FØR write: Layer B (scan-adversarial-content.mjs) + Step 3s pre-write unicode/carrier-scan (detectAdversarial uten path → temp-fil, R6 Step 3). Denne er alltid-på og autoritativ (§5 Layer B); Layer A er kun tidlig-varsling.
4. Frossen judge → foreground er obligatorisk. Det fetchede innholdet prompt-fences IKKE inn i judgen: v3.1-judgen er frosset (G1-adoptert), og å legge til fencing ville være en judge-bump (Non-goal §7, krever re-måling av P/R). Uten judge-side fencing er foreground fetch obligatorisk — det er det som lar Layer A-hooken (og operatørens øye) se innholdet før judgen. (§5s fencing-ambisjon gjelder transform-steget, ikke den frosne judgen.)
5. Bekreft Layer B-substratet før R7. llm-security-sibling MÅ være til stede (ev. via LLM_SECURITY_ROOT) — ellers fail-closer Layer B og BLOCKer alt (gaten blir en vegg). Bekreft at ../llm-security/scanners/… løser før første fetch.

R7 starter KUN når 1–5 er grønne.

Rollback: aktiveringen i pkt. 1 er reverserbar — fjern post-mcp-verify-oppføringen fra ~/.claude/settings.json (og ev. LLM_SECURITY_ROOT) for å ta ned Layer A igjen. Layer B (deterministisk, in-repo) er upåvirket av denne rollback-en.

6. No local solutions — the deterministic scanner is a shared asset

Per house policy (ingen lokale løsninger, showcase reusable patterns): the Layer B scanner is not a bespoke ms-ai-architect script. Two horizons:

• Near-term: run the existing llm-security deterministic scanners (unicode-scanner, injection-patterns lexicon, entropy/base64) as a CLI over the changed files — already ToS-safe (local scripts, no Claude), already the plan of record in G6 layer (b).
• Target: that same deterministic scanner is the core of the sibling llm-ingestion-pipeline-security library. Both this plugin's Layer B and claude-code-llm-wiki's A13 output-lint should converge on one implementation and one lexicon dataset, so the pattern table does not drift into two copies. This brief is the second consumer that justifies extracting it.

Coordination (2026-07-05). The durable cross-repo mechanism is hub-and-spoke: the canonical convergence contract lives in the hub (llm-ingestion-pipeline-security, the designated shared asset); ms-ai-architect's spoke pointer is roadmap R19. See the outgoing brief delivered to the hub repo (2026-07-05) for the proposed contract, the confirmed language data points (ms-ai-architect Layer B = Node), and the open claude-code-llm-wiki language brick that locks the polyglot-vs-Node-primary decision.

7. Non-goals

• Not a query-time guardrail (downstream sessions can layer their own).
• Not a replacement for the correctness / judge / authority machinery — this is orthogonal, adversarial-content defense. Both run.
• Not an auto-fixer — it blocks and flags; humans adjudicate.
• Not a ToS change — detection stays Claude-free; the deterministic scan is local; in-session hooks are unaffected.

8. Verification (testable, TDD / Iron Law)

Write the failing tests first; no production scan code without a red test.

• Seeded adversarial fixtures blocked before commit. A synthetic "MS Learn" doc carrying (a) a spoofed <system> / "note to the assistant" payload, (b) zero-width / bidi smuggling, (c) a base64 blob inside a code sample — each is rejected by the Layer B create-guard before write, and blocks the commit if staged. Test asserts no file written and non-zero guard exit.
• Clean doc passes untouched (byte-identical), guard exit 0.
• Provenance tiering. A high-severity hit in a code-sample/Q&A-tier chunk hard-fails; the same string in an authored-doc-tier chunk yields WARN + review, not a silent block. Test both directions.
• Composition with correctness. A doc that is factually correct but carries a payload is still blocked (proves the gate is orthogonal to the judge).
• Full suite node --test tests/kb-update/*.test.mjs tests/kb-eval/*.test.mjs exit 0; corpus untouched by the security-code change.
• Layer A firing check documented for one live fetch session (post-mcp-verify observed firing), with the headless caveat noted.

9. Relationship to the roadmap

This brief IS the design for R6 punkt (d) / G6 §8. Sequencing is unchanged: gate designed at R6, enforced from R7 and in the kb-update cadence. The G6 activation rule already applies immediately to any fetch session (/architect:kb-update, generate-skills, research, judge-pass) — this brief does not change that; it specifies what the enforced gate must do.

10. Verification log (sources; what is code-verified vs manifest-sourced)

• Code-verified (read directly this session): the in-session-MCP architecture, fetch tools, write/commit chokepoints, and existing defenses — all from commands/kb-update.md (line refs cited inline) and CLAUDE.md.
• Decision text: G6 register and R6 punkt (d) quoted from docs/ref-kb-correctness-program-2026-06.md §8 and docs/plugin-roadmap-2026-07.md.
• Manifest-sourced, confirm in code before wiring: the exact write/commit insertion points for generate-skills, research/research-agent, and the R7 judge-pass follow the same pattern per CLAUDE.md:39,56 but were not read line-by-line here (an initial recon subagent was interrupted by a model usage limit; this brief is grounded in the files read directly by the main session). The implementing session must confirm those insertion points against the live command/agent code before wiring Layer B into them.
• Sibling contract: llm-ingestion-pipeline-security/docs/BRIEF.md (the write-time ingestion contract and the source-trust disposition principle).