1
0
Fork 0

docs(brief): fold in ms-ai-architect Layer B learnings (first output-side impl)

Based on the working, tested Layer B ingestion-gate in ms-ai-architect (2026-07-04):
- A: correct consumer 2 — it fetches authored Learn docs + code samples via the
  microsoft-learn MCP, NOT the open Q&A forum / MSDN / Stack Overflow (that claim
  was unverified/overstated); low-trust surface is intra-document. Repo/name
  reconciled: the consumer is ms-ai-architect, no separate "MS AI Security plugin".
- B: §4.7 — trust tiers WITHIN a document (code sample / localized string vs
  authored prose), not only across sources; carriers + critical block in any tier.
- C: §12 — consume the lexicon as pure imported functions, not the llm-security
  scan CLI (which under-covers Markdown prose + base64-in-code-block, verified).
- D: §4.6 — fail-secure extends to scanner-unavailable: un-scannable ⇒ BLOCK.
Adds ms-ai-architect as the second reference implementation (output side).
This commit is contained in:
Kjell Tore Guttormsen 2026-07-04 08:32:38 +02:00
commit d983aa3c95

View file

@ -90,7 +90,12 @@ bearing part. The brief must lead with the contract, not the regex.
content is a failure mode, not a safe default. content is a failure mode, not a safe default.
6. **Fail-secure under compound signals.** Injection-scan hit *and* a transform 6. **Fail-secure under compound signals.** Injection-scan hit *and* a transform
failure is treated as a probable forced-fallback attack: halt + alert, never failure is treated as a probable forced-fallback attack: halt + alert, never
an auto-committed verbatim entry. an auto-committed verbatim entry. The same default applies when the output
scanner itself is **unavailable or errors** (a missing/unresolvable detector
dependency, a scanner exception): treat an un-scannable artifact as a BLOCK —
never persist it unscanned. Fail closed, not open. (Verified in the first
output-side implementation, 2026-07-04: an unresolvable detector is turned into
a hard-fail, not a silent pass.)
7. **Disposition scales with source trust.** The *same* lexicon should not carry 7. **Disposition scales with source trust.** The *same* lexicon should not carry
the *same* disposition for every source. A pinned, reputable, single-author the *same* disposition for every source. A pinned, reputable, single-author
source (an official changelog) warrants **WARN** — false positives from source (an official changelog) warrants **WARN** — false positives from
@ -102,6 +107,19 @@ bearing part. The brief must lead with the contract, not the regex.
lower than publishing a poisoned one. Source trust level is therefore a lower than publishing a poisoned one. Source trust level is therefore a
first-class input to the disposition policy, not a global constant. first-class input to the disposition policy, not a global constant.
**Trust also tiers *within* a document, not only across sources** (verified in
the first output-side implementation, 2026-07-04). A fenced code sample or a
localized (non-English) string inside an otherwise-authoritative doc is the
low-trust surface *even when the document's source is reputable* — a spoofed
`<system>` block or a base64 blob in a code sample is far likelier a real
payload than a false positive, whereas the same lexicon hit in authored,
en-locale prose is likelier a doc legitimately discussing the pattern. So the
*same* hit hard-fails inside a code block / localized string but WARNs in
authored prose. Intra-document provenance (in-code-fence, non-en locale) is a
first-class disposition input alongside source-level trust. Invisible carriers
(zero-width / bidi / Unicode-tag) and critical injection are the exception:
they have no legitimate place in a reference file and block in any tier.
## 5. Proposed module layout ## 5. Proposed module layout
``` ```
@ -228,10 +246,25 @@ The contract in §6 is not hypothetical — it is extracted from a working pipel
the `claude-code-llm-wiki` Stage B enrichment path (`tools/wiki_ingest/`), the `claude-code-llm-wiki` Stage B enrichment path (`tools/wiki_ingest/`),
which already implements sanitize + spotlight-fence + tool-less quarantined SDK which already implements sanitize + spotlight-fence + tool-less quarantined SDK
call + per-stage credential isolation + schema-validated output + fail-secure call + per-stage credential isolation + schema-validated output + fail-secure
`ForcedFallbackHalt`. That pipeline is the reference implementation. The semantic `ForcedFallbackHalt`. That pipeline is the reference implementation for the *full
contract* (tool-less transform, credential isolation, fail-secure). The semantic
lexicon seed is the `injection-patterns.mjs` table from the `llm-security` plugin lexicon seed is the `injection-patterns.mjs` table from the `llm-security` plugin
(a pure regex+label+severity dataset, portable as data). (a pure regex+label+severity dataset, portable as data).
**Integration lesson (verified 2026-07-04) — consume the lexicon as a pure
function, not via the scanner CLI.** The `llm-security` orchestrator CLI
(`llm-security scan`) is repo/directory-oriented and, empirically, does **not**
apply the injection lexicon to arbitrary Markdown prose, nor does its
entropy-scanner flag a base64 blob inside a fenced code block — a full deep-scan
over three seeded fixtures caught only the invisible-Unicode class. The load-
bearing reuse seam is therefore the **importable pure primitives**
(`scanForInjection(text)` from `injection-patterns.mjs`; `isBase64Like` /
`shannonEntropy` from `string-utils.mjs`; the `unicode-scanner` module), driven
per line/region so the consumer controls provenance tiering. This library should
ship the lexicon **as data** and detection **as pure `text -> findings`
functions** (design principle 3) so no consumer has to shell out to an
orchestrator that under-covers its content type.
**Target consumers (the reason the library exists):** **Target consumers (the reason the library exists):**
1. **`claude-code-llm-wiki`** — low-untrust source (a pinned, reputable 1. **`claude-code-llm-wiki`** — low-untrust source (a pinned, reputable
@ -239,14 +272,34 @@ lexicon seed is the `injection-patterns.mjs` table from the `llm-security` plugi
architectural controls already close the severe outcomes, and the lexicon runs architectural controls already close the severe outcomes, and the lexicon runs
at WARN. This consumer is *retrofitting* the output-side widening late (at its at WARN. This consumer is *retrofitting* the output-side widening late (at its
A13), which is precisely the cost this library exists to avoid for the next one. A13), which is precisely the cost this library exists to avoid for the next one.
2. **MS AI Security plugin** — a heavier ingestion pipeline over **Microsoft 2. **`ms-ai-architect`** (the plugin in `ktg-plugin-marketplace`; there is **no**
Learn** content, which includes genuine **user-generated content**: Microsoft separate "MS AI Security plugin" — repo/name reconciled 2026-07-04) — an
Q&A (`learn.microsoft.com/answers`) is open-contribution, and its knowledge ingestion pipeline over **Microsoft Learn** content fetched via the
base ingests MSDN Forums and Stack Overflow. This is the *high-untrust* end, `microsoft-learn` MCP (`microsoft_docs_fetch` / `_search` /
where the contract is **load-bearing, not hygiene**, and disposition (§4.7) `_code_sample_search`): authored, reviewed Learn docs **plus code samples and
should default stricter (QUARANTINE_REVIEW / hard-fail on high-severity). localized strings**. **Correction (verified against the live command/agent code
*(Disk note: not yet a separate repo; the related existing plugin is 2026-07-04):** the earlier characterization — that it is the *high-untrust* end
`ktg-plugin-marketplace/ms-ai-architect`. Name/repo to be reconciled.)* ingesting the open Microsoft Q&A forum (`learn.microsoft.com/answers`), MSDN
Forums and Stack Overflow — is **unverified and overstated**. The pipeline's
fetch surface is authored Learn docs + code samples; it does not ingest the open
Q&A forum as part of its flow. Its low-trust surface is therefore
**intra-document** (a fenced code sample or a localized string inside an
otherwise-authoritative doc), which is exactly why disposition must tier at the
*chunk* level (§4.7), not treat the whole consumer as globally high-untrust.
Even so the contract is **load-bearing, not hygiene** here: the KB is publicly
distributed and re-served as instruction-adjacent context to every downstream
agent session, so one poisoned reference file is a supply-chain compromise.
A working, tested implementation of the **output-scan-before-persist** gate (§6
step 6) plus **provenance-tiered disposition** (§4.7) now exists in this plugin
— "Layer B" of its ingestion security gate, wired as a sibling to the existing
create-guard at the single write chokepoint, with a fail-closed exit-code
contract (0 clean / 1 block / 2 warn). It is the **second reference
implementation** for this library — specifically for the *output side* (the
RAG-poisoning gate + disposition), complementing consumer 1's reference for the
*full contract* (tool-less transform + credential isolation). It consumes the
`llm-security` lexicon by **in-process import of the pure functions** (see the
integration lesson above), which is a concrete data point for open decision §13.3.
**Why day-1, not retrofit.** The load-bearing parts of the contract are **Why day-1, not retrofit.** The load-bearing parts of the contract are
architectural — tool-less quarantine, per-stage credential isolation, architectural — tool-less quarantine, per-stage credential isolation,