docs(linkedin-studio): ingestion-guard adoption plan — persist-gate integration map (status: planned)
Read `llm-ingestion-guard` v0.2 adoption brief; mapped this repo's untrusted-ingest
surface (brief §7 checklist) against 882f6ee via two independent read-only surveys.
- New `docs/ingestion-guard/plan.md`: when/where to wire the guard at the deterministic
persist gates (`screen_output` at trends `capture`, `brain ingest`, specifics-bank
`ekstern` bindings, analytics CSV). Ranked by automated-reinjection risk; the trends ->
`session-start.mjs` reinjection is the one live poison->trusted-read loop (priority 1).
- Honest scope: only the `screen_output` half maps cleanly (fetch/transform happen inside
the model turn — no `your_model()` code seam); `prepare_input` has no clean wiring point.
- Python<->Node interop is the blocker (plan §7); no code wired (brief = plan-only).
- OKF `import_bundle` has no seam today (brain is export-only); relevant only when SB-S4
connector or a cross-plugin shared skill lands.
Machine-readable marker line for the guard repo's roll-up lives in STATE.md
(LOCAL-ONLY / gitignored, so the roll-up is machine-local).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: 57adea42-c8f1-4b88-acb6-2453e1239c79
This commit is contained in:
parent
882f6eee5e
commit
d67552eab1
1 changed files with 209 additions and 0 deletions
209
docs/ingestion-guard/plan.md
Normal file
209
docs/ingestion-guard/plan.md
Normal file
|
|
@ -0,0 +1,209 @@
|
|||
# Ingestion-guard integration plan — when & where to wire `llm-ingestion-guard`
|
||||
|
||||
> **Status: `planned` (2026-07-16).** Plan only — nothing wired yet, per the adoption brief's
|
||||
> "don't implement now" instruction. This document is the repo's durable answer to *when* and *where* a
|
||||
> write-time ingestion guard earns its place at our persist gates.
|
||||
>
|
||||
> **Guard:** `llm-ingestion-guard` `v0.2` (alpha, public API may change). Write-time sibling of query-time
|
||||
> chatbot guardrails; hardens *untrusted content flowing through an LLM step into a persisted,
|
||||
> downstream-read store*. Python, stdlib-only core, 3.10+.
|
||||
>
|
||||
> **Adoption brief (authoritative, self-contained):**
|
||||
> `https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security/raw/branch/main/docs/ADOPTION-BRIEF.md`
|
||||
> (§7 planning checklist, §3 the 8-step contract, §4 the OKF adapter, §8 honest limitations).
|
||||
>
|
||||
> Grounded against `882f6ee` by two independent read-only surveys (ingest surface + persist seams),
|
||||
> 2026-07-16. Every seam below carries a `file:line` anchor.
|
||||
|
||||
## 1. Decision
|
||||
|
||||
**Status = `planned`.** The plugin has live untrusted-ingest paths (a react-to-URL command, an auto-fetching
|
||||
trend agent, an external-research newsletter fan-out), and it persists their output into stores a downstream
|
||||
agent later reads as trusted context. That satisfies the brief's §7 decisive condition. Nothing is wired.
|
||||
The guard is registered here as a **dependency to add before the first *automated* external-origin ingest
|
||||
path goes fully live** — and one such path (trends → session re-injection, §5.1) is arguably already live,
|
||||
so it is the first integration candidate, not a distant one.
|
||||
|
||||
We do **not** implement now. The brief is explicit ("kun planlegg og oppdater planene"), and wiring is gated
|
||||
on an unresolved interop decision (§7).
|
||||
|
||||
## 2. What the guard is (and is not) for us
|
||||
|
||||
It is **not** a query-time guardrail between the user and the model. It is a **write gate**: the last place
|
||||
the provenance of a piece of content is still known before it is committed to a store that a *later* agent
|
||||
reads as trusted. For this plugin that store is the second-brain (`brain/profile.md`, `ingest/published/`),
|
||||
the trends store, the specifics-bank, the post queue, and the state file's `## Recent Posts` — all of which
|
||||
are re-surfaced into future model context. A poisoned concept committed at write time is read back later with
|
||||
its origin forgotten; the write gate is the only place to catch it.
|
||||
|
||||
## 3. Ingest-surface analysis (untrusted vs first-party)
|
||||
|
||||
Scored against the brief's §7 checklist. "Downstream trusted reader" = a command/agent/hook that later reads
|
||||
the store **as context**, which is what turns a write into a poisoning surface.
|
||||
|
||||
| # | Path | Origin | Untrusted? | Persist target | Downstream trusted reader | Live? |
|
||||
|---|------|--------|-----------|----------------|---------------------------|-------|
|
||||
| 1 | **trend-spotter agent** (`agents/trend-spotter.md`) → trends CLI `capture` (`scripts/trends/src/cli.ts:294`) | Auto-fetched web/vendor/regulator content (WebSearch/WebFetch + research MCPs) | **YES — external** | `trends/trends.json` — `source`/`title`/`url`/`summary` stored **verbatim** (`scripts/trends/src/item.ts:24-48`) | **`session-start.mjs:38-78` auto-reinjects into the next session's context — no human in the loop** | **LIVE** |
|
||||
| 2 | **`brain ingest` / `scanInbox`** (`scripts/brain/src/ingest.ts:172-192,:247`) | User's own published posts, dropped into `ingest/inbox/` (manual) | Origin first-party today; **untrusted-*capable*** (drop-zone accepts any file; SB-S4 connector would automate it) | `ingest/published/<id>.md` | `voice-trainer` gold source (`agents/voice-trainer.md:136,144`); `brain consolidate` → `brain/profile.md`; `assemble`/`reconcile` | LIVE (manual) |
|
||||
| 3 | **`/linkedin:newsletter`** research fan-out (`commands/newsletter.md:461-475`) | Open-web research agents (WebSearch/WebFetch) | **YES — external** | specifics-bank `ekstern` bindings (`scripts/specifics-bank/src/kilder.ts:51-52`) + `NN-kilder.md` + `queue.json` | Edition prose + sources ledger; queue readers | LIVE |
|
||||
| 4 | **`/linkedin:react`, `:post`, `:pipeline`, `:batch`** URL ingest (`commands/react.md:52`, `post.md:62`, `pipeline.md:13`, `batch.md:12`) | External URL (news, blog, YouTube, social threads) via WebFetch | **YES — external** | react/post: clipboard + state `## Recent Posts` **metadata only** (raw content not persisted); pipeline/batch: draft files + `queue.json` | state `## Recent Posts` re-injected (`user-prompt-context.mjs:109-117`, `session-start.mjs:341`); queue readers | LIVE |
|
||||
| 5 | **`/linkedin:import`** CSV (`scripts/analytics/src/parsers/csv-parser.ts`) | User's own LinkedIn analytics CSV export | First-party origin; **container-layer** parse surface | `analytics/posts/*.json` (`storage.ts:143-160`) | `report`/`audit`/`analyze`, `analytics-interpreter`, `brain assemble` | LIVE |
|
||||
| 6 | **`/linkedin:competitive`, `:outreach`** (`commands/competitive.md:33`, `outreach.md:223`) | WebSearch of competitor/partner/event content | YES — external | **Nothing durable** (inline report only) | — (no persist) | LIVE |
|
||||
| 7 | Received **third-party OKF bundle** | External bundle | YES — external | *does not exist* | — | **FUTURE / not built** |
|
||||
| 8 | `setup`, `onboarding`, `first-post`, `quick`, `specifics-bank`, voice-samples | User's own typed/pasted content | No — first-party | state file / voice-samples / specifics-bank | content commands | LIVE |
|
||||
|
||||
## 4. Where the guard applies — and where it deliberately does not
|
||||
|
||||
**Applies (wire here):** the untrusted boundaries — rows 1–5. Trust follows the data's *origin*, not the
|
||||
insertion channel (brief §7): a manual paste of an external article (`/linkedin:react`) is still external.
|
||||
|
||||
**Does not apply (out of scope by design):**
|
||||
- **First-party authoring** (row 8): onboarding, `setup`, typed post ideas, `specifics-bank` (human-only),
|
||||
voice-samples, the user's own profile edits. The guard's threat model does not target trusted-author
|
||||
in-place edits.
|
||||
- **Fetch-but-no-persist** (row 6): `competitive`/`outreach` fetch external content but write nothing
|
||||
durable — there is no downstream-trusted store to poison, so the *write-time* guard has no seam. (Their
|
||||
risk is query-time, a different tool's job.)
|
||||
- **Row 2 today** is first-party by origin (the user's own posts). It becomes an untrusted boundary the
|
||||
moment SB-S4 (the EU/EEA DMA connector) or any received-bundle path automates the inbox — see §9.
|
||||
|
||||
## 5. Integration points (persist gates) + minimal wiring
|
||||
|
||||
**Key structural finding:** every external fetch in this plugin goes through the *model's* WebFetch/WebSearch/
|
||||
MCP tools — there is **no raw-HTTP `your_model()` seam in plugin code**. The fetch → transform → persist
|
||||
pipeline is: *model tool call → agent reasoning → deterministic CLI write*. So the guard's classic two-bookend
|
||||
model (`prepare_input` → `your_model` → `screen_output`) only **half-maps**: `screen_output`
|
||||
(scan-before-persist) wires in cleanly at the deterministic CLI write points; `prepare_input` (sanitize+fence
|
||||
before the model) has no clean code seam because the fetch and transform happen *inside the model's turn*
|
||||
(§6).
|
||||
|
||||
**Coverage gap to know:** `hooks/scripts/content-gatekeeper.mjs` is the only `PreToolUse(Write|Edit)`
|
||||
choke-point, but it inspects the file **path, never the content bytes**, and only fires for the **Write/Edit
|
||||
tool** on drafts/assets paths. The four durable trusted-context stores below are all written via
|
||||
**Bash-invoked `node`/`tsx`**, so they **bypass the gate entirely**. A byte-level guard must wire at the CLI
|
||||
write points, not solely at `content-gatekeeper`.
|
||||
|
||||
Ranked by automated-reinjection risk (highest first):
|
||||
|
||||
### 5.1 trends `capture` — the one live automated poison→reinject loop *(priority 1)*
|
||||
- **Seam:** before `store.ts` persists to `trends/trends.json`, in `scripts/trends/src/cli.ts` `capture`.
|
||||
- **Wiring:** `screen_output(item.title + "\n" + item.summary, PRESET_USER_UPLOAD)`; on
|
||||
`Disposition.FAIL_SECURE`, route the item into the CLI's existing `errors[]` channel
|
||||
(`cli.ts:294-320`) instead of persisting — a drop, not a crash.
|
||||
- **Why first:** external content, stored verbatim, auto-surfaced back into future model context by
|
||||
`session-start.mjs` with no human gate. This is the sharpest write→trusted-read loop in the plugin.
|
||||
|
||||
### 5.2 brain `ingest` / `scanInbox` — the voice/profile gold seam *(priority 2)*
|
||||
- **Seam:** `writePublished(rec)` in `scripts/brain/src/ingest.ts:172-192`; especially the `scanInbox`
|
||||
(`:247`) path that reads user-dropped `ingest/inbox/*.md`.
|
||||
- **Wiring:** `screen_output(body, ...)` before write; `FAIL_SECURE` → do not promote to
|
||||
`ingest/published/`, log to a rejects sidecar. Once a bundle-shaped receive lands, switch to
|
||||
`okf.import_bundle` (§8).
|
||||
- **Why:** `ingest/published/` is the ranked-#1 gold source for `voice-trainer` and feeds `profile.md` via
|
||||
consolidation — the highest-trust downstream read in the plugin. Note the existing `provenance=published`
|
||||
guard here is an **anti-model-collapse** control (authorship axis), **not** an anti-injection control
|
||||
(origin axis) — the two are orthogonal; this seam has the former, not the latter.
|
||||
|
||||
### 5.3 newsletter research → specifics-bank `ekstern` bindings *(priority 3)*
|
||||
- **Seam:** before an external research finding persists as an `ekstern` binding
|
||||
(`scripts/specifics-bank/src/kilder.ts:51-52`).
|
||||
- **Wiring:** `screen_output` on the claim text + source; `FAIL_SECURE` → quarantine, surface to the operator
|
||||
in the fact-check sweep the newsletter pipeline already runs.
|
||||
|
||||
### 5.4 analytics CSV import — the container-layer gate *(priority 4)*
|
||||
- **Seam:** `saveBatch` in `scripts/analytics/src/utils/storage.ts:143-160` (or in `parseLinkedInCSV`).
|
||||
- **Wiring:** row-content scan (CSV formula-injection `= + - @`, active-content) complementing the existing
|
||||
**filename-only** sanitization (`sanitizeDate/Id/Week/Month` + `verifyPathWithinDirectory`,
|
||||
`storage.ts:110-137`). Lower priority: origin is first-party and the JSON is never re-emitted to a
|
||||
spreadsheet, so the practical blast radius is small — but there is currently **no** row-content
|
||||
sanitization layer, so it is a real (if narrow) gap.
|
||||
|
||||
## 6. The `prepare_input` caveat (honest scope of what we can wire)
|
||||
|
||||
The input-side bookend (`sanitize` + fence before the model, contract steps 1–2) has **no clean code seam**
|
||||
here: the untrusted content is fetched by the model's WebFetch tool and transformed by the drafting agent
|
||||
*within the same model turn*, so plugin code never holds the raw input to wrap. Partial mitigations exist —
|
||||
a sanitize pass on fetched text inside the URL commands' prompts, or reading `tool_input.content` in
|
||||
`content-gatekeeper` for the Write-tool draft path — but neither is the clean `prepare_input(untrusted)`
|
||||
call the library assumes. What we **can** wire cleanly and fully is `screen_output` at the persist gates (§5).
|
||||
|
||||
Partly, the contract's *real* security already holds structurally: the drafting agents largely reason over
|
||||
fetched text (close to "tool-less transform"), and the persist step is a deterministic CLI ("output as data,
|
||||
parsed to a schema"). The lexicon/entropy scan is defense-in-depth on top of that, not the wall.
|
||||
|
||||
## 7. Python ↔ Node interop — the real integration cost
|
||||
|
||||
The guard is **Python** (stdlib, 3.10+). This plugin's runtime is **Node ESM `.mjs` hooks (deliberately
|
||||
zero-npm-dep) + TypeScript engine via `tsx`**. The only Python in the repo is one **build-time** script
|
||||
(`hooks/scripts/compile-hooks.py`), never on a data path. So wiring the guard means crossing a subprocess
|
||||
boundary. Options:
|
||||
|
||||
| Option | Shape | Trade-off |
|
||||
|--------|-------|-----------|
|
||||
| **(a) `spawnSync('python3', …)` at the engine CLI write points** *(recommended)* | The `scripts/{trends,brain,specifics-bank,analytics}` CLIs (already `tsx`, not the zero-dep hook hot path) shell out to a `python -m llm_ingestion_guard` scan | Adds a `python3` + `pip install llm-ingestion-guard` runtime dependency to the *engine layer only*; keeps the zero-dep Node **hooks** untouched. Cleanest fit. |
|
||||
| (b) Port minimal `sanitize` + `scan_output` to a `.mjs` twin | Reimplement in Node | Defeats the point of adopting a *maintained* guard; the coverage matrix (126/126) would not apply to the port. Rejected unless a hard no-Python constraint appears. |
|
||||
| (c) Bash step inside the command, not the hook | Command invokes the scan before the CLI write | Non-deterministic (depends on the agent running the step); weaker than a code-enforced gate. |
|
||||
|
||||
**Blocker to resolve first:** whether a `python3` + one-package runtime dependency is acceptable given the
|
||||
plugin's zero-dep design value. This is the gate on any wiring work.
|
||||
|
||||
## 8. OKF `import_bundle` — future / conditional
|
||||
|
||||
The plugin is **export-only** toward OKF: its brain *emits* OKF-compatible form (`type:` + per-level
|
||||
`index.md` + root `okf_version`, landed 2026-06-26, `docs/okf-convergence-brief.md`). There is **no
|
||||
`import_bundle` and no third-party-bundle receive path** anywhere in the repo, and inbox auto-classify/convert
|
||||
is explicitly deferred (brief §11). So the guard's `okf.import_bundle(bundle, origin=EXTERNAL,
|
||||
channel=AUTOMATIC)` adapter has **no seam today**.
|
||||
|
||||
It becomes relevant if/when either lands: **(i)** SB-S4 — the EU/EEA DMA portability connector auto-feeding
|
||||
`ingest/inbox/`; or **(ii)** a cross-plugin shared retrieval skill (a *separate standalone plugin*, per the
|
||||
convergence brief §8) that merges *other* plugins' brains. At that point, wire `okf.import_bundle` at
|
||||
`scanInbox` with `allow_reserved` chosen per channel (received bundle → `True`; materialised individual
|
||||
uploads → `False`, per brief §4).
|
||||
|
||||
## 9. When — roadmap triggers
|
||||
|
||||
1. **Now:** `planned`. No wiring. (This doc + the STATE.md marker.)
|
||||
2. **First wiring candidate — trends `screen_output` (§5.1):** the trends→session-reinjection loop is already
|
||||
live, so per the brief's "include it before the first untrusted ingest path goes live," this is the
|
||||
earliest concrete target once §7 is resolved.
|
||||
3. **Hard trigger (not optional) — before SB-S4 or any received-bundle path:** the EU/EEA DMA connector, or a
|
||||
cross-plugin shared skill, turns `ingest/inbox/` from "user's own manual paste" into an automated
|
||||
external-origin ingress. Wire §5.2 (and §8's `import_bundle`) **before** that path goes live — this is the
|
||||
brief's "when, not if" moment.
|
||||
4. **Opportunistic:** §5.3 (newsletter) and §5.4 (CSV container-layer) can ride whichever hardening session
|
||||
touches those CLIs.
|
||||
|
||||
Sequencing note: none of this is on the current `docs/plan-2026-07/` roadmap (N1–N32). It is a new,
|
||||
security-scoped work item to slot in after §7 is decided — most naturally as its own hardening slice, not by
|
||||
displacing the agreed N-plan.
|
||||
|
||||
## 10. Honest limitations (carried from brief §8 — a green scan is not "safe")
|
||||
|
||||
- **Semantic / factual poisoning is invisible** to lexicon + entropy — a plausible-but-wrong concept (wrong
|
||||
metric, wrong runbook step) carries no suspicious token and passes clean. **Highest impact for a
|
||||
second-brain.** Our existing anti-sycophancy / evidence-threshold / keep-both-timestamped stance in the
|
||||
consolidation loop is the human-in-the-loop mitigation; the deterministic guard does not judge semantics.
|
||||
- **Dormant / broken-link injection:** a link to a not-yet-existing target passes a write-time scan; payload
|
||||
planted later. Relevant to the brain's cross-links.
|
||||
- **A document that *describes* attacks is a false positive** — security notes documenting injection payloads
|
||||
trip carrier-strip. Matters if the plugin ever ingests security content.
|
||||
- **Text-only, extracted-text-only** — no file parsing in the core; extract text first, scan with high-untrust
|
||||
upload provenance.
|
||||
|
||||
## 11. Existing write-time precedents the guard complements (not replaces)
|
||||
|
||||
The plugin already has three narrow write-gate defenses; the guard generalizes the class rather than
|
||||
duplicating them:
|
||||
- **`state-updater.mjs`** uses a replacement *function* (not string) on every section-append to neutralize
|
||||
`$&`/`` $` ``/`$'`/`$$`/`$n` from `$`-bearing user topics (`:14-25,:117-125`) — defends the regex mechanics,
|
||||
not content semantics.
|
||||
- **`analytics/storage.ts`** sanitizes filenames + `verifyPathWithinDirectory` (`:110-137`) — path-traversal
|
||||
defense, not row content.
|
||||
- **brain `provenance=published`** — model-collapse guard (authorship), orthogonal to injection (origin).
|
||||
|
||||
## 12. Out of scope for this plan
|
||||
|
||||
No guard code, no `pip install`, no CLI wiring, no hook changes. This is the *when/where* map and the
|
||||
dependency registration only, per the adoption brief. Implementation is a separate, operator-approved work
|
||||
item gated on §7.
|
||||
Loading…
Add table
Add a link
Reference in a new issue