1
0
Fork 0

docs: version-sync + SECURITY/CONTRIBUTING + honest-limits (Session E)

- README: tests badge 275->357; status v0.1->v0.2 (repo is 0.2.0; the v1.0
  bump belongs to the Session G freeze, not this docs pass); add three
  honest-limits — lone-HIGH-in-trusted-prose->WARN, vacuous quarantine-floor,
  Cyrillic/Latin homoglyph-mix false positive.
- docs/BRIEF.md: drop "No code yet" pre-implementation framing -> implemented v0.2.
- docs/OKF-INGESTION-BRIEF.md 4: correct cross-link control language —
  absolute https / references/ targets are spec-permitted, not rejected.
- Add SECURITY.md (private Forgejo disclosure) + CONTRIBUTING.md (stdlib-only
  core, Iron-Law TDD, no trailers, Forgejo-only invariants).
This commit is contained in:
Kjell Tore Guttormsen 2026-07-15 10:08:24 +02:00
commit 1625f3893b
5 changed files with 157 additions and 5 deletions

65
CONTRIBUTING.md Normal file
View file

@ -0,0 +1,65 @@
# Contributing
Thanks for looking. This is a small, security-focused library; contributions are
welcome as long as they hold the invariants below. When in doubt, open a discussion
before a large change.
## Where the project lives
- Canonical repository: **Forgejo**`git.fromaitochitta.com/open/llm-ingestion-pipeline-security`.
- There is no GitHub repository; do not open pull requests there.
## Non-negotiable invariants
A change that breaks one of these will not be merged, however useful it is otherwise:
1. **Test-first (the Iron Law).** No production code without a failing test first.
Add the red test, then the minimal implementation that makes it green. A pure
refactor keeps the existing suite green *unchanged* as its proof of no behaviour
change.
2. **Stdlib-only core.** `pyproject.toml` `[project].dependencies` stays `[]`. The
deterministic core imports only the standard library. Extraction / ML / judge
libraries (`pypdf`, `python-docx`, `python-pptx`, `openpyxl`, ML detectors) live
**only** in an extra (`[dev]`, `[ml]`, `[judge]`), never in core `dependencies`.
The upload front-end and its parsers are dev-scoped showcase code under `tests/`.
3. **Report / mutation separation.** Detectors report findings; they never mutate
the scanned text. `neutralize` is the only defanging surface and it is opt-in and
byte-identical on clean input.
4. **Fail-secure / fail-closed stays total.** `guard()` and `screen_output` must
never turn an un-scannable or malformed input into a silent persist.
5. **Calibration lives in one place.** Tunable thresholds (entropy floors,
`MAX_SCAN_CHARS`, disposition ranks, cognitive-load lengths, severities) belong in
`src/llm_ingestion_guard/calibration.py`, so the planned Node/TS port can mirror
the exact numbers. Do not scatter magic numbers back into detectors.
6. **The shared lexicon is never split.** `injection_lexicon.json` is the single
pattern source of truth for both the Python core and the future TS port.
7. **Never ship a live payload.** Build attack strings in tests from `chr(0x…)`
fragments so the gitleaks pre-commit hook and the repo itself stay clean. Never
bypass the hook with `--no-verify`.
## Running the tests
```bash
python -m venv .venv
.venv/bin/pip install -e '.[dev]' # dev extra: extraction libs for the front-end showcase
PYTHONPATH=src .venv/bin/pytest # the full suite must be green
```
The suite is the release gate: a change is not done until the whole suite is green.
## Commit style
- **Conventional Commits:** `type(scope): description` (e.g.
`feat(egress): decode-rescan feeds base64 plaintext to secret-egress`).
- **No trailers.** Do not add `Co-Authored-By` / `Signed-off-by` / tool-attribution
trailers.
- Keep commits scoped: one logical change per commit; version bumps sync every file
that names the version in the same commit.
## Scope and honest limits
New detection is welcome, but the project ships its **limitations** as a control (see
`README.md`*Honest limitations*). If a change narrows a stated gap, update that
section. If it introduces a new deliberate boundary, document it there rather than
leaving a silent miss. Absolute claims ("catches all …", "cannot be bypassed") do not
belong in this codebase.

View file

@ -3,7 +3,7 @@
![Version](https://img.shields.io/badge/version-0.2.0-blue)
![Status](https://img.shields.io/badge/status-alpha-orange)
![Python](https://img.shields.io/badge/python-3.10%2B-purple)
![Tests](https://img.shields.io/badge/tests-275_passing-green)
![Tests](https://img.shields.io/badge/tests-357_passing-green)
![License](https://img.shields.io/badge/license-MIT-lightgrey)
A reusable, minimal, dependency-light defensive layer for **LLM ingestion
@ -27,7 +27,7 @@ poisoned artifact committed at write time is read by a *downstream* agent whose
guardrail never sees where it came from. (Nearest neighbours surveyed in
`docs/BRIEF.md` §11.)
**Status:** `v0.1`, alpha. The stdlib-only core is built and tested — ten
**Status:** `v0.2`, alpha. The stdlib-only core is built and tested — ten
detector/contract modules and the top-level wiring, exercised by an end-to-end
showcase and adversarial + false-positive corpora. The public API may still
change. There are real limitations, stated plainly below; read them.
@ -108,6 +108,22 @@ that a green scan means safe content:
bypassable in isolation; character-injection and novel phrasings evade it. The
*contract* (tool-less transform, capability isolation, fail-secure) is what
carries the security — the lexicon is defense-in-depth, not a wall.
- **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.**
Under `PRESET_TRUSTED_SOURCE`, §4.7 trust-scaling downgrades a single HIGH to
WARN (a reputable single-author source is expected to carry security
vocabulary), and one HIGH is not "compound" — compound escalation needs ≥2
findings at MEDIUM+. So a HIGH-severity injection reproduced verbatim in output
under a *trusted* policy persists with only a WARN. This is the §4.7 design, not
a miss: if your "trusted" sources can carry attacker-influenced text, run them as
untrusted (or add a quarantine floor) instead.
- **The upload preset's quarantine floor is currently vacuous.**
`PRESET_USER_UPLOAD` sets `quarantine_default` ("any finding → at least
QUARANTINE_REVIEW"), but every detector emits CRITICAL/HIGH/MEDIUM only (no
LOW/INFO), and under untrusted trust a MEDIUM already escalates to
QUARANTINE_REVIEW. The floor therefore never changes an outcome in today's
severity set — it is defensive headroom for a future LOW/INFO finding (e.g. a
grounding "unchecked" marker), not an active control. Documented so the preset's
guarantee is not over-read.
- **Semantic / factual poisoning is invisible** to lexicon + entropy: a
factually false claim in clean prose carries no suspicious token. The
`grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic
@ -132,6 +148,14 @@ that a green scan means safe content:
there is no way to distinguish "*about* an attack" from "*carrying* an attack";
such content needs a deliberate, explicitly-marked escaped path, never a silent
allow.
- **Bilingual text trips the Cyrillic/Latin homoglyph rule.**
`homoglyph:cyrillic-latin-mix` (MEDIUM) flags any Latin letter adjacent to a
Cyrillic look-alike, so genuine bilingual prose (e.g. Russian, or mixed
Norwegian/Cyrillic) trips MEDIUM → under untrusted → QUARANTINE_REVIEW — a real
false positive for an inbox that *expects* multilingual content. A calibration
fix is pending (require ≥N mixed pairs, or only flag when a folded variant also
hits another pattern); until then, multilingual corpora over-quarantine on this
rule.
- **Insider in-place edits** by a trusted author are out of the untrusted-content
threat model.
- **Text-only.** The core is `text -> findings`: it parses no files (no

62
SECURITY.md Normal file
View file

@ -0,0 +1,62 @@
# Security policy
`llm-ingestion-guard` is a defensive library for LLM ingestion pipelines. Its own
security posture matters: a flaw here can silently admit a poisoned artifact into a
downstream corpus. Reports are welcome.
## Supported versions
The project is pre-1.0 (`0.2.x`, alpha). Only the latest published version receives
fixes; there are no back-ported security branches yet. Pin a version and watch the
`CHANGELOG.md` `### Security` entries.
## Reporting a vulnerability
**Do not open a public issue for a vulnerability.** Public disclosure before a fix
gives an attacker a window against every downstream consumer.
Instead, report it **privately** to the maintainer via the canonical repository on
Forgejo:
- Repository: `git.fromaitochitta.com/open/llm-ingestion-pipeline-security`
- Contact the maintainer directly through that Forgejo instance (private message /
maintainer contact) and mark the subject `SECURITY`.
Please include:
- affected version / commit,
- a minimal reproduction (input → observed disposition/finding vs. expected),
- the impact you see (e.g. a poisoned artifact that disposes `WARN` instead of
`FAIL_SECURE`).
Obfuscate any real payloads the same way the test corpus does — build attack strings
from `chr(0x…)` fragments so the report itself does not ship a live carrier.
## What counts as a vulnerability
In scope (a real finding):
- a bypass of a **stated** control — e.g. an invisible carrier that reaches the
persist gate without failing secure, a credential that egresses without a
`decoded:egress:*` / `egress:*` label, a `guard()` path that fails *open*;
- a `prepare_input` / `screen_output` code path that raises instead of failing
closed;
- a ReDoS or unbounded-resource input against the scanner.
Out of scope (documented boundaries — see the **Honest limitations** section of
`README.md`, not vulnerabilities):
- semantic / factual poisoning invisible to lexicon + entropy;
- a HIGH finding in *trusted* prose disposing to `WARN` (§4.7 trust-scaling);
- hex-wrapped (non-base64) secret egress;
- multimodal / binary-layer carriers (OCR, font stego, VBA/macros, encrypted files);
- the multilingual homoglyph-mix false positive.
If you are unsure whether something is in scope, report it privately anyway.
## Disclosure
This is a small project without a formal embargo SLA. The maintainer will
acknowledge a report, agree a fix + disclosure timeline with the reporter, and
credit the reporter in the `CHANGELOG.md` `### Security` entry unless they prefer to
remain anonymous.

View file

@ -3,8 +3,9 @@
**A reusable, minimal, dependency-light defensive layer for LLM *ingestion*
pipelines — the write-time siblings of query-time chatbot guardrails.**
Status: brief / pre-implementation. This document defines what the repo should
contain and why. No code yet.
Status: implemented — v0.2 (alpha). This document defines what the repo contains
and why; the stdlib-only core is built and tested (see `README.md` for usage and
`docs/PLAN.md` for the build order).
---

View file

@ -57,7 +57,7 @@ Status column verified against the code at commit `5397ba1` on 2026-07-06 (§9).
| **Markdown body** | Prompt injection, hidden carriers (zero-width, BIDI, Unicode-tag, HTML comments, `data:` URIs) | Carrier-strip → fence → tool-less transform → output scan → fail-secure | ✅ Covered (core contract) |
| **YAML frontmatter** | Injection in `title`/`description`/`tags` + arbitrary unknown keys; `description` propagates into `index.md` (read **first** under progressive disclosure); YAML anchor/alias DoS + dangerous type coercion | Same sanitize/scan on frontmatter *values*; parse YAML with a safe loader | ⚠️ **Partial** — values are scanned **iff** the caller passes the whole document (frontmatter included) as text; the core **never parses YAML**, so the safe-loader is a genuinely new gate at an OKF-adapter boundary (§9) |
| **`resource` URL** | `data:`/`javascript:`/`file:`/SSRF target that a consumer or visualizer fetches | Scheme allowlist (`https` only), validate before commit | ❌ **New control**`neutralize` defangs schemes for human audit but there is **no reject-gate** and no `resource`-field concept (§9) |
| **Cross-link graph** | "Dead links are valid" → *dormant* injection: plant a link to a non-existent concept-ID now, write the poisoned target later | Constrain link targets to relative in-bundle paths + scheme check; re-scan on write of a link target | ❌ **New control** (graph level) |
| **Cross-link graph** | "Dead links are valid" → *dormant* injection: plant a link to a non-existent concept-ID now, write the poisoned target later | Resolve in-bundle `.md` links to concept-IDs; flag dangling links (the dormant-injection signal); reject dangerous-scheme / bundle-escaping targets. Absolute external `https` URLs and `references/` paths are spec-permitted link targets, not rejected. Re-scan on write of a link target | ❌ **New control** (graph level) |
| **File path / reserved names** | Concept-ID = file path minus `.md`; path traversal (`../`) and shadowing of reserved `index.md`/`log.md` | Sanitize/normalize paths; reject `..` and reserved filenames as concept names | ❌ **New control** — no path validation in the core (§9) |
| **`log.md` / provenance** | No authenticity at the format level | Stamp disposition + trust tier per concept | ↔️ **Machinery exists**`Trust` × `Provenance` × `Disposition` types are built; emission-to-`log.md` + an origin/channel stamp is new wiring on top (§9) |