Add a single declarative manifest proving, in one place, every vulnerability
class the guard stops — and the documented gaps it does not. This is the
real-case validation gate ahead of any v1.0 freeze (v1.0 stays parked until
verified on real cases).
- src/llm_ingestion_guard/coverage.py: stdlib-only manifest (CORE_CASES) +
narrated runner. `python -m llm_ingestion_guard.coverage` prints
class -> OWASP -> expected -> observed -> verdict; exit 0 iff every caught
class is caught and every documented gap holds. 126 caught classes + 4 gaps.
Lexicon cases are generated from load_lexicon() via a payload dict, so a
pattern with no payload fails loudly at import (self-verifying).
- tests/test_coverage_matrix.py: asserts total recall, that every documented
gap still holds, and completeness (every lexicon id + every OWASP anchor has
a case). Adds the full 25-pattern LLM02 secret-egress set (fixtures assembled
from split tokens so no secret shape sits in source) and the container-layer
front-end classes (CSV formula-injection, zip-slip, zip-bomb, symlink).
- README + CHANGELOG: point to the runnable matrix.
+165 tests (357 -> 522). No core dependency added.
Session D: move every calibration constant (entropy floors 5.4/128, 5.1/64,
4.7/40 + shape floors; MAX_SCAN_CHARS; rot13-min; cognitive-load lengths
2000/2500; disposition ranks; active-content severities) into one documented
calibration.py, so a parallel Node/TS port can mirror exactly the same numbers.
Pure refactor, zero behavior change: calibration is a leaf module (imports only
report.Severity) that entropy/lexicon/disposition/active_content now source
their thresholds from. MAX_SCAN_CHARS is re-exported from lexicon so output.py
and existing callers are unaffected. The 347 pre-existing tests pass unmodified;
new test_calibration.py freezes the values and asserts each detector actually
reads its threshold from calibration (identity-checked, not a dead copy).
Output gate step 3 now runs scan_secret_egress over every decoded base64
blob's plaintext, not only scan_lexicon. A base64-wrapped credential that
formerly vanished (decode fed the lexicon, which has no secret patterns)
now surfaces as decoded:egress:* carrying the blob offset. Evidence stays
length-only, so the decoded finding never leaks the secret value.
Hex-wrapped secrets remain a documented honest-limit (entropy exposes
decoded plaintext for base64 only). README honest-limits + CLAUDE.md
Kontekst updated; 3 tests added (347 passed, was 344).
A received OKF bundle MAY legitimately carry index.md (directory listing, read
first under progressive disclosure) and log.md (update history) at any level
(spec §3.1/§6/§7). import_bundle previously hard-rejected those basenames in the
T4 path gate, so a conformant third-party bundle was over-blocked in full
(FAIL_SECURE) — and because the reject fired before scan_concept, index.md's
body (the highest-priority injection surface) was never scanned.
import_bundle now defaults allow_reserved=True: reserved basenames are scanned
as structural files (path-safety checks — traversal / absolute / backslash / .md
— still apply). The shadow-reject (an *upload* masquerading as index.md) is
preserved: the front-end passes allow_reserved=False so a materialized upload
landing on a reserved basename is still refused. That front-end opt-in was
required to keep the shadow-reject once the default flipped (not in the plan's
Filer set; traced from the code).
- okf.py: validate_concept_path/_validate_concept/import_bundle gain the
keyword; validate_concept_path default stays False (strict standalone).
- tests: +3 (legit index/log admit; injection in index.md body caught;
okf_version frontmatter admits). Per-concept-iteration test switched to a
traversal vector; mode-b showcase's index.md surface reframed from
reserved-name-reject to index.md-body-scan.
- README honest-limits + CLAUDE.md context note the mode-b/upload distinction.
Suite: 341 -> 344 passed. Core invariant intact (dependencies=[]).
Close the EchoLeak wiring hole (CVE-2025-32711 class): markdown images/
links, reference definitions, autolinks, raw active HTML and data: URIs
now surface as report-only findings (active:*, OWASP LLM05) in
scan_output step 6, so screen_output and okf.import_bundle dispose of
them instead of admitting them with findings=[].
- new active_content.py: canonical home of the shared pattern table +
scan_active_content; neutralize refactored to import it (mutating API
and behavior unchanged, all neutralize tests pass as-is)
- images/links flagged only for absolute/protocol-relative URLs:
relative in-bundle links are legitimate wiki/OKF mechanism (principle 5)
- evidence carries defanged URLs only (hxxps://evil[.]example)
- EchoLeak vectors planted in both showcases; detach proofs cover them
- README export list + checklist step 6, CLAUDE.md context line updated
Suite: 321 -> 341 passed. Core invariant intact (dependencies=[]).
Two structural regions the office extractors missed, no new dependency:
- .docx table cells (they live outside doc.paragraphs);
- grouped .pptx shapes (add_group_shape moves a shape inside the group, so the
extractor recurses through MSO_SHAPE_TYPE.GROUP to reach it).
Each proven by an injection-in-region REJECT plus a clean-region ADMIT. Tests
310 -> 314.
The pptx payload hides where an audience watching the slides does not look. The
extractor surfaces all three regions into the concept text so the stage-2 scan
catches the injection:
- speaker notes (slide.notes_slide.notes_text_frame.text);
- off-slide (off-canvas) text boxes (shape.text_frame.text, position-agnostic);
- image/shape alt-text (cNvPr@descr, read off the XML — python-pptx 1.0.2 has no
stable public accessor across shape types).
Detach-proof: same visible slide without notes ADMITs. python-pptx imported
lazily (dev/showcase-scoped, not a core dep). Tests 305 -> 310.
The docx payload hides where a human reviewing the file in Word does not look.
The extractor surfaces all three regions into the concept text so the stage-2
scan catches the injection:
- hidden/vanish runs (w:vanish) — still runs, so paragraph.text includes them;
- review comments (doc.comments[].text);
- core metadata properties (subject/keywords/comments/title/category/author).
Detach-proof: the same visible body WITHOUT the hidden run ADMITs, so it is the
extractor surfacing the hidden region that caught it, not merely 'a docx'.
python-docx is imported lazily (dev/showcase-scoped, not a core dep). Tests
300 -> 305.
- .csv: cells leading with =/+/-/@ (leading whitespace stripped first) are the
CSV-injection/DDE vector the guard cannot recognize, so the front-end refuses
them; the raw cell text is still materialized so a prompt-injection phrase in a
cell is caught by the stage-2 scan (T1). Detach-proof: plain-cell version of
the same file ADMITs. Numeric -/+ leads are the accepted FP (honest-limits).
- folder: walked member-by-member with relative paths preserved, so a reserved
basename member (index.md) lands on the guard's T4 gate; symlinks refused.
Refactor: per-file dispatch shared by top-level drops and folder walk (strict
raises on unsupported top-level suffix, folder skips). Tests 293 -> 300.
The front-end reads zip entries in memory (never extracts to disk), so it owns
the container caps while a traversal entry maps onto the guard's path gate:
- zip-slip: a '../../evil.md' entry materializes onto a traversal concept path
(preserved verbatim, not normalized) -> stage-2 T4 -> FAIL_SECURE -> REJECT.
- zip-bomb: per-entry + per-archive uncompressed-size caps (OWASP LLM10) refuse
an oversize entry before its bytes are read; a bounded read defends a lying
header. Detach-proof: a generous cap admits the same archive, so the cap is
load-bearing.
- symlink entry: refused at the front-end (no legitimate concept meaning).
Caps are kwargs on extract_inbox/receive (small in tests, generous by default).
Tests 288 -> 293.
Realistic-upload showcase (PLAN §247), first slice. A stage-1 front-end reads
dropped files and materializes them into an OKF bundle {concept_path: text} +
provenance; receive() wires extract -> import_bundle -> verdict. This slice
covers .txt/.md (stdlib only); .zip/.csv/folder/.docx/.pptx follow.
- Front-end lives in tests/ (showcase/dev-scoped), core stays stdlib-only:
dependencies=[] untouched; python-docx/python-pptx added to the [dev] extra
(used from stage 2d/2e), never a public [extract] extra (not v1 per PLAN).
- .txt injection -> guard T1 -> REJECT; clean .txt -> ADMIT; a dropped .md keeps
its frontmatter so a dangerous value -> T2 REJECT; an index.* upload
materializes onto the reserved uploads/index.md -> T4 REJECT.
- Detach-proof: neuter extraction to an empty bundle -> the poisoned upload
ADMITs, proving the verdict depends on extraction carrying the payload.
Tests 282 -> 288.
The OKF analogue of tests/test_showcase.py: one received external bundle plants
one attack per OKF surface (T1 body + frontmatter-description, T3 non-https
resource, T4 path-traversal + reserved-name, T2 dangerous frontmatter value,
T5a dangerous-scheme link, §7.2 dangling link, homoglyph-obfuscated body) run
through the public okf surface as an upload-inbox consumer would compose it.
Aggregate fails secure -> REJECT; a clean bundle admits (WARN); the log marks
rejected concepts. Detach-proof: an always-admit gate catches none of the
surfaces, so the assertions have teeth. Stream-2 centrepiece (PLAN §212);
Stage-1 realistic upload extraction (§247) is the next increment.
Tests 277 -> 282.
Pre-release hardening from an independent adversarial review; each fixed
test-first (failing test -> fix -> green). 214 tests pass.
- entropy (M1): decode-and-rescan now runs BEFORE false-positive suppression,
so an SRI/media-prefixed injection blob is still decoded and lexicon-rescanned.
Suppression gates only the entropy finding, never the decode.
- output/disposition (M3): the invisible-carrier invariant now holds on the
persist gate. scan_output flags zero-width/BIDI presence and disposition
treats those + lexicon:unicode-tags-present as any-tier carriers, so a carrier
in model output fails secure even under a trusted policy.
- contract (M2): assert_credential_allowlist catches a bare <PROVIDER>_KEY
(e.g. STRIPE_KEY) that the old regex silently missed (fail-open). Deliberately
broad: also flags PARTITION_KEY/SORT_KEY as loud, allowlistable FPs -- fail-loud
beats fail-silent for an isolation control.
- disposition (m6): guard runs decide inside its guarded block -> total
fail-closed even on a malformed report.
- output (m4): egress placeholder suppression anchors word markers (example,
todo, ...) to a word boundary, closing a fail-open where a real secret merely
containing such a word was suppressed.
Docs: CHANGELOG Security subsection; README honest-limit for lexicon dedup (m5,
documented tradeoff, not fixed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Module 11's final deliverable (BRIEF §9, PLAN §132), built last. Exercises the
full public toolkit as a consumer would compose it — the _ingest helper doubles
as the README worked example.
- test_showcase: ONE poisoned artifact planting a vulnerability per detection
channel — 5 invisible carriers (zero-width/BIDI/unicode-tag/html-comment/
data-uri), injection across raw + homoglyph + rot13 + whole-string-base64
(decode-and-rescan) channels with DISTINCT lexicon ids so dedup won't merge
them, plus a credential the mock transform leaks into its output (LLM02). All
10 planted labels caught; disposition FAIL_SECURE; egress evidence never
carries the secret value; a clean document + clean transform WARNs (no FP).
- test_corpus: adversarial recall == 100% over the planted classes (a silent
drop is a regression), and a false-positive corpus of attack-resembling legit
content (injection vocabulary, secret-shaped placeholders/varrefs, high-entropy
checksums) that disposes WARN under a trusted source — most producing NO
finding (the suppression rules hold). One MEDIUM homoglyph case proves hard-fail
is an explicit opt-in: same finding WARNs trusted, QUARANTINEs under upload.
Secret fixtures assembled from fragments (gitleaks-safe); every payload verified
against the real detectors before assertion. 12 new tests; 201 green total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Module 11 (final) — the top-level wiring. The library never makes the model call
(no SDK imported by the core), so the public surface is the toolkit plus two
library-side bookends around the caller's tool-less transform (Form 1, chosen
with the operator over an export-only toolkit and a full orchestrator — the
bookends fit existing pipelines with least friction, encode the two halves the
library can stand for, and impose no control flow):
- prepare_input(text, source=INPUT) -> PreparedInput(fenced, nonce, report):
§6 steps 1-2, sanitize THEN fence (carrier can never smuggle a forged
delimiter). Merged report carries both steps' findings; renders no disposition.
- screen_output(text, policy, *, provenance, transform_failed) -> DispositionResult:
§6 steps 6-7, scan_output under guard() so a scanner error fails CLOSED
(FAIL_SECURE, never a silent persist). transform_failed routes the compound
forced-fallback rule.
- __all__ exports the full framework-agnostic surface: detectors, result types,
disposition machinery + presets, contract asserters, the grounding seam.
Docs: README refreshed from the stale "brief / pre-implementation" line to a v0.1
alpha status with a Form-1 quickstart, the §6 adopt-this checklist, and an honest
-limitations section (structural unsolvability at the text layer; semantic
poisoning invisible to lexicon+entropy; text-only, no multimodal). CHANGELOG
seeded; CLAUDE.md remote/status lines corrected (remote IS set, no longer
brief-stage).
10 wiring tests (public surface, prepare_input compose, screen_output fail-closed
+ compound). 189 green (showcase + corpora follow).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Module 10 of the build order — interface only in the stdlib core (PLAN §67/§94,
BRIEF §7/§10). lexicon and entropy see surface signals; they are structurally
blind to semantic poisoning — a factually false or subtly biased claim, in clean
prose, carrying no suspicious token. Catching that needs grounding against a
source of truth (embedding classifier, retrieval check, LLM judge) — a model
call, and models never live in this core.
So this module ships exactly two things and no detector logic:
- SourceGroundingCheck: the text -> Report protocol a [judge] implementation
must satisfy to plug in. runtime_checkable for a coarse callable-vs-not
isinstance gate; a conforming impl may be a plain function or a stateful
callable holding a retriever/client — the grounding source of truth is the
impl's concern, never a seam parameter. Its findings flow through disposition
like any other detector's, with zero grounding-specific plumbing.
- no_grounding_check (bound as DEFAULT_GROUNDING_CHECK): the pass-through
default returning an empty Report. The honest scope note made structural —
semantic poisoning is not solved at the text layer, and the core does not
pretend to.
Design choice, tested: the default returns NO finding (not an INFO "unchecked"
marker). An INFO finding would flip every artifact to found and trip the
quarantine_default floor in disposition, quarantining every upload for a check
that never ran. The seam stays silent; the honesty lives in the contract.
8 new tests (pass-through empties, silence on semantic poison, protocol
conformance incl. a stateful judge stub, and a composition test proving a
plugged-in check's HIGH finding fails secure through disposition unchanged);
169 green total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Module 9 of the build order — the differentiator. Where the other modules
detect and report, these are pure functions that ENFORCE an invariant and
raise: the write-time analogue of runtime least-privilege. They harden the
call a pipeline makes around its quarantined transform; the library makes no
model call itself.
Three asserts, matching the reusable-contract checklist (BRIEF §6 steps 3-4):
- assert_tool_less(request): the model request carries no tool surface —
tools/functions/tool_choice/function_call/mcp_servers, populated. Covers
both Anthropic and OpenAI request shapes; empty/None is genuinely tool-less.
- assert_credential_allowlist(env, allowed): the process env holds no
credential beyond this stage's allowlist (subset = least-privilege; the
enrichment stage sees only the model key, never the publish credential).
- scoped_env(env, allowed): the isolation primitive — strips off-allowlist
credentials, keeps PATH etc.; the assert then passes by construction.
Generalized from claude-code-llm-wiki tools/wiki_ingest/enrich.py
assert_quarantine (a pipeline-specific gate) into framework-agnostic pieces.
Two deliberate safety choices inherited from the reference: credential
detection is name-based (env VALUES are never read — value-scanning is the
output module's job); and a raised ContractViolation names only the offending
KEY, never a value, so it is safe to route to an alert channel (minimal-alert,
BRIEF §6 step 8). CREDENTIAL_NAME_RE ported verbatim from the proven reference.
23 new tests; 161 green total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Module 7 of the build order: the last gate before model output is persisted.
Composes scan_lexicon + scan_entropy over the emitted text, feeds each base64
blob entropy decoded back through the lexicon (decode-and-rescan → decoded:*
findings with blob offset), and adds the LLM02 secret/credential egress layer
(cloud/provider keys, PEM headers, DB conn-strings, JWT, labelled
password/secret/api-key assignments) with placeholder/varref FP-suppression.
Report-only (never mutates; neutralize is the separate opt-in mutator). Evidence
carries only a description + match length — never the secret value. Self-safe:
single input-size cap, linear egress patterns. PEM patterns use -{5} form so the
module itself never trips a secret scanner. 26 new tests; 108 green total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Build-order step 6. Close the EchoLeak class (CVE-2025-32711): active content in
persisted model OUTPUT that a downstream renderer auto-fetches or makes clickable,
exfiltrating data zero-click. These carriers are neither injection strings nor
high-entropy, so lexicon + entropy miss them — a distinct control (OWASP LLM05,
Improper Output Handling).
Defang, don't delete. URLs in active-content position are rewritten to a
non-resolvable but auditable form (https://evil.com -> hxxps://evil[.]com;
data:/javascript: colon neutralized to [:]); raw active HTML is escaped so a
renderer shows inert literal text. Visible information survives review; only the
machine-actionable affordance dies. Dot-defang is idempotent (never [[.]]).
Six classes, each a Finding: markdown-image (HIGH, the zero-click primitive),
inline-link (MEDIUM), reference-link definition (MEDIUM, the documented
image-filter bypass), angle-bracket autolink (MEDIUM), raw active HTML (HIGH,
inherently-active tag OR event/URL attribute — benign <b>/<em> left untouched),
standalone data: URI (HIGH). Processing order prevents double-counting.
Opt-in and separate: calling neutralize() IS the opt-in to mutate; the report-only
gate stays pure (design principles 3 & 4). Byte-identical on clean output, mirror
of the sanitizer invariant. Scope conceded in the docstring: a targeted defanger,
not a full HTML sanitizer.
17 tests: byte-identity + FP guards (lone <>[], metadata:, benign HTML), image
defang + non-resolvability, secret-exfil URL, inline/reference/autolink, raw-html
escape + script neutralization, data: URI, no double-count, counts, source.
[skip-docs]: README positioning + honest-limitations remains the deliberate
build-order step-11 deliverable (steps 1-5 likewise left README frozen).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
Build-order step 5. Wrap untrusted content in a quarantine fence a downstream
trusted prompt can rely on: everything between the delimiters is data, never
instructions.
Two load-bearing properties:
- Per-call cryptographic nonce (secrets.token_hex, 128-bit) in the delimiter, so
an attacker embedded in the payload cannot forge the matching closing marker to
break out — the nonce is unpredictable and fresh every call.
- Marker-strip FIRST: fabricated fence markers already in the payload are removed
(any/no nonce, case-insensitive, ReDoS-safe negated-class regex) and flagged as
fence:marker-injection (HIGH, LLM01) before wrapping — defense in depth against
a lucky guess of the static skeleton, and it surfaces the attempt.
Pure text -> (fenced_text, report, nonce); only mutation is the marker-strip.
Nonce exposed so the caller can reference the fence in the trusted prompt.
10 tests: wrap/preserve, per-call randomness, nonce length, breakout containment,
marker-strip (+ case-insensitive), prose-word FP guard, source, empty input.
[skip-docs]: README positioning + honest-limitations is a deliberate build-order
step-11 deliverable (steps 1-4 likewise left README frozen). README status line
("pre-implementation") is stale and flagged for the step-11 refresh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K8GmKRCdsPjWYAKWsNgeQS
Build order step 4 — the load-bearing port from the llm-security seed
(injection-patterns.mjs + string-utils.mjs), stdlib-only.
- injection_lexicon.json: 83 patterns (CRITICAL/HIGH/HYBRID/MEDIUM) as the
single source of truth (regex + id + severity + owasp + desc), compiled once
by a thin loader. Decoupled from the engine for a future TS port.
- scan_lexicon(text, source, max_scan_chars) -> Report: matches every pattern
against a deduped variant set (raw / normalized / homoglyph-folded / rot13),
plus unicode-tag presence signal and the cognitive-load trap.
- normalize_for_scan chain ported: unicode-tags -> bidi -> HTML-entities ->
unicode/hex/URL escapes -> whole-string base64 (reuses entropy.try_decode_base64)
-> collapse letter-spacing; plus fold_homoglyphs / rot13.
- Self-safety (OWASP LLM10): input-size cap (scan prefix + flag oversize) and
ReDoS-safe port — the two nested-.*? sub-agent patterns bounded to
(?:\S+\s+){0,N}?; verified true positives still fire.
- Non-Latin data (homoglyph map, BIDI block) built from explicit code points;
JSON non-ASCII kept as \uXXXX escapes.
24 tests; 55 green total.
[skip-docs]: README positioning + honest-limitations is a deliberate build-order
step-11 deliverable (steps 1-3 likewise left README frozen). README status line
("pre-implementation") is stale and flagged for the step-11 refresh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K8GmKRCdsPjWYAKWsNgeQS
Build order step 3. Pure text -> findings detector ported from the
llm-security entropy-scanner seed:
- Length-calibrated Shannon-entropy classification (CRITICAL 5.4/128,
HIGH 5.1/64, MEDIUM 4.7/40).
- Shape floor: base64-like (len>100) / hex (len>64) reach at least MEDIUM
even when entropy alone does not trigger — the only path that catches hex
(16-symbol alphabet caps H at 4.0 < 4.7).
- Decode-and-rescan (must-have): base64 blobs that decode to printable text
are exposed on EntropyResult.decoded for a later lexicon rescan.
- FP suppression scoped to the text-relevant subset (base64 media data-URI
prefixes + SRI sha*- prefix); source-code-specific seed rules omitted.
Exposes ported primitives shannon_entropy / is_base64_like / is_hex_blob /
try_decode_base64. 16 new tests; full suite 31 green. Stdlib-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K8GmKRCdsPjWYAKWsNgeQS
