1
0
Fork 0
llm-ingestion-pipeline-secu.../CHANGELOG.md
Kjell Tore Guttormsen 5397ba15a1 fix(security): harden 5 adversarial-review findings (M1/M2/M3 + m4/m6) via TDD
Pre-release hardening from an independent adversarial review; each fixed
test-first (failing test -> fix -> green). 214 tests pass.

- entropy (M1): decode-and-rescan now runs BEFORE false-positive suppression,
  so an SRI/media-prefixed injection blob is still decoded and lexicon-rescanned.
  Suppression gates only the entropy finding, never the decode.
- output/disposition (M3): the invisible-carrier invariant now holds on the
  persist gate. scan_output flags zero-width/BIDI presence and disposition
  treats those + lexicon:unicode-tags-present as any-tier carriers, so a carrier
  in model output fails secure even under a trusted policy.
- contract (M2): assert_credential_allowlist catches a bare <PROVIDER>_KEY
  (e.g. STRIPE_KEY) that the old regex silently missed (fail-open). Deliberately
  broad: also flags PARTITION_KEY/SORT_KEY as loud, allowlistable FPs -- fail-loud
  beats fail-silent for an isolation control.
- disposition (m6): guard runs decide inside its guarded block -> total
  fail-closed even on a malformed report.
- output (m4): egress placeholder suppression anchors word markers (example,
  todo, ...) to a word boundary, closing a fail-open where a real secret merely
  containing such a word was suppressed.

Docs: CHANGELOG Security subsection; README honest-limit for lexicon dedup (m5,
documented tradeoff, not fixed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
2026-07-05 10:45:05 +02:00

55 lines
3 KiB
Markdown

# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] — v0.1.0 (alpha)
The stdlib-only core, built test-first (TDD) per `docs/PLAN.md`.
### Added
- `report` — shared `Finding` / `Report` / `Severity` / `Source` types.
- `sanitize` — carrier stripping (zero-width, BIDI, Unicode-tag, HTML comment,
`data:`); byte-identical on clean input.
- `entropy` — Shannon / base64-like / hex-blob detection; base64 decode-and-rescan.
- `lexicon` — JSON pattern data + loader; raw/normalized/homoglyph/rot13 variants;
ReDoS-bounded, size-capped.
- `fence` — randomized per-call spotlight delimiter; attacker marker-strip.
- `neutralize` — opt-in defang of active-content output (byte-identical when clean).
- `output` — compose lexicon + entropy + decode-rescan over emitted text; secret
egress patterns (OWASP LLM02); report-only, never mutates.
- `disposition` — WARN | QUARANTINE_REVIEW | FAIL_SECURE under a source-trust
policy; compound-signal escalation; fail-**closed** when the scanner errors.
- `contract` — write-time asserters that raise: `assert_tool_less`,
`assert_credential_allowlist`, `scoped_env`.
- `grounding` — the `SourceGroundingCheck` seam for semantic poisoning (interface
only; `[judge]` implementation plugs in behind an extra).
- Top-level wiring — the `prepare_input` / `screen_output` §6 bookends plus the
full public surface; end-to-end showcase and adversarial + false-positive corpora.
### Security
Pre-release hardening from an independent adversarial review (all TDD, failing
test first):
- `entropy` — decode-and-rescan now runs **before** false-positive suppression,
so an injection blob prefixed with an SRI/media marker (to dodge the entropy
finding) is still decoded and rescanned by the lexicon. Suppression gates only
the entropy finding, never the decode.
- `output` — the invisible-carrier invariant now holds on the persist gate:
`scan_output` flags zero-width / BIDI presence (`output:zero-width-present`,
`output:bidi-present`) and `disposition` treats those plus
`lexicon:unicode-tags-present` as any-tier carriers, so a carrier in model
output fails secure even under a trusted policy.
- `contract``assert_credential_allowlist` catches a bare `<PROVIDER>_KEY`
(e.g. `STRIPE_KEY`); the previous regex silently missed it (fail-open). The
rule is deliberately broad (also flags `PARTITION_KEY`/`SORT_KEY` as loud,
allowlistable false positives) — fail-loud beats fail-silent for isolation.
- `disposition``guard` runs `decide` inside its guarded block, so a malformed
report can no longer escape the fail-closed guarantee.
- `output` — secret-egress placeholder suppression anchors word markers
(`example`, `todo`, …) to a word boundary, so a real secret that merely
*contains* such a word is no longer suppressed (fail-open egress miss closed).