1
0
Fork 0

refactor(calibration): consolidate tunable thresholds into calibration.py

Session D: move every calibration constant (entropy floors 5.4/128, 5.1/64,
4.7/40 + shape floors; MAX_SCAN_CHARS; rot13-min; cognitive-load lengths
2000/2500; disposition ranks; active-content severities) into one documented
calibration.py, so a parallel Node/TS port can mirror exactly the same numbers.

Pure refactor, zero behavior change: calibration is a leaf module (imports only
report.Severity) that entropy/lexicon/disposition/active_content now source
their thresholds from. MAX_SCAN_CHARS is re-exported from lexicon so output.py
and existing callers are unaffected. The 347 pre-existing tests pass unmodified;
new test_calibration.py freezes the values and asserts each detector actually
reads its threshold from calibration (identity-checked, not a dead copy).
This commit is contained in:
Kjell Tore Guttormsen 2026-07-15 09:44:53 +02:00
commit ee402e4ea8
6 changed files with 212 additions and 34 deletions

View file

@ -0,0 +1,77 @@
"""calibration — the one place every tunable threshold lives.
Every detector in this package is calibrated by a handful of numbers: entropy
floors, an input-size cap, minimum lengths, and two small severity/rank tables.
Scattered across four modules, those numbers are impossible to audit and the
concrete driver here impossible for a port to mirror *exactly*. A parallel
Node/TypeScript implementation of this gate must classify byte-for-byte the same
way, which means it must share the same constants. This module is that shared
contract: one documented surface the port copies verbatim.
**This module holds values, never logic.** It depends only on
:mod:`~llm_ingestion_guard.report` (for the :class:`Severity` enum used by the
active-content table) and is imported by ``entropy``, ``lexicon``,
``disposition`` and ``active_content`` a leaf in the dependency graph, so no
import cycle is possible.
Changing any number here is a deliberate recalibration, not a refactor:
``tests/test_calibration.py`` freezes these values and asserts each detector
actually sources its threshold from here.
"""
from __future__ import annotations
from .report import Severity
# --- entropy: length-calibrated Shannon-entropy tiers -----------------------
# Bits-per-char floor paired with a minimum length, because the achievable
# entropy maximum is length-dependent (a short base64 string cannot reach the
# entropy of a long one). Empirically calibrated against the seed scanner:
# plaintext prose H ~3.5-4.2; base64 len64 H ~5.2; base64 len128 H ~5.6.
ENTROPY_CRITICAL_H, ENTROPY_CRITICAL_LEN = 5.4, 128
ENTROPY_HIGH_H, ENTROPY_HIGH_LEN = 5.1, 64
ENTROPY_MEDIUM_H, ENTROPY_MEDIUM_LEN = 4.7, 40
# Shape floors: structured encodings suspicious by size even when their entropy
# sits below the classification floor. The hex floor is the *only* path that
# classifies a hex blob (a 16-symbol alphabet caps entropy at log2(16)=4.0,
# below the 4.7 MEDIUM floor).
ENTROPY_BASE64_FLOOR_LEN = 100
ENTROPY_HEX_FLOOR_LEN = 64
# --- lexicon: self-safety + variant thresholds ------------------------------
# Input-size cap (OWASP LLM10): large enough for a real ingested document;
# beyond it the scanner reads the prefix and flags, so runtime stays bounded
# even on a decompression-bomb-sized input.
MAX_SCAN_CHARS = 1_000_000
# Minimum length before the rot13 variant is scanned — shorter strings hit
# rot13-look-alike false positives.
ROT13_MIN_LEN = 40
# Cognitive-load trap: a CRITICAL pattern found *only* past the first
# COGNITIVE_LOAD_TAIL_START chars of text at least COGNITIVE_LOAD_MIN_LEN long is
# a human-in-the-loop trap (an override buried at the tail of verbose output).
COGNITIVE_LOAD_MIN_LEN = 2500
COGNITIVE_LOAD_TAIL_START = 2000
# --- disposition: gate-decision ordering ------------------------------------
# Rank of each disposition, keyed by its enum *value* string (kept primitive so
# this leaf module needs no import from ``disposition``, which would cycle).
# ``disposition`` rebuilds the enum-keyed map from this. Higher = more severe.
DISPOSITION_RANK = {
"warn": 0,
"quarantine_review": 1,
"fail_secure": 2,
}
# --- active_content: per-construct severities -------------------------------
# Zero-click auto-fetch / auto-execute constructs are HIGH; click-required ones
# are MEDIUM. Mirrors ``neutralize``'s defang classes.
ACTIVE_CONTENT_SEVERITY = {
"markdown-image": Severity.HIGH,
"markdown-link": Severity.MEDIUM,
"reference-link": Severity.MEDIUM,
"autolink": Severity.MEDIUM,
"raw-html": Severity.HIGH,
"data-uri": Severity.HIGH,
}