1
0
Fork 0

feat(disposition): separate the assessment axis from the action

`decide` returned a `Disposition` — WARN / QUARANTINE_REVIEW / FAIL_SECURE —
which names an ACTION. But BRIEF design principle 4 says the library reports
and the pipeline decides, and disposition.py admitted the gap in its own
docstring: "It imposes no blocking of its own." So we returned an action we
cannot enforce, having discarded the judgement that produced it. A consumer
wanting different behaviour had to reinterpret the action itself — which is
why a consumer ends up pinning our GRADING: the action was all they got.

`Risk` (NONE/LOW/ELEVATED/SEVERE) now carries that judgement, and
`Policy.action_map` lets a caller map it to their own action. Both overlays
move the assessment rather than the action, so a custom map cannot silently
drop the compound escalation or the quarantine floor. `guard`'s fail-closed
path pins both axes and deliberately bypasses the map: downgrading SEVERE
means "I accept this class of finding", never "I accept a crashed scanner".

`DispositionResult.assessment` is required with no default. `Risk.NONE` is the
natural-looking default and the wrong one — a site that forgot the field would
report clean, and the axis would fail open.

MEASURED ADDITIVE, not assumed:
  - 703 -> 715 tests, no existing test changed
  - coverage matrix 128/128 recall, 6/6 documented gaps still hold
  - the PRESET_USER_UPLOAD grading table locked in 0.3.1 re-measured row by
    row: ordinary link/image/autolink/refdef -> warn on BOTH doors, unchanged

Both locked consumer promises in docs/PLAN-v1.md were checked against that
measurement and neither fires: the grading is untouched (linkedin-studio), and
the relative-target asymmetry is untouched (llm-ingestion-okf).

Scope held to disposition, per PLAN-v1.md:380. Version stays 0.4.0; the 0.5.0
bump lands in its release commit with all five version surfaces at once —
that is the fix for the defect where the v0.4.0 tag carried a 0.3.4 README.

Records limitation 32: `Severity` still carries disposition intent on the
DETECTION side, which this change does not address and cannot without moving
the grading.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-10 20:55:46 +02:00
commit de097110d2
7 changed files with 329 additions and 38 deletions

View file

@ -7,7 +7,56 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
Nothing yet. ### Added — the axis separation: assessment (`Risk`) vs action (`Disposition`)
> **Additive, and measured to be so.** Every disposition 0.4.0 rendered is
> rendered identically: the full suite went 703 → 715 with no test changed, the
> coverage matrix holds at 128/128 recall with 6/6 documented gaps, and the
> `PRESET_USER_UPLOAD` grading table locked in 0.3.1 was re-measured row by row
> and is unchanged. A caller that never reads the new field sees no difference.
`decide` and `guard` returned a `Disposition``WARN` / `QUARANTINE_REVIEW` /
`FAIL_SECURE` — which names an **action**. But BRIEF design principle 4 says the
library reports and the *pipeline* decides, and `disposition.py` admitted the
gap in its own docstring: *"It imposes no blocking of its own."* So the library
returned an action it cannot enforce, while discarding the judgement that
produced it. A consumer wanting different behaviour had to reinterpret the
action itself, which is why a consumer ends up pinning our *grading* — the
action was all they got.
- **`Risk`** — the new assessment axis: `NONE` / `LOW` / `ELEVATED` / `SEVERE`.
It answers *how dangerous is this artifact given its source context*, and is
trust-aware exactly as BRIEF §4.7 describes the domain: the same finding
genuinely is a different judgement in authored prose than in a code fence.
- **`DispositionResult.assessment`** — carries that judgement alongside the
action. The field is **required, with no default**: `Risk.NONE` would be the
natural-looking default and is the wrong one, since a construction site that
forgot it would report *clean* and the axis would fail open.
- **`Policy.action_map`** — an optional `Risk -> Disposition` override, so
"hold for review where you would block" is a policy statement rather than a
reason to pin our grading. Defaults to `None`, which means
`DEFAULT_ACTION_MAP` and keeps an untouched `Policy` hashable as before. A
partial map falls back per-level instead of raising.
- Both overlays — compound escalation and the quarantine floor — now move the
**assessment**, so a custom action map cannot silently drop them.
- The fail-closed path in `guard` pins both axes to their most severe value and
deliberately does **not** route through the action map: a policy that
downgrades `SEVERE` means "I accept this class of finding", never "I accept a
scanner that crashed on crafted input" (§4.6).
`NONE` and `LOW` both map to `WARN`, which is the point rather than an
oversight: a clean document and one carrying only low-severity findings were a
single indistinguishable value through 0.4.0.
### Known limitation recorded (32, was 31)
`Severity` still carries disposition intent on the *detection* side — the
separation above is caller-side only. Two places say so outright:
`ACTIVE_CONTENT_ORDINARY_SEVERITY = LOW` exists because grading an ordinary
external image `HIGH` fail-secured ordinary uploads, and the quarantine floor
was raised to `MEDIUM+` to repair the same regression from the other end.
Closing it changes the grading and so fires a consumer-notification promise;
deferred deliberately. See `docs/LIMITATIONS.md`.
## [0.4.0] — 2026-08-10 ## [0.4.0] — 2026-08-10

View file

@ -236,7 +236,7 @@ a green scan means safe content. The highest-impact items:
egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`), egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`),
pure beaconing, and short opaque URL segments. pure beaconing, and short opaque URL segments.
**Full list — 31 items, each with the mechanism, plus the out-of-scope boundary:** **Full list — 32 items, each with the mechanism, plus the out-of-scope boundary:**
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from [`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
consumer corpora, including the false positives the URL-shape rule actually produces. consumer corpora, including the false positives the URL-shape rule actually produces.

View file

@ -334,6 +334,25 @@ fails the test, forcing this doc to be updated:
5. **Pure beaconing** — a bare-path remote image on a hostile host is LOW (above). 5. **Pure beaconing** — a bare-path remote image on a hostile host is LOW (above).
6. **Short opaque URL segment (<24 chars)** — below what entropy can resolve (above). 6. **Short opaque URL segment (<24 chars)** — below what entropy can resolve (above).
- **`Severity` still carries disposition intent on the detection side.** The
0.5.0 axis separation split the *assessment* (`Risk`) from the *action*
(`Disposition`), but only on the caller side of the boundary. Inside the
detectors, a finding's `Severity` is still calibrated partly for the
disposition it will produce rather than purely for what was observed, and two
places in the tree say so outright: `calibration.py`'s
`ACTIVE_CONTENT_ORDINARY_SEVERITY = LOW` exists because grading an ordinary
external image `HIGH` fail-secured ordinary uploads (measured on v0.3.0), and
`disposition.py`'s quarantine floor was raised from *any finding* to *MEDIUM+*
to repair the same regression from the other end. Both fixes are correct for
the dispositions they produce; the cost is that an ordinary external image is
recorded as low-severity rather than as *a real outward-fetch capability that
is not evidence of an attack* — so no policy, however strict, can act on that
capability, because the detector already decided it did not matter. Closing
this means giving detectors a channel that says what was seen separately from
how bad it is, which changes the grading and therefore fires the
consumer-notification promise in `docs/PLAN-v1.md`. Deferred deliberately, not
overlooked.
## Out-of-scope (documented boundary) ## Out-of-scope (documented boundary)
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal

View file

@ -40,8 +40,10 @@ from .disposition import (
Policy, Policy,
Trust, Trust,
Provenance, Provenance,
Risk,
Disposition, Disposition,
DispositionResult, DispositionResult,
DEFAULT_ACTION_MAP,
PRESET_TRUSTED_SOURCE, PRESET_TRUSTED_SOURCE,
PRESET_USER_UPLOAD, PRESET_USER_UPLOAD,
) )
@ -135,9 +137,9 @@ __all__ = [
"neutralize", "NeutralizeResult", "neutralize", "NeutralizeResult",
# output-side # output-side
"scan_output", "scan_secret_egress", "scan_active_content", "scan_output", "scan_secret_egress", "scan_active_content",
# disposition # disposition — `Risk` is the assessment axis, `Disposition` the action
"decide", "guard", "Policy", "Trust", "Provenance", "decide", "guard", "Policy", "Trust", "Provenance",
"Disposition", "DispositionResult", "Risk", "Disposition", "DispositionResult", "DEFAULT_ACTION_MAP",
"PRESET_TRUSTED_SOURCE", "PRESET_USER_UPLOAD", "PRESET_TRUSTED_SOURCE", "PRESET_USER_UPLOAD",
# contract asserters # contract asserters
"assert_tool_less", "assert_credential_allowlist", "assert_tool_less", "assert_credential_allowlist",

View file

@ -90,6 +90,34 @@ DISPOSITION_RANK = {
"fail_secure": 2, "fail_secure": 2,
} }
# --- assessment: the risk axis (0.5.0 axis separation) ----------------------
# Rank of each risk level, same value-keyed convention as DISPOSITION_RANK.
# ``Risk`` answers *how dangerous is this artifact given its source context*;
# ``Disposition`` answers *what should the pipeline do*. They were one enum
# through 0.4.0, which meant a consumer wanting a different action had to
# re-derive it from the action itself — the assessment was already discarded.
RISK_RANK = {
"none": 0,
"low": 1,
"elevated": 2,
"severe": 3,
}
# The default risk -> disposition mapping, keyed by both enum *value* strings.
# This map is what keeps the separation additive: it reproduces every
# disposition 0.4.0 rendered, so a caller that ignores the new axis sees no
# change at all. A ``Policy`` may override it; ``None`` means "use this".
#
# NONE and LOW both map to ``warn`` deliberately — that collapse is precisely
# the information 0.4.0 could not express, since a clean document and one
# carrying only low-severity findings were the same single value.
DEFAULT_ACTION_MAP = {
"none": "warn",
"low": "warn",
"elevated": "quarantine_review",
"severe": "fail_secure",
}
# --- active_content: per-construct severities ------------------------------- # --- active_content: per-construct severities -------------------------------
# Zero-click auto-fetch / auto-execute constructs are HIGH; click-required ones # Zero-click auto-fetch / auto-execute constructs are HIGH; click-required ones
# are MEDIUM. Mirrors ``neutralize``'s defang classes. These are the severities # are MEDIUM. Mirrors ``neutralize``'s defang classes. These are the severities

View file

@ -23,12 +23,35 @@ from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
from enum import Enum from enum import Enum
from typing import Callable, Optional from typing import Callable, Mapping, Optional
from .calibration import DISPOSITION_RANK from .calibration import DEFAULT_ACTION_MAP as _DEFAULT_ACTION_MAP_VALUES
from .calibration import DISPOSITION_RANK, RISK_RANK
from .report import Report, Severity, severity_rank from .report import Report, Severity, severity_rank
class Risk(str, Enum):
"""How dangerous the artifact is *given its source context* — the assessment.
The other half of the 0.5.0 axis separation. :class:`Disposition` names an
**action**; ``Risk`` names the **judgement that action was derived from**.
Through 0.4.0 only the action was returned, which had two costs: a consumer
who wanted different behaviour had to reinterpret an action whose reasoning
was already gone, and ``NONE`` vs ``LOW`` a clean document versus one
carrying only low-severity findings were indistinguishable, since both
render as ``WARN``.
Risk is **trust-aware**, exactly as BRIEF §4.7 describes the domain: the
same finding genuinely *is* a different judgement in authored prose than in
a code fence, not merely a different action taken on one shared judgement.
"""
NONE = "none" # no findings at all
LOW = "low" # findings present, none dispositive in this context
ELEVATED = "elevated" # suspicious enough to hold for a human
SEVERE = "severe" # treat as a real payload
class Disposition(str, Enum): class Disposition(str, Enum):
"""The gate decision, ordered by :data:`_DISPOSITION_RANK`.""" """The gate decision, ordered by :data:`_DISPOSITION_RANK`."""
@ -54,19 +77,41 @@ class Provenance(str, Enum):
@dataclass(frozen=True) @dataclass(frozen=True)
class Policy: class Policy:
"""A named source-trust policy. See the ``PRESET_*`` constants.""" """A named source-trust policy. See the ``PRESET_*`` constants.
``action_map`` overrides how an assessed :class:`Risk` becomes a
:class:`Disposition`. ``None`` the default means
:data:`DEFAULT_ACTION_MAP`, which reproduces every disposition 0.4.0
rendered. It is deliberately ``None`` rather than the map itself so an
untouched ``Policy`` stays hashable exactly as before.
This is the seam a consumer needs: wanting *hold for review* where we
render *fail secure* is now a policy statement, not a reason to pin our
grading.
"""
trust: Trust trust: Trust
quarantine_default: bool = False # any finding -> at least QUARANTINE_REVIEW quarantine_default: bool = False # any finding -> at least QUARANTINE_REVIEW
action_map: Optional[Mapping[Risk, Disposition]] = None
@dataclass(frozen=True) @dataclass(frozen=True)
class DispositionResult: class DispositionResult:
"""The decision plus an auditable trail of which rules fired.""" """The decision plus an auditable trail of which rules fired.
``assessment`` is the risk the rules actually established; ``disposition``
is that risk mapped through the policy's action map. Consumers that gate on
the assessment are insulated from a later recalibration of the mapping.
"""
disposition: Disposition disposition: Disposition
reasons: tuple[str, ...] reasons: tuple[str, ...]
max_severity: Optional[Severity] max_severity: Optional[Severity]
assessment: Risk
# Deliberately *required*, with no default. ``Risk.NONE`` would be the
# natural-looking default and is precisely the wrong one: a construction
# site that forgot the field would report a clean assessment, so the axis
# would fail open. Fail-loud beats fail-silent for a gate (BRIEF §4.7).
# Invisible carriers have no legitimate place in a reference file: they block in # Invisible carriers have no legitimate place in a reference file: they block in
@ -83,20 +128,52 @@ _CARRIER_LABELS = frozenset({
"lexicon:unicode-tags-present", "lexicon:unicode-tags-present",
}) })
# Enum-keyed rank rebuilt from calibration's value-keyed source of truth # Enum-keyed ranks rebuilt from calibration's value-keyed source of truth
# (calibration is a leaf module and cannot import the Disposition enum without a # (calibration is a leaf module and cannot import these enums without a cycle).
# cycle). Higher = more severe. # Higher = more severe.
_DISPOSITION_RANK = {d: DISPOSITION_RANK[d.value] for d in Disposition} _DISPOSITION_RANK = {d: DISPOSITION_RANK[d.value] for d in Disposition}
_RISK_RANK = {r: RISK_RANK[r.value] for r in Risk}
DEFAULT_ACTION_MAP: Mapping[Risk, Disposition] = {
Risk(risk_value): Disposition(disposition_value)
for risk_value, disposition_value in _DEFAULT_ACTION_MAP_VALUES.items()
}
"""The default :class:`Risk` -> :class:`Disposition` mapping.
Reproduces every disposition 0.4.0 rendered, so the axis separation is additive:
a caller that never reads ``assessment`` sees no behavioural change.
"""
def _more_severe(a: Disposition, b: Disposition) -> Disposition: def _more_severe(a: Risk, b: Risk) -> Risk:
return a if _DISPOSITION_RANK[a] >= _DISPOSITION_RANK[b] else b return a if _RISK_RANK[a] >= _RISK_RANK[b] else b
def _escalate(disposition: Disposition) -> Disposition: def _escalate(risk: Risk) -> Risk:
if disposition is Disposition.WARN: """Escalate one tier on the assessment axis.
return Disposition.QUARANTINE_REVIEW
return Disposition.FAIL_SECURE ``NONE`` is unreachable here the only caller is the compound overlay,
which needs two MEDIUM+ findings and so implies at least ``LOW`` but it
escalates rather than being a no-op, so the function is total.
"""
if risk is Risk.SEVERE:
return Risk.SEVERE
return Risk(next(
r for r in Risk if _RISK_RANK[r] == _RISK_RANK[risk] + 1
))
def _action(risk: Risk, policy: Policy) -> Disposition:
"""Map an assessed ``risk`` to an action under ``policy``.
An action map that omits a risk level falls back to the default rather than
raising: a partial override is a likely way to use this, and a ``KeyError``
from inside the gate would be turned into a fail-closed by :func:`guard`
anyway silently, and with a useless reason.
"""
if policy.action_map is not None and risk in policy.action_map:
return policy.action_map[risk]
return DEFAULT_ACTION_MAP[risk]
def _carrier_label(report: Report) -> Optional[str]: def _carrier_label(report: Report) -> Optional[str]:
@ -126,21 +203,24 @@ def decide(
reasons: list[str] = [] reasons: list[str] = []
max_sev = report.max_severity() max_sev = report.max_severity()
def result(risk: Risk) -> DispositionResult:
return DispositionResult(_action(risk, policy), tuple(reasons), max_sev, risk)
# Overlay A — compound forced-fallback (§4.6): a scan hit plus a failed # Overlay A — compound forced-fallback (§4.6): a scan hit plus a failed
# transform is a probable forced-fallback attack. Overrides everything. # transform is a probable forced-fallback attack. Overrides everything.
if transform_failed and report.found: if transform_failed and report.found:
reasons.append("compound-forced-fallback: transform failed with active findings") reasons.append("compound-forced-fallback: transform failed with active findings")
return DispositionResult(Disposition.FAIL_SECURE, tuple(reasons), max_sev) return result(Risk.SEVERE)
# Any-tier exceptions (§4.7): invisible carriers and CRITICAL findings block # Any-tier exceptions (§4.7): invisible carriers and CRITICAL findings block
# regardless of trust or provenance. # regardless of trust or provenance.
carrier = _carrier_label(report) carrier = _carrier_label(report)
if carrier is not None: if carrier is not None:
reasons.append(f"any-tier: invisible carrier ({carrier})") reasons.append(f"any-tier: invisible carrier ({carrier})")
return DispositionResult(Disposition.FAIL_SECURE, tuple(reasons), max_sev) return result(Risk.SEVERE)
if max_sev is Severity.CRITICAL: if max_sev is Severity.CRITICAL:
reasons.append("any-tier: CRITICAL finding") reasons.append("any-tier: CRITICAL finding")
return DispositionResult(Disposition.FAIL_SECURE, tuple(reasons), max_sev) return result(Risk.SEVERE)
# Effective low-trust: an untrusted source, or a low-trust region within an # Effective low-trust: an untrusted source, or a low-trust region within an
# otherwise-trusted document (a code fence or a localized string). # otherwise-trusted document (a code fence or a localized string).
@ -149,38 +229,47 @@ def decide(
Provenance.LOCALIZED, Provenance.LOCALIZED,
) )
disposition = _base_disposition(report, max_sev, low_trust, policy, reasons) risk = _base_risk(report, max_sev, low_trust, policy, reasons)
# Overlay B — compound escalation (§4.6): several weaker signals escalate one # Overlay B — compound escalation (§4.6): several weaker signals escalate one
# tier even when each alone would only WARN. # tier even when each alone would only WARN. Escalating the *assessment*
# rather than the action is what keeps the overlay from being silently lost
# under a custom action map.
if _is_compound(report): if _is_compound(report):
escalated = _escalate(disposition) escalated = _escalate(risk)
if escalated is not disposition: if escalated is not risk:
reasons.append("compound: >=2 findings at MEDIUM+ -> escalated one tier") reasons.append("compound: >=2 findings at MEDIUM+ -> escalated one tier")
disposition = escalated risk = escalated
return DispositionResult(disposition, tuple(reasons), max_sev) return result(risk)
def _base_disposition( def _base_risk(
report: Report, report: Report,
max_sev: Optional[Severity], max_sev: Optional[Severity],
low_trust: bool, low_trust: bool,
policy: Policy, policy: Policy,
reasons: list[str], reasons: list[str],
) -> Disposition: ) -> Risk:
"""Assess ``report`` before the overlays, on the risk axis.
The reason strings still name the *disposition* each branch yields, because
they are an audit trail consumers already read; the axis separation must not
silently reword it. They are rendered through :func:`_action` so a policy
that remaps an action gets a trail that matches what it actually did.
"""
tier = "low" if low_trust else "high" tier = "low" if low_trust else "high"
if max_sev is None: if max_sev is None:
disposition = Disposition.WARN risk = Risk.NONE
reasons.append("clean: no findings") reasons.append("clean: no findings")
elif max_sev is Severity.HIGH: elif max_sev is Severity.HIGH:
disposition = Disposition.FAIL_SECURE if low_trust else Disposition.WARN risk = Risk.SEVERE if low_trust else Risk.LOW
reasons.append(f"HIGH under {tier}-trust -> {disposition.value}") reasons.append(f"HIGH under {tier}-trust -> {_action(risk, policy).value}")
elif max_sev is Severity.MEDIUM: elif max_sev is Severity.MEDIUM:
disposition = Disposition.QUARANTINE_REVIEW if low_trust else Disposition.WARN risk = Risk.ELEVATED if low_trust else Risk.LOW
reasons.append(f"MEDIUM under {tier}-trust -> {disposition.value}") reasons.append(f"MEDIUM under {tier}-trust -> {_action(risk, policy).value}")
else: # LOW or INFO else: # LOW or INFO
disposition = Disposition.WARN risk = Risk.LOW
reasons.append(f"{max_sev.value} -> WARN") reasons.append(f"{max_sev.value} -> WARN")
# quarantine_default floor: a finding at MEDIUM+ is held for review. # quarantine_default floor: a finding at MEDIUM+ is held for review.
@ -194,12 +283,12 @@ def _base_disposition(
if policy.quarantine_default and max_sev is not None and ( if policy.quarantine_default and max_sev is not None and (
severity_rank(max_sev) >= severity_rank(Severity.MEDIUM) severity_rank(max_sev) >= severity_rank(Severity.MEDIUM)
): ):
floored = _more_severe(disposition, Disposition.QUARANTINE_REVIEW) floored = _more_severe(risk, Risk.ELEVATED)
if floored is not disposition: if floored is not risk:
reasons.append("quarantine-floor: MEDIUM+ finding -> QUARANTINE_REVIEW") reasons.append("quarantine-floor: MEDIUM+ finding -> QUARANTINE_REVIEW")
disposition = floored risk = floored
return disposition return risk
def guard( def guard(
@ -221,10 +310,16 @@ def guard(
report = scan_fn() report = scan_fn()
return decide(report, policy, provenance=provenance, transform_failed=transform_failed) return decide(report, policy, provenance=provenance, transform_failed=transform_failed)
except Exception as exc: # noqa: BLE001 — fail closed on ANY scan/dispose error except Exception as exc: # noqa: BLE001 — fail closed on ANY scan/dispose error
# Both axes are pinned to their most severe value, and deliberately NOT
# routed through the policy's action map: an un-scannable artifact is
# not a risk judgement a caller gets to remap. A policy that downgrades
# SEVERE means "I accept this class of finding", never "I accept a
# scanner that crashed on crafted input" (BRIEF §4.6, fail closed).
return DispositionResult( return DispositionResult(
Disposition.FAIL_SECURE, Disposition.FAIL_SECURE,
(f"fail-closed: scan/dispose error: {type(exc).__name__}",), (f"fail-closed: scan/dispose error: {type(exc).__name__}",),
None, None,
Risk.SEVERE,
) )

View file

@ -5,10 +5,12 @@ import pytest
from llm_ingestion_guard.report import Finding, Report, Severity, Source from llm_ingestion_guard.report import Finding, Report, Severity, Source
from llm_ingestion_guard.disposition import ( from llm_ingestion_guard.disposition import (
DEFAULT_ACTION_MAP,
Disposition, Disposition,
DispositionResult, DispositionResult,
Policy, Policy,
Provenance, Provenance,
Risk,
Trust, Trust,
decide, decide,
guard, guard,
@ -258,3 +260,99 @@ def test_floor_and_escalation_compose_to_fail_secure():
_finding(severity=Severity.MEDIUM, label="entropy:base64", detector="entropy"), _finding(severity=Severity.MEDIUM, label="entropy:base64", detector="entropy"),
) )
assert decide(report, PRESET_USER_UPLOAD).disposition is Disposition.FAIL_SECURE assert decide(report, PRESET_USER_UPLOAD).disposition is Disposition.FAIL_SECURE
# --- 0.5.0 axis separation: assessment (how dangerous) vs disposition (what to
# --- do). PLAN-v1.md:294/:380 — change DISPOSITION, never the grading.
#
# `Disposition` is three *actions*, but BRIEF principle 4 says the library
# reports and the *pipeline* decides. `decide` therefore returned an action it
# cannot enforce, and a consumer wanting a different action had to re-derive it
# from that action — the assessment which produced it was already gone. That is
# why a consumer ends up pinning the *grading*: the action is all they get. The
# assessment axis hands them the input instead of the verdict.
def test_result_carries_an_assessment_distinct_from_the_disposition():
# The payoff: a clean report and a LOW-severity report are BOTH `WARN`
# today, and indistinguishable without re-reading the report. The assessment
# axis separates them while leaving the action identical.
clean = decide(_report(), UNTRUSTED)
low = decide(_report(_finding(severity=Severity.LOW, label="active:markdown-link")),
UNTRUSTED)
assert clean.disposition is Disposition.WARN
assert low.disposition is Disposition.WARN # action: identical
assert clean.assessment is Risk.NONE
assert low.assessment is Risk.LOW # assessment: distinct
def test_default_action_map_reproduces_todays_outcomes():
# The default mapping is a no-op by construction: this is the contract that
# keeps `PRESET_USER_UPLOAD` grading untouched (locked promise 1).
assert DEFAULT_ACTION_MAP == {
Risk.NONE: Disposition.WARN,
Risk.LOW: Disposition.WARN,
Risk.ELEVATED: Disposition.QUARANTINE_REVIEW,
Risk.SEVERE: Disposition.FAIL_SECURE,
}
@pytest.mark.parametrize("severity,policy,provenance,expected_risk", [
(Severity.CRITICAL, TRUSTED, Provenance.PROSE, Risk.SEVERE),
(Severity.HIGH, TRUSTED, Provenance.PROSE, Risk.LOW),
(Severity.HIGH, TRUSTED, Provenance.CODE_FENCE, Risk.SEVERE),
(Severity.HIGH, UNTRUSTED, Provenance.PROSE, Risk.SEVERE),
(Severity.MEDIUM, TRUSTED, Provenance.PROSE, Risk.LOW),
(Severity.MEDIUM, UNTRUSTED, Provenance.PROSE, Risk.ELEVATED),
(Severity.LOW, UNTRUSTED, Provenance.PROSE, Risk.LOW),
])
def test_assessment_tracks_danger_given_trust(severity, policy, provenance, expected_risk):
# The assessment is trust-aware, exactly as BRIEF §4.7 describes: the *same*
# hit is a different assessment in prose vs a code fence — not merely a
# different action on one shared assessment.
result = decide(_report(_finding(severity=severity)), policy, provenance=provenance)
assert result.assessment is expected_risk
assert result.disposition is DEFAULT_ACTION_MAP[expected_risk]
def test_custom_action_map_changes_the_action_not_the_assessment():
# THE point of the separation. A consumer that wants to hold for review
# rather than block says so in the policy, and the assessment it was derived
# from is unchanged — so they never have to pin our grading to get their
# behaviour.
report = _report(_finding(severity=Severity.HIGH))
strict = Policy(trust=Trust.UNTRUSTED)
lenient = Policy(trust=Trust.UNTRUSTED, action_map={
**DEFAULT_ACTION_MAP,
Risk.SEVERE: Disposition.QUARANTINE_REVIEW,
})
assert decide(report, strict).disposition is Disposition.FAIL_SECURE
assert decide(report, lenient).disposition is Disposition.QUARANTINE_REVIEW
assert decide(report, strict).assessment is Risk.SEVERE
assert decide(report, lenient).assessment is Risk.SEVERE
def test_overlays_escalate_the_assessment_not_only_the_action():
# Compound escalation and the quarantine floor are assessment-level moves;
# if they only moved the action, a custom action_map would silently drop
# them. Two MEDIUM findings untrusted: ELEVATED escalated to SEVERE.
report = _report(
_finding(severity=Severity.MEDIUM, label="lexicon:config"),
_finding(severity=Severity.MEDIUM, label="entropy:base64", detector="entropy"),
)
result = decide(report, PRESET_USER_UPLOAD)
assert result.assessment is Risk.SEVERE
assert result.disposition is Disposition.FAIL_SECURE
def test_guard_fails_closed_with_a_severe_assessment():
# The fail-closed path must not leave the assessment unset, or a consumer
# mapping on the assessment alone would read a scanner crash as clean.
def boom() -> Report:
raise RuntimeError("detector exploded")
result = guard(boom, PRESET_USER_UPLOAD)
assert result.disposition is Disposition.FAIL_SECURE
assert result.assessment is Risk.SEVERE