open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

feat(conformance): ship the six active-content cases; the id space already existed

Browse source 
The corpus goes 83 -> 89 and scope_covered gains signatures/active-content.json.

The blocker in STATE dissolved under measurement, the same way last session's
13-pattern one did. "An id space for carriers/active-content/secret-egress" was one
question in name only; the three tables have three unrelated problems:

- active-content needed NO id space invented. label_format ("active:{class}") and the
  constructs keys were already extracted verbatim from the seed runtime, and their
  concatenation IS what it emits - verified by comparing the six keys to the six class
  strings at its _flag call sites. What blocked these cases was never naming; it was
  spec section 1, fixed in the parent commit.
- carriers has no adoptable id space AND an entry-point dependence underneath it.
- secret-egress is not an id question at all: the two runtimes carry DIFFERENT tables.
  18 entries here against the guard's 25, cut at different granularities - this file's
  single `GitHub Token` is four ids there, `Private Key PEM Block` three, `Database
  connection string` four - with membership diverging both ways. `aws-access-key-id`
  is the one clean 1:1, which is why exactly one egress case was ever offered. That
  number was a symptom, not modesty.

Both blockers are now recorded under scope_planned.blockers, measured, replacing a
blanket "no runtime has agreed to an id space" that was wrong for both.

Generated from measurement, not written. Payloads were extracted from the seed
runtime's coverage.py by AST - evaluating each _scan_case argument in that module's
own namespace rather than retyping detection data - then run through its public
output gate, the same entry point the 83 lexicon cases used. The fixtures were then
re-read from disk by a separate checker that re-computed every digest, re-scanned the
bytes and applied exact-within-scope independently of the generator: 6 cases, 0
failed checks.

Six built from seven offered. The runtime's matrix drives two payloads at
`active:markdown-image`; measured, their in-scope finding sets are identical, and the
second's only distinguishing signal (entropy:base64-blob) falls outside every table
this repository publishes. Dropped rather than given a discriminated case id, and
named under omitted_payloads so the count reads as a decision.

These six prove LESS than the 83, and the manifest says so: their payloads come from
the only runtime implementing the table, so no second implementation's agreement
could be measured. They pin one runtime's behaviour as a contract a future
implementer can be held to - less than cross-runtime agreement, more than nothing.

llm-security's absence of the table is measured at b0de0ca, not assumed: a tree-wide
search finds no implementation, and `git log -S` over --all returns zero commits,
closing the "it was there once" reading. Absent table is not absent capability -
their entropy scanner reaches markdown-image URLs by another route - and the manifest
says that too.

Provenance and measurement for the active-content half are kept in their own blocks:
a different source structure at a different commit, and one pin must not stand for
two measurements. The guard's HEAD moved twice during the work (3c56d50 -> de09711 ->
398eb74); measurement ran at de09711 and the drift is recorded, including that
active_content.py is NOT identical to the 0bf0729 the data file pins - the change
adds a scan-cap self-safety finding and touches no construct.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVouC9nsfrfV5jRSejxbvQ
This commit is contained in:
Kjell Tore Guttormsen 2026-08-10 21:15:25 +02:00
commit bdcb1f1080
16 changed files with 367 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

44
signatures/active-content.json
View file

@ -1,11 +1,53 @@
{
"version": "0.1.0",
"version": "0.2.0",
"id": "active-content",
"description": "Active content in persisted text: constructs that fetch, render or execute on their own when a renderer touches them. The EchoLeak class (CVE-2025-32711). Report-only data: no construct here is defined as something to remove.",
"owasp": "LLM05",
"detector": "active_content",
"label_format": "active:{class}",
"pattern_id_space": {
"$comment": "Stable, commons-owned identifier for each construct, so that a conformance fixture can name an active-content finding. Unlike the injection lexicon's id space, nothing was constructed here: `label_format` and the `constructs` keys were already extracted verbatim from the seed runtime, and their concatenation IS what that runtime emits. This block states an id space that the file already had implicitly rather than introducing one.",
"field": "\"active:\" + the key of the entry in `constructs`",
"ids": [
"active:autolink",
"active:data-uri",
"active:markdown-image",
"active:markdown-link",
"active:raw-html",
"active:reference-link"
],
"verified": {
"method": "The six `constructs` keys were compared to the six class strings the seed runtime passes to its finding emitter (`_flag(cls, …)` call sites, active_content.py:337-369, which build `label=f\"active:{cls}\"` at line 309). Set-equal, exact strings, no normalisation applied.",
"date": "2026-08-10",
"at_commit": "de09711",
"at_commit_note": "Named because that is where the comparison was actually run, which is NOT the commit this file's provenance pins (0bf0729). active_content.py did change over that range: `git diff 0bf0729..de09711` adds 23 lines. Measured, not assumed - the change is a self-safety length cap (a new `max_scan_chars` parameter, and an `active:oversize-input` finding when the input exceeds it). It adds no construct, removes none, and leaves every `_flag` call site and every construct pattern untouched, so the six-way comparison above holds at both commits. The drift is recorded rather than smoothed over: a reader checking this file against 0bf0729 will see a different active_content.py and is entitled to know why that does not invalidate the check."
},
"not_every_active_label": {
"$comment": "The seed runtime emits at least one `active:`-prefixed label that is NOT in this id space, and a consumer that matches by prefix rather than by membership will mis-scope it.",
"known_example": "active:oversize-input",
"what_it_is": "A self-safety flag (OWASP LLM10) raised when the input exceeds the runtime's scan cap, announcing that the tail went unread. It reports the detector's own coverage, not a construct found in the text, which is why it is not a member of this table.",
"rule": "Membership in this id space is by presence in `ids`, never by the `active:` prefix. A finding carrying that prefix but absent from `ids` does not belong to this data file, and is therefore OUT of scope for a conformance case scoped here - see spec/conformance-corpus.md section 4.",
"introduced": "Between 0bf0729 and de09711; absent at the commit this file's provenance pins."
},
"stability": "An id is a stable identifier. Changing one is a BREAKING change for any consumer matching on it, and requires a major bump of this file.",
"single_runtime": {
"$comment": "This id space is ratified by ONE runtime, not two, and that is a weaker status than the injection lexicon's - stated here rather than left to be discovered by a consumer who assumes the two files carry equal weight.",
"implemented_by": [
"llm_ingestion_guard"
],
"not_implemented_by": [
"llm_security"
],
"absence_evidence": "Measured in llm-security at b0de0ca, not assumed: a case-insensitive tree-wide search for an active-content detector, class table or emitted `active:` label returns no implementation - only prose in `knowledge/` and an entropy-scanner test that mentions the markdown-image construct. `git log -S 'active_content'` and `git log -S 'active:markdown'` over `--all` return zero commits, so the table was never present and later removed either.",
"not_a_coverage_claim": "That runtime having no active-content TABLE is not the same as it being blind to these constructs. Its entropy scanner reaches markdown-image URLs from a different angle (rule E18, host- and query-aware). What is absent is this table, which is all `not_implemented_by` asserts.",
"conformance_consequence": "Under spec/conformance-corpus.md section 1.1, a runtime that does not declare this file reports cases scoped to it as `not-applicable` - neither pass nor error."
},
"aliases": {
"$comment": "No `aliases` object is carried per construct, and the omission is deliberate rather than pending. The lexicon needs aliases because two runtimes name the same pattern differently; here the sole implementing runtime's emitted label IS the id, so an alias would be a copy of the key. A second runtime adopting this table registers its alias then - see the lexicon's aliases block for the shape."
}
},
"$comment": "Extracted verbatim, without behaviour change, from llm-ingestion-pipeline-security v0.3.4 (commit 0bf0729, 2026-08-03): src/llm_ingestion_guard/active_content.py (pattern table, pass order, URL-shape analysis, defang) and src/llm_ingestion_guard/calibration.py (per-construct severities and opacity floors). That module documents itself as the canonical home of this table with the Node port sharing its severities, which is why the guard is the seed source here rather than llm-security.",
"provenance": {