feat(conformance): ship the six active-content cases; the id space already existed
The corpus goes 83 -> 89 and scope_covered gains signatures/active-content.json.
The blocker in STATE dissolved under measurement, the same way last session's
13-pattern one did. "An id space for carriers/active-content/secret-egress" was one
question in name only; the three tables have three unrelated problems:
- active-content needed NO id space invented. label_format ("active:{class}") and the
constructs keys were already extracted verbatim from the seed runtime, and their
concatenation IS what it emits - verified by comparing the six keys to the six class
strings at its _flag call sites. What blocked these cases was never naming; it was
spec section 1, fixed in the parent commit.
- carriers has no adoptable id space AND an entry-point dependence underneath it.
- secret-egress is not an id question at all: the two runtimes carry DIFFERENT tables.
18 entries here against the guard's 25, cut at different granularities - this file's
single `GitHub Token` is four ids there, `Private Key PEM Block` three, `Database
connection string` four - with membership diverging both ways. `aws-access-key-id`
is the one clean 1:1, which is why exactly one egress case was ever offered. That
number was a symptom, not modesty.
Both blockers are now recorded under scope_planned.blockers, measured, replacing a
blanket "no runtime has agreed to an id space" that was wrong for both.
Generated from measurement, not written. Payloads were extracted from the seed
runtime's coverage.py by AST - evaluating each _scan_case argument in that module's
own namespace rather than retyping detection data - then run through its public
output gate, the same entry point the 83 lexicon cases used. The fixtures were then
re-read from disk by a separate checker that re-computed every digest, re-scanned the
bytes and applied exact-within-scope independently of the generator: 6 cases, 0
failed checks.
Six built from seven offered. The runtime's matrix drives two payloads at
`active:markdown-image`; measured, their in-scope finding sets are identical, and the
second's only distinguishing signal (entropy:base64-blob) falls outside every table
this repository publishes. Dropped rather than given a discriminated case id, and
named under omitted_payloads so the count reads as a decision.
These six prove LESS than the 83, and the manifest says so: their payloads come from
the only runtime implementing the table, so no second implementation's agreement
could be measured. They pin one runtime's behaviour as a contract a future
implementer can be held to - less than cross-runtime agreement, more than nothing.
llm-security's absence of the table is measured at b0de0ca, not assumed: a tree-wide
search finds no implementation, and `git log -S` over --all returns zero commits,
closing the "it was there once" reading. Absent table is not absent capability -
their entropy scanner reaches markdown-image URLs by another route - and the manifest
says that too.
Provenance and measurement for the active-content half are kept in their own blocks:
a different source structure at a different commit, and one pin must not stand for
two measurements. The guard's HEAD moved twice during the work (3c56d50 -> de09711 ->
398eb74); measurement ran at de09711 and the drift is recorded, including that
active_content.py is NOT identical to the 0bf0729 the data file pins - the change
adds a scan-cap self-safety finding and touches no construct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVouC9nsfrfV5jRSejxbvQ
This commit is contained in:
parent
807c0d45d8
commit
bdcb1f1080
16 changed files with 367 additions and 20 deletions
86
CHANGELOG.md
86
CHANGELOG.md
|
|
@ -41,11 +41,97 @@ case ids, disposition semantics). Each JSON file additionally carries its own
|
|||
incorrect half past the check — which is precisely the defect
|
||||
`severity_provenance.not_from` was written to warn about, one level up.
|
||||
|
||||
### Added
|
||||
|
||||
- **`not-applicable`, a third conformance verdict** (`spec/conformance-corpus.md` §1.1). A
|
||||
runtime now declares the set of commons data files it implements; a case whose `scope`
|
||||
names a file outside that set is `not-applicable` rather than failed. §1 alone would have
|
||||
reported an architectural difference as a defect — one seeding runtime has no
|
||||
active-content table and never will, and 7 permanent failures say nothing a reader can use.
|
||||
The verdict is fenced so it cannot become an exit: it attaches to a **table**, never to a
|
||||
case (per-case opt-out is the silent skip §1 forbids), a declared set MUST NOT be narrowed
|
||||
to convert failures into `not-applicable`, and such cases MUST still be enumerated rather
|
||||
than dropped from the denominator. §8 now states the consequence: a pass count is
|
||||
unreadable without the declared set beside it.
|
||||
|
||||
- **Six active-content conformance cases** — `active__markdown-image`, `active__markdown-link`,
|
||||
`active__reference-link`, `active__autolink`, `active__raw-html`, `active__data-uri`. The
|
||||
corpus goes 83 → 89, and `scope_covered` gains `signatures/active-content.json`.
|
||||
|
||||
Generated from measurement, not written: payload strings were extracted from the seed
|
||||
runtime's `coverage.py` by AST — evaluating each `_scan_case` argument in that module's own
|
||||
namespace rather than retyping detection data — then run through its public output gate.
|
||||
The fixtures were then re-read from disk by a separate checker that re-computed every
|
||||
digest, re-scanned the bytes and applied `exact-within-scope` independently of the
|
||||
generator, because a generator agreeing with itself proves nothing: 6 cases, 0 failed
|
||||
checks.
|
||||
|
||||
**These six prove less than the 83, and the manifest says so.** Their payloads come from
|
||||
the only runtime implementing the table, so no second implementation's agreement could be
|
||||
measured. They pin one runtime's behaviour as a contract a future implementer can be held
|
||||
to — which is less than cross-runtime agreement and more than nothing.
|
||||
|
||||
- `signatures/active-content.json` **0.1.0 → 0.2.0** — a `pattern_id_space` block. Unlike the
|
||||
lexicon's, nothing was constructed: `label_format` and the `constructs` keys were already
|
||||
extracted verbatim, and their concatenation *is* what the seed runtime emits. The block
|
||||
states an id space the file already had implicitly, and records that it is ratified by
|
||||
**one** runtime rather than two.
|
||||
|
||||
### Changed
|
||||
|
||||
- `lexicon/injection-lexicon.json` **0.5.0 → 0.5.1** — provenance metadata only; no pattern,
|
||||
id, alias, family or severity value changes.
|
||||
|
||||
- `conformance/manifest.json` **0.1.1 → 0.2.0** — the six cases, `scope_covered`,
|
||||
`count_by_scope`, separate provenance and measurement blocks for the active-content half
|
||||
(a different source structure at a different commit; one pin must not stand for two
|
||||
measurements), and `scope_planned.blockers`.
|
||||
|
||||
- **`spec/conformance-corpus.md` §4 no longer claims scoping "asks a question both can
|
||||
answer."** That held only while every case was scoped to the one table both runtimes
|
||||
implement, and stopped being true the moment a case was scoped to a single-runtime table.
|
||||
Scope narrows *what* is compared; it does not make every runtime a valid addressee. The
|
||||
superseded sentence is named in place rather than edited away.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **§4 now states that "belongs to a data file" means published there, never "shares its
|
||||
prefix."** The distinction has a live witness: the seed runtime emits `active:oversize-input`,
|
||||
a self-safety flag about its own scan cap, which carries the `active:` prefix but is no
|
||||
construct in `signatures/active-content.json`. A prefix-matching runtime would pull it into
|
||||
the comparison and fail a case over a finding the corpus makes no claim about. Recorded in
|
||||
that file under `pattern_id_space.not_every_active_label` as well.
|
||||
|
||||
- **§6 now states the derivation's cost.** `case_id` derives from `pattern_id` alone, so a
|
||||
single-finding scope holds at most one case per pattern id — there is nowhere in the name
|
||||
for a second. The seed runtime's matrix drives *two* payloads at `active:markdown-image`;
|
||||
measured, their in-scope finding sets are identical, and the second's only distinguishing
|
||||
signal (`entropy:base64-blob`) falls outside every table this repository publishes. It was
|
||||
dropped rather than given a discriminated id, which would have broken the reverse
|
||||
transform, and it is named in `conformance/manifest.json` under `omitted_payloads` so that
|
||||
6 built from 7 offered reads as a decision rather than a miscount.
|
||||
|
||||
### Measured, not shipped
|
||||
|
||||
- **The remaining four cases are blocked on two distinct unresolved questions**, now recorded
|
||||
under `scope_planned.blockers` instead of the earlier blanket "no runtime has agreed to an
|
||||
id space". That framing was wrong for both:
|
||||
|
||||
- **Carriers (3).** No adoptable id space, and a second problem underneath. The guard emits
|
||||
two *stage-coupled* labels for one carrier — `sanitize:zero-width` on input,
|
||||
`output:zero-width-present` on output, same split for bidi and unicode-tag — while
|
||||
llm-security emits prose titles. A commons id must be invented stage-neutral, which no
|
||||
other id space here required. And since `exact-within-scope` compares a finding *set*, an
|
||||
id aliasing both labels makes the verdict depend on which entry point the runtime was
|
||||
measured through — an entry-point dependence the lexicon cases do not have.
|
||||
- **Secret egress (1).** Not an id-naming question at all. The two runtimes carry
|
||||
**different tables**: 18 entries here against the guard's 25, cut at different
|
||||
granularities (this file's single `GitHub Token` is four ids there, `Private Key PEM
|
||||
Block` three, `Database connection string` four), with membership diverging both ways.
|
||||
`aws-access-key-id` is the one clean 1:1 — which is why exactly one egress case was ever
|
||||
offered. That number was a symptom, not modesty. A shared id space presupposes a table
|
||||
reconciliation nobody has done.
|
||||
|
||||
## [0.1.0] — 2026-08-10
|
||||
|
||||
Initial extraction. Runtime-neutral detection data, the finding contract, and a conformance
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue