Compare commits
4 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c362717818 | |||
| e56812eb39 | |||
| f0a511369d | |||
| c75c546614 |
7 changed files with 202 additions and 6 deletions
129
CHANGELOG.md
129
CHANGELOG.md
|
|
@ -9,6 +9,135 @@ Versioning note: the repository tag versions **the contract** (file set, key nam
|
||||||
case ids, disposition semantics). Each JSON file additionally carries its own
|
case ids, disposition semantics). Each JSON file additionally carries its own
|
||||||
`"version"` field, bumped when that file changes.
|
`"version"` field, bumped when that file changes.
|
||||||
|
|
||||||
|
## [0.9.0] — 2026-08-13
|
||||||
|
|
||||||
|
**A normative rule stated its own premise and then applied itself beyond it.**
|
||||||
|
`spec/conformance-corpus.md` §7 justified the fixture-is-ground-truth ordering with *"**Two
|
||||||
|
implementations** that return different verdicts…"* and then stated the rule with no scope at
|
||||||
|
all. For `signatures/active-content.json` there is no second implementation — the seed runtime
|
||||||
|
authored both the payloads and the table — and that runtime has stated that the classification
|
||||||
|
behind it is calibration it does not freeze. §7 as written made a reserved change on their side
|
||||||
|
into a bug on their side.
|
||||||
|
|
||||||
|
**Breaking in category, minor in number.** This changes disposition semantics, which the
|
||||||
|
versioning note at the top of this file counts as contract. The repository is in 0.x, where a
|
||||||
|
breaking change is a minor bump by the rules — the same reading `[0.3.0]` recorded: *read the
|
||||||
|
entry, not the version number*.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **`spec/conformance-corpus.md` — new §7.1, *Where the second paragraph does not hold*.** The
|
||||||
|
scope is keyed on a **structural property**, never on a table name: a case whose scope is a
|
||||||
|
table only one runtime implements, whose payload that runtime authored. A rule naming
|
||||||
|
`active-content` would rot the day a second runtime implements it. §7's own second paragraph
|
||||||
|
already carried the premise; §7.1 makes it explicit and states the disposition for the case
|
||||||
|
the premise excludes — the fixture is not rewritten on the divergence alone, the divergence is
|
||||||
|
recorded against the version pinned, and re-pinning is a separate release. That is the
|
||||||
|
disposition §5 already applies to a stale `observed_out_of_scope` entry, extended to the one
|
||||||
|
place where it can reach a verdict.
|
||||||
|
|
||||||
|
**It creates no fourth verdict, and that constraint shaped the wording.**
|
||||||
|
`schema/conformance-declaration.schema.json` closes `result` with `additionalProperties: false`
|
||||||
|
over four counts plus two arithmetic invariants; a fifth verdict would have broken every
|
||||||
|
consumer's parser, which is a worse break than the one intended. A case whose expected findings
|
||||||
|
are not produced still **fails** and is still named in `failed_cases`. What §7.1 changes is what
|
||||||
|
the failure licenses concluding, not what is reported.
|
||||||
|
|
||||||
|
Two limits stated in the section rather than left to be inferred: it does **not** reach a
|
||||||
|
third-party implementer of the same table — against them the fixture is the contract, exactly
|
||||||
|
as §7 says, and that is the only thing these cases can prove while one runtime is all there is
|
||||||
|
— and it is **not** a licence for a runtime to self-declare, since the exemption is carried by
|
||||||
|
the corpus's provenance record for the scope and not asserted per case by whoever failed.
|
||||||
|
|
||||||
|
Superseded text is named rather than edited away, following §6's own pattern: *"Through corpus
|
||||||
|
version 0.8.1 this section stated the rule above with no scope at all."*
|
||||||
|
|
||||||
|
**The competing reading was tested and disposed of**, because it is the one that would have
|
||||||
|
avoided this release: that §7's existing hatch (*"unless the fixture itself is proven wrong"*)
|
||||||
|
already covered it. It does not. The hatch's consequence is that **the fixture changes**, and
|
||||||
|
the manifest field asserts the opposite — pinned, not rewritten, re-pinning a separate
|
||||||
|
decision. And a runtime recalibrating does not prove the earlier classification wrong: the
|
||||||
|
fixture measured `de09711` / `0.4.0` correctly, and a later release does not reach back and
|
||||||
|
falsify an earlier measurement. The case fits neither of §7's two dispositions, which is the
|
||||||
|
defect.
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` `0.6.1` → `0.6.2` —
|
||||||
|
`active_content_provenance.pins_a_version_not_a_frozen_classification` no longer records an
|
||||||
|
open question.** The retirement is **partial and it is quoted, not dropped**, per the house
|
||||||
|
style this field established one release ago (*"a correction that does not say what it corrects
|
||||||
|
cannot be audited"*). What falls is only the open-question status; the clause *"section 7 …
|
||||||
|
is NOT amended by this block"* **stays true and is kept**, because §7 was amended by its own
|
||||||
|
release and not by a data file. Value change only — read back from disk against `HEAD` with a
|
||||||
|
flattened key diff: `added: 0, removed: 0, changed: 2` (the field and `version`), and the new
|
||||||
|
string printed and read rather than inferred from the count, since a value edit reports
|
||||||
|
`changed: 1` whatever it wrote.
|
||||||
|
|
||||||
|
Six prose dashes in the new text were written `--` and promoted to `—` before commit: `--` is
|
||||||
|
the variant-suffix separator token of §6's case-id grammar, and every other occurrence of it in
|
||||||
|
this file is that token, a real case id, or a CLI flag.
|
||||||
|
|
||||||
|
### Neighbours — measured, and the ones left alone are named
|
||||||
|
|
||||||
|
A sweep for the retired premise was run over the whole repository, widened past *"ground truth"*
|
||||||
|
to the second paragraph's own wording (*"one of them has a bug"*, *"two implementations"*), since
|
||||||
|
a restatement in that phrasing would have survived the first search.
|
||||||
|
|
||||||
|
- **`CONVENTIONS.md` — changed.** Carried the rule unscoped and called the proven-wrong hatch
|
||||||
|
*"the one way that reverses"*. There are now two, and both are listed.
|
||||||
|
- **`CLAUDE.md` — changed.** The Norwegian restatement that governs sessions in this repository
|
||||||
|
carried the same unscoped rule; left alone, the next session here would have acted on it.
|
||||||
|
- **`SECURITY.md` §2 — minimal cross-reference only.** Its claim is about a fixture that expects
|
||||||
|
**too little**, and §7.1 narrows *who the rule reaches*, not that direction. The conclusion
|
||||||
|
survives intact, so it was not rewritten.
|
||||||
|
- **`SECURITY.md` "Why a confirmed defect is usually not fixed here first" — untouched.** Its
|
||||||
|
*"two implementations answering differently"* is about extracted **data** diverging from its
|
||||||
|
source, not about fixtures.
|
||||||
|
- **`README.md` — untouched.** Its conformance row says *"Ground truth"* as a descriptor and does
|
||||||
|
not restate the disagreement rule, and it already names the asymmetry it would otherwise hide:
|
||||||
|
the seven active-content cases are *"measured against the one runtime that implements that
|
||||||
|
table"*. Nothing there became false.
|
||||||
|
- **`docs/extraction-plan.md` — untouched, and it is supporting evidence rather than a stale
|
||||||
|
neighbour.** It already records that the calibration file *"inverts this repository's central
|
||||||
|
rule"* — so this is the second place the unscoped rule was known not to hold, and the first was
|
||||||
|
documented before this release.
|
||||||
|
|
||||||
|
### Not in this release
|
||||||
|
|
||||||
|
Whether the seven active-content cases still pass at the seed runtime's `v1.1.0` is **unmeasured**,
|
||||||
|
and §7.1 is silent on it. No case was minted, no data file touched, no id string proposed.
|
||||||
|
|
||||||
|
## [0.8.1] — 2026-08-13
|
||||||
|
|
||||||
|
**The field 0.8.0 added to make the exposure precise stated it with a hand-derived count, and the
|
||||||
|
count was wrong.** Caught in the same session, before any consumer read it, and corrected inside
|
||||||
|
the field rather than by rewriting it. No measurement changed and no verdict moved.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **`conformance/manifest.json` 0.6.0 → 0.6.1 —
|
||||||
|
`active_content_provenance.pins_a_version_not_a_frozen_classification` now enumerates instead of
|
||||||
|
totalling.** As published it read *"three of the four dimensions they name as calibration cannot
|
||||||
|
move one of these cases at all. The fourth can: which `active:` ids a payload yields IS the
|
||||||
|
classification"*. Two defects in one sentence. First, the total was derived by hand over a
|
||||||
|
taxonomy the field had itself recategorized: the seed runtime's four calibration dimensions are
|
||||||
|
severities, thresholds, **lexicon entries** and dispositions, and lexicon entries are *not* absent
|
||||||
|
from these fixtures — `active__data-uri` carries `data-uri:executable` and `active__raw-html`
|
||||||
|
carries `hybrid-xss:script-tag` in `observed_out_of_scope`, both verified as members of
|
||||||
|
`lexicon/injection-lexicon.json` and non-members of `signatures/active-content.json`. Second,
|
||||||
|
*"the fourth"* silently substituted the classification for lexicon entries as the fourth item of
|
||||||
|
their sentence, which it is not — the classification is what they addressed separately.
|
||||||
|
|
||||||
|
- The replacement names three things and totals none of them: severities/thresholds/dispositions
|
||||||
|
are absent and move no verdict; lexicon entries move no verdict either — spec section 5 forbids
|
||||||
|
failing a runtime over `observed_out_of_scope` — but a lexicon calibration change **ages** those
|
||||||
|
two entries as evidence, which is the exposure
|
||||||
|
`active_content_measurement_0_7_0.movement_sweep.residue_is_the_field_no_test_protects` already
|
||||||
|
names as a class, and this corpus pins a stale residue entry rather than rewriting it; and the
|
||||||
|
active-content classification is the one thing that can move a verdict. The retired sentence is
|
||||||
|
**quoted** in the field's `AMENDED IN 0.6.1` clause, not merely dropped, for the same reason
|
||||||
|
`scope_planned.$comment` quotes what it retired: a correction that does not say what it corrects
|
||||||
|
cannot be audited.
|
||||||
|
|
||||||
## [0.8.0] — 2026-08-13
|
## [0.8.0] — 2026-08-13
|
||||||
|
|
||||||
**The seven active-content fixtures pin a VERSION of the seed runtime, and nothing said so.**
|
**The seven active-content fixtures pin a VERSION of the seed runtime, and nothing said so.**
|
||||||
|
|
|
||||||
|
|
@ -65,6 +65,13 @@ Ingen. Data + prosa. Filformater: JSON (data + schema), Markdown (spec), rå tek
|
||||||
- `expected.json` er ground truth. Er en runtime uenig med `expected.json`, er runtimen
|
- `expected.json` er ground truth. Er en runtime uenig med `expected.json`, er runtimen
|
||||||
feil — med mindre fixturen selv bevises feil, og da endres fixturen i eget commit med
|
feil — med mindre fixturen selv bevises feil, og da endres fixturen i eget commit med
|
||||||
begrunnelse.
|
begrunnelse.
|
||||||
|
- **Regelen over er skopet, og skopet er bærende.** Er casens scope en tabell bare ÉN runtime
|
||||||
|
implementerer, og den runtimen skrev payloaden, finnes ikke den andre implementasjonen
|
||||||
|
regelen dømmer mellom. Da er en divergens fra *den* runtimen verken en bevist feil fixture
|
||||||
|
eller nødvendigvis deres bug: fixturen skrives ikke om på divergensen alene, den føres mot
|
||||||
|
versjonen som er pinnet, og re-pinning er en egen release. Mot en TREDJEPARTS-implementasjon
|
||||||
|
av samme tabell gjelder §7 uendret. Til og med `v0.8.1` sto regelen uskopet. Se
|
||||||
|
`spec/conformance-corpus.md` §7.1.
|
||||||
- **En case er ikke mintbar uten inngangspunkt for sitt scope.** Korpuset pinner ikke
|
- **En case er ikke mintbar uten inngangspunkt for sitt scope.** Korpuset pinner ikke
|
||||||
lenger ett inngangspunkt per runtime for alt — `manifest.json` →
|
lenger ett inngangspunkt per runtime for alt — `manifest.json` →
|
||||||
`entry_points_by_scope` bærer inngangspunkt, **findings-accessor** og
|
`entry_points_by_scope` bærer inngangspunkt, **findings-accessor** og
|
||||||
|
|
|
||||||
|
|
@ -107,10 +107,16 @@ this document:
|
||||||
- `<case-id>` is stable and descriptive. **Changing a case id is a breaking change** — a
|
- `<case-id>` is stable and descriptive. **Changing a case id is a breaking change** — a
|
||||||
published conformance result names it.
|
published conformance result names it.
|
||||||
- `expected.json` is **ground truth**. If a runtime disagrees with it, the runtime is wrong.
|
- `expected.json` is **ground truth**. If a runtime disagrees with it, the runtime is wrong.
|
||||||
- The one way that reverses: the fixture is proven wrong. Then the fixture changes **in its own
|
- One way that reverses: the fixture is proven wrong. Then the fixture changes **in its own
|
||||||
commit, with the reason written down** — never folded into a change that does something else,
|
commit, with the reason written down** — never folded into a change that does something else,
|
||||||
because a fixture edit is the one edit that can make every conforming runtime wrong
|
because a fixture edit is the one edit that can make every conforming runtime wrong
|
||||||
identically.
|
identically.
|
||||||
|
- The other, added in `v0.9.0`: where a case's scope is a table only one runtime implements and
|
||||||
|
that runtime authored the payload, a divergence by **that** runtime is neither a proven-wrong
|
||||||
|
fixture nor necessarily its bug. The fixture is not rewritten on the divergence alone — it is
|
||||||
|
recorded against the version pinned, and re-pinning is a separate release. Through `v0.8.1`
|
||||||
|
this list carried only the first way. See
|
||||||
|
[`spec/conformance-corpus.md` §7.1](spec/conformance-corpus.md).
|
||||||
- A case declares the data files it is `scope`d to. A runtime that does not implement a scoped
|
- A case declares the data files it is `scope`d to. A runtime that does not implement a scoped
|
||||||
table reports the case `not-applicable` — a third verdict beside pass and fail, and one that
|
table reports the case `not-applicable` — a third verdict beside pass and fail, and one that
|
||||||
must be reported rather than dropped from the denominator. See
|
must be reported rather than dropped from the denominator. See
|
||||||
|
|
|
||||||
12
README.md
12
README.md
|
|
@ -16,6 +16,18 @@ unicode-carrier smuggling or active content in untrusted text, on any runtime.
|
||||||
|
|
||||||
**It holds no runnable code.** Data, specifications and fixtures only.
|
**It holds no runnable code.** Data, specifications and fixtures only.
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
- [Install](#install)
|
||||||
|
- [Requirements](#requirements)
|
||||||
|
- [What it does](#what-it-does)
|
||||||
|
- [Non-goals](#non-goals)
|
||||||
|
- [Known limitations](#known-limitations)
|
||||||
|
- [Contributing](#contributing)
|
||||||
|
- [Reporting a wrong entry](#reporting-a-wrong-entry)
|
||||||
|
- [Changelog](#changelog)
|
||||||
|
- [License](#license)
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
Nothing to install — this repository is **vendored into consumers**, not installed.
|
Nothing to install — this repository is **vendored into consumers**, not installed.
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ detector, and that is a working bypass against every consumer until it is closed
|
||||||
|
|
||||||
Report privately by email:
|
Report privately by email:
|
||||||
|
|
||||||
- **hello@fromaitochitta.com**, with `SECURITY` at the start of the subject.
|
- **security@fromaitochitta.com**, with `SECURITY` at the start of the subject.
|
||||||
|
|
||||||
Pull requests are not the channel either — they are switched off on the canonical
|
Pull requests are not the channel either — they are switched off on the canonical
|
||||||
repository, and not as an oversight. This repository is vendored into independent runtimes
|
repository, and not as an oversight. This repository is vendored into independent runtimes
|
||||||
|
|
@ -45,8 +45,9 @@ In scope — all of these are real reports:
|
||||||
escaping is wrong for the declared dialect, missing or wrong flags, a pattern that fails
|
escaping is wrong for the declared dialect, missing or wrong flags, a pattern that fails
|
||||||
to compile in a documented engine and gets skipped rather than reported.
|
to compile in a documented engine and gets skipped rather than reported.
|
||||||
2. **A conformance fixture that sanctions a miss.** `expected.json` is ground truth: a
|
2. **A conformance fixture that sanctions a miss.** `expected.json` is ground truth: a
|
||||||
runtime that disagrees with it is deemed wrong. A fixture that expects too little makes
|
runtime that disagrees with it is deemed wrong (as scoped by `spec/conformance-corpus.md`
|
||||||
every conforming runtime wrong identically, and the corpus will not catch it.
|
§7.1, which narrows who that reaches and not this direction). A fixture that expects too
|
||||||
|
little makes every conforming runtime wrong identically, and the corpus will not catch it.
|
||||||
3. **A normative clause that mandates unsafe behaviour.** The `spec/` files bind the
|
3. **A normative clause that mandates unsafe behaviour.** The `spec/` files bind the
|
||||||
implementations that consume them, so a weak rule propagates to all of them.
|
implementations that consume them, so a weak rule propagates to all of them.
|
||||||
4. **A real secret or personal data in the repository or its history.** The history is
|
4. **A real secret or personal data in the repository or its history.** The history is
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -284,6 +284,47 @@ This ordering is the whole point of the repository. Two implementations that ret
|
||||||
different verdicts on the same input are not holding different opinions; one of them has a
|
different verdicts on the same input are not holding different opinions; one of them has a
|
||||||
bug.
|
bug.
|
||||||
|
|
||||||
|
### 7.1 Where the second paragraph does not hold
|
||||||
|
|
||||||
|
**Through corpus version 0.8.1 this section stated the rule above with no scope at all**, and
|
||||||
|
the scope was load-bearing: the justification names *two* implementations. Where a case's
|
||||||
|
scope is a table only one runtime implements, and that runtime authored the payload the case
|
||||||
|
was extracted from, there is no second implementation whose disagreement the paragraph could
|
||||||
|
adjudicate. Which cases those are is recorded in the corpus, not asserted per run — see
|
||||||
|
`active_content_provenance.asymmetry` in
|
||||||
|
[`conformance/manifest.json`](../conformance/manifest.json).
|
||||||
|
|
||||||
|
For such a case, a disagreement by the **seed runtime itself** is a third thing, and it is
|
||||||
|
neither of the two the paragraph offers:
|
||||||
|
|
||||||
|
- The fixture is not proven wrong. It recorded that runtime's behaviour correctly at the
|
||||||
|
commit and version its own measurement block pins, and a later classification does not
|
||||||
|
reach back and falsify an earlier measurement.
|
||||||
|
- The runtime does not necessarily have a bug. Where the seed runtime has stated that the
|
||||||
|
classification behind such a table is calibration it does not freeze, a release that
|
||||||
|
classifies the payload differently is a change it reserved, not a defect.
|
||||||
|
|
||||||
|
So: the fixture MUST NOT be rewritten on the strength of the divergence alone; the divergence
|
||||||
|
SHOULD be recorded against the version pinned; and re-pinning the case to a later version of
|
||||||
|
the seed runtime is a separate decision, taken deliberately and released on its own. This is
|
||||||
|
the disposition §5 already applies to a stale `observed_out_of_scope` entry, extended to the
|
||||||
|
one place where it can reach a verdict — and a divergence recorded here is the signal that
|
||||||
|
the re-pinning decision is due, not a reason to leave it open.
|
||||||
|
|
||||||
|
Three things this does **not** do.
|
||||||
|
|
||||||
|
- **It creates no fourth verdict.** The counts of §1.1 and
|
||||||
|
[`schema/conformance-declaration.schema.json`](../schema/conformance-declaration.schema.json)
|
||||||
|
are unchanged: a case whose expected findings are not produced still **fails**, and is still
|
||||||
|
named in `failed_cases`. What changes is what the failure licenses concluding, not what is
|
||||||
|
reported.
|
||||||
|
- **It does not reach a third-party implementer** of the same table. Against them the fixture
|
||||||
|
is the contract, exactly as §7 states — which is what these cases were minted to provide,
|
||||||
|
and the only thing they can prove while one runtime is all there is.
|
||||||
|
- **It is not a licence to self-declare.** The exemption is carried by the corpus's own
|
||||||
|
provenance record for the scope. A runtime MUST NOT claim it for a case by asserting that
|
||||||
|
its own divergence is calibration.
|
||||||
|
|
||||||
## 8. What conformance does and does not prove
|
## 8. What conformance does and does not prove
|
||||||
|
|
||||||
Passing this corpus proves that a runtime agrees with the other runtimes that pass it, on
|
Passing this corpus proves that a runtime agrees with the other runtimes that pass it, on
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue