docs(conventions): the merge button is off for a reason, and the reason now lives in the repo
CONVENTIONS.md consolidates rules that already applied but were readable only to the maintainer: the charter (CLAUDE.md), versioning and vendoring (README.md), the reporting route (SECURITY.md), and file conventions visible only in the shape of the files. Two things stop being inferences. Why pull requests are off: this repository is vendored into runtimes that pin a tag, so a data change changes what they find, and that must be coordinated before it exists. And when a value may change: the three recorded mechanisms - re-extraction, retraction, owner-directed authoring - each tied to its source_fidelity key, with merit named as not among them. The four offline checks are included and were each proven to go red on a planted violation, not merely green on a clean tree. They are one-liners because a script would be .sh and check 4 would fail on it. No data file changed; a runtime passing 0.4.1 passes 0.4.2 unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KDRVXSFhKpFTzdoyjpjL17
This commit is contained in:
parent
2d86151e3b
commit
fe5e6b290b
3 changed files with 256 additions and 4 deletions
199
CONVENTIONS.md
Normal file
199
CONVENTIONS.md
Normal file
|
|
@ -0,0 +1,199 @@
|
|||
# Conventions
|
||||
|
||||
The rules a change to this repository is held to, in one place.
|
||||
|
||||
Nothing here is new policy. Every rule below was already being applied — some of it stated in
|
||||
[README.md](README.md), some in [SECURITY.md](SECURITY.md), some only visible in the shape of
|
||||
the files themselves. It is collected here because a convention that only exists in the
|
||||
maintainer's head is not a convention an outside reader can meet, and because two of the
|
||||
decisions this repository makes — that nothing here runs, and that pull requests are switched
|
||||
off — look arbitrary until the reason is written down next to them.
|
||||
|
||||
This file binds **contributions to this repository**. It does not bind the runtimes that read
|
||||
the data; that is what `spec/` is for, and those files say `Status: normative` and mean it.
|
||||
|
||||
## The charter: nothing here runs
|
||||
|
||||
**This repository contains no executable code, and it will not acquire any.**
|
||||
|
||||
Not permitted, without exception:
|
||||
|
||||
- `.mjs`, `.js`, `.ts`, `.py`, `.sh` — or any other file that implements detection, scanning,
|
||||
normalisation, scoring or I/O;
|
||||
- `package.json`, `pyproject.toml`, lockfiles, dependencies, build steps;
|
||||
- an import of, or knowledge of, any framework, SDK or runtime;
|
||||
- network access, model calls, dependence on the clock, or randomness.
|
||||
|
||||
Permitted: JSON data, normative specifications in Markdown, and conformance fixtures
|
||||
(`input.txt` plus `expected.json`).
|
||||
|
||||
The reason is `fork-and-own`. A consumer on a runtime nobody here has thought of should be able
|
||||
to vendor this repository without inheriting a single technology choice. A build step is a
|
||||
technology choice; so is a test runner. The moment one exists, the set of runtimes that can
|
||||
adopt this core shrinks to the set that tolerates it.
|
||||
|
||||
The consequence is that **this repository cannot validate itself**. There is no CI in this
|
||||
organisation and nothing runs on push. The checks below are yours to run, and they are the only
|
||||
ones there are.
|
||||
|
||||
## How a change gets in — and why not by pull request
|
||||
|
||||
Pull requests are switched off on the canonical repository at
|
||||
`git.fromaitochitta.com/open/llm-security-commons`, and issues are not the reporting channel
|
||||
either. That is deliberate, and the reason is stronger than a preference about tooling.
|
||||
|
||||
This repository is **vendored into independent runtimes** — a Claude Code plugin on Node/ESM, a
|
||||
Python guard, an advisory surface — each pinning a tag. The contract between them is semver, and
|
||||
a change to detection data changes what those runtimes *find*. A patch to a pattern table is not
|
||||
a contribution that can be merged and then socialised; it is a contract change that has to be
|
||||
coordinated with every consumer **before it exists**, because the moment it is tagged, the next
|
||||
consumer to pull it gets different findings than the one that pulled yesterday. A merge button
|
||||
does not do that, and nothing downstream of a merge button can.
|
||||
|
||||
So the routes in are:
|
||||
|
||||
1. **Fork and own it.** MIT, and an intended use rather than a tolerated one. If you need a
|
||||
different value in your runtime, this is the fast path and it is fully supported.
|
||||
2. **Report it privately** — see [SECURITY.md](SECURITY.md). A wrong entry in a detection table
|
||||
is a silent false negative in every runtime that reads it, so a report about one is a
|
||||
security report even though nothing here executes. That file also explains why a confirmed
|
||||
defect in extracted data is usually decided in the runtime it came from before it changes
|
||||
here.
|
||||
|
||||
If you maintain a consumer, the coordination channel is direct contact with the maintainer, not
|
||||
this repository's forge surface.
|
||||
|
||||
## Data files (JSON)
|
||||
|
||||
- **Every JSON file carries a top-level `"version"`** — a semver string. No exceptions.
|
||||
- **Every JSON file states what it is and where its data came from**, in a top-level
|
||||
`"$comment"` or `"description"`. Provenance is not optional metadata here; it is what makes
|
||||
the difference between a table and a rumour.
|
||||
- 2-space indentation, LF line endings, a trailing newline, UTF-8 without BOM.
|
||||
- **Code points are written as strings** — `"U+200B"` — never as the raw invisible character.
|
||||
Review cannot see what it cannot render, and a reviewer who cannot see a character cannot
|
||||
check it. The single exception is `conformance/*/input.txt`, which by definition contains the
|
||||
actual bytes.
|
||||
- **Keys are stable identifiers.** Consumers match on them. **Changing a key is a breaking
|
||||
change** and is versioned as one.
|
||||
|
||||
The four files that carry detection material — `codepoints/carriers.json`,
|
||||
`signatures/secret-egress.json`, `signatures/malware-signatures.json`,
|
||||
`signatures/active-content.json` — take one further rule, which is the most important line in
|
||||
this document:
|
||||
|
||||
> **They are changed only from verified source data** — a dump from the owning repository, the
|
||||
> Unicode standard, published vendor documentation. Never from memory, never "filled in for
|
||||
> consistency". A guessed code point or a regex with wrong escaping is a silent false negative:
|
||||
> a detector that looks like it is working and is not looking. If an entry cannot be verified,
|
||||
> leave it out, or mark it explicitly unverified in its `$comment`.
|
||||
|
||||
## Specifications (Markdown)
|
||||
|
||||
- A normative specification carries **`Status: normative`** at the top and uses RFC 2119 terms
|
||||
(MUST / MUST NOT / SHOULD / MAY) in uppercase, deliberately. These files bind the
|
||||
implementations that read them.
|
||||
- An informative document (`docs/`) carries **`Status: informative`** and is **never ground
|
||||
truth**. It records measurements, history and open disagreements; a runtime is not wrong for
|
||||
disagreeing with one.
|
||||
- Naming a file that does not exist yet is allowed where the layout is part of the contract —
|
||||
`spec/decode-pipeline.md` is named in README.md and marked **Planned**. It is not a link, and
|
||||
nothing depends on it. A normative spec guessed at would be worse than an absent one.
|
||||
|
||||
## Conformance cases
|
||||
|
||||
- One directory per case: `conformance/<case-id>/input.txt` and
|
||||
`conformance/<case-id>/expected.json`.
|
||||
- `<case-id>` is stable and descriptive. **Changing a case id is a breaking change** — a
|
||||
published conformance result names it.
|
||||
- `expected.json` is **ground truth**. If a runtime disagrees with it, the runtime is wrong.
|
||||
- The one way that reverses: the fixture is proven wrong. Then the fixture changes **in its own
|
||||
commit, with the reason written down** — never folded into a change that does something else,
|
||||
because a fixture edit is the one edit that can make every conforming runtime wrong
|
||||
identically.
|
||||
- A case declares the data files it is `scope`d to. A runtime that does not implement a scoped
|
||||
table reports the case `not-applicable` — a third verdict beside pass and fail, and one that
|
||||
must be reported rather than dropped from the denominator. See
|
||||
[`spec/conformance-corpus.md`](spec/conformance-corpus.md).
|
||||
|
||||
## When a value may change
|
||||
|
||||
Detection values do not change here because someone here judged them wrong. Three mechanisms
|
||||
have moved a value so far, and each is recorded in the file itself rather than only in the
|
||||
changelog:
|
||||
|
||||
1. **Re-extraction** — the owning runtime changed its own value, and this repository re-read the
|
||||
source at a pinned public commit. Recorded in `source_fidelity.post_extraction_drift`.
|
||||
2. **Retraction** — this repository described its own provenance wrongly. The stored value may
|
||||
have been right all along; the account of where it came from was not. Recorded in
|
||||
`source_fidelity.retracted`.
|
||||
3. **Owner-directed authoring** — the owning runtime decided a value and asked this repository
|
||||
to carry it, because the dependency has inverted: the source now reads *this* file and holds
|
||||
no literal to re-read. Recorded separately, in `source_fidelity.owner_directed_changes`,
|
||||
precisely because calling it drift would assert that the source moved and commons followed —
|
||||
which would be false in the one direction that matters.
|
||||
|
||||
What is **not** on that list is merit. Data extracted from an implementation is kept
|
||||
behaviour-identical to it on purpose, because a copy that disagrees with its source is the exact
|
||||
failure this repository exists to prevent. Producing one as a *fix* would be self-defeating. If
|
||||
you believe an extracted value is wrong, say so — and expect the decision to be taken in the
|
||||
runtime where the pattern is under test.
|
||||
|
||||
Data **authored here** rather than extracted — conformance payloads, flagged as
|
||||
`authored_payloads` in `conformance/manifest.json` — is this repository's own to correct.
|
||||
|
||||
## Versioning
|
||||
|
||||
Two version numbers, and they are not locked to each other:
|
||||
|
||||
- **The repository tag** (`vX.Y.Z`) versions **the contract**: the file set, the key names, the
|
||||
case ids, the disposition semantics.
|
||||
- **Each JSON file's own `"version"`** is bumped when *that file* changes.
|
||||
|
||||
What counts as which:
|
||||
|
||||
| Change | Bump |
|
||||
| --- | --- |
|
||||
| New data field, new entry | minor |
|
||||
| Changed or removed key, case id, or layout | **major** — consumers break |
|
||||
| A change to what a conforming runtime *finds* | minor in 0.x, and the changelog says so |
|
||||
|
||||
That last row is why **the changelog entry is the thing to read before upgrading, not the
|
||||
version number**. Pre-1.0, a release that changes findings is still a minor bump; only the entry
|
||||
tells you whether your assertions move.
|
||||
|
||||
Consumers vendor **a tag, never `main`** — a conformance result can only be attributed to a tag.
|
||||
Nothing polls for updates; when a change moves detection data, the maintainer notifies known
|
||||
consumers directly, and their upgrade remains their own action on their own schedule.
|
||||
|
||||
## Checks to run before proposing a change
|
||||
|
||||
There is no CI. These four are the validation surface, they run offline in a second, and each
|
||||
one has been confirmed to go red on a violation rather than merely green on a clean tree.
|
||||
|
||||
```bash
|
||||
# 1. Every JSON file is well-formed
|
||||
find . -name '*.json' -not -path './.git/*' -print0 \
|
||||
| xargs -0 -n1 python3 -m json.tool > /dev/null && echo OK
|
||||
|
||||
# 2. Every JSON file outside conformance/ carries a top-level "version"
|
||||
find . -name '*.json' -not -path './.git/*' -not -path './conformance/*' -print0 \
|
||||
| xargs -0 python3 -c 'import json,sys
|
||||
missing=[p for p in sys.argv[1:] if "version" not in json.load(open(p))]
|
||||
print("\n".join("MISSING version: "+p for p in missing) or "OK")'
|
||||
|
||||
# 3. Every spec carries its normative marker (prints offending files, nothing = clean)
|
||||
grep -L 'Status: normative' spec/*.md || echo OK
|
||||
|
||||
# 4. Charter guard: no executable code has crept in
|
||||
find . -type f \( -name '*.mjs' -o -name '*.js' -o -name '*.ts' -o -name '*.py' -o -name '*.sh' \) \
|
||||
-not -path './.git/*' | grep . && echo 'CHARTER VIOLATION' || echo OK
|
||||
```
|
||||
|
||||
They are written as shell one-liners rather than shipped as a script because a script would be
|
||||
`.sh`, and check 4 would then fail on the tooling meant to enforce it.
|
||||
|
||||
What they do **not** check: whether a value is *correct*. Nothing offline can. That is what the
|
||||
conformance corpus is for, and it runs in each consumer's own test suite against a pinned tag —
|
||||
constraining two of the seven data files, which is a real limit and is stated in
|
||||
[README.md](README.md) under **Known limitations**.
|
||||
Loading…
Add table
Add a link
Reference in a new issue