`VERSION-UNAVAILABLE` (SKIP/notRun) becomes `VERSION-NONE` (OK). A repo that claims no version in any of the four places one can be written down has not left a check un-run — the check ran, saw all four, and found no subject. That is the shape `checkReadmeLanguage` has always answered with OK. 0.8.0 deferred this and recorded the reason in the engine: re-levelling "moves a repo's status". Measured false across 19 clones — an added OK cannot worsen the worst *judged* finding, and all three repos emitting the code already read OK. Status moves only for a repo whose entire finding set was skips. OK cannot bless a real gap here: no class requires a version file, and a `plugin` missing its manifest is an independent FILE-MISSING ERROR. Renamed, not just re-levelled: `-UNAVAILABLE` is this engine's naming for a notRun skip, and one OK-level `-UNAVAILABLE` would mislead exactly the reader this change serves. A source scan keeps the old name out. A/B sweep, both engines offline, 19 clones: 223 judged findings, every status and buckets identical; 3 conversions (notChecked 14→13, 3→2, 14→13), byDesign untouched. 170 tests. `--refresh`: register 21 = forge 21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CTNGvxzCLZp3UXfibYG4kb
12 KiB
repo-standard
Per-repo gate for the open/ presentation standard, packaged as a marketplace
plugin.
Context
Two components, one boundary:
- Engine (
scripts/repo-standard-check.mjs) — pure classifiers with all I/O resolved into their input, mirroring the marketplace'scheck-versions.mjs. Findings areERROR/WARN/SKIP/OK; exit 1 onERROR. Pinned byscripts/repo-standard-check.test.mjs(npm test). - Skill (
skills/repo-standard/) — the judgement the script cannot encode. No checking logic lives here; it calls the engine.
register/repos.json is the single taxonomy register (name → class, per-class
requirements, and the known non-repo names). Central by design: per-repo copies
would recreate, in data, exactly the drift this plugin exists to remove.
Invariants
- This repo has a PUBLIC remote.
STATE.mdis LOCAL-ONLY and gitignored from the first commit. Repos on a private remote track theirs — do not carry that habit across in either direction. - The gate sees ONE repo. Anything needing a view across the whole org (topic coverage, competing install forms, catalog-vs-forge divergence) does not belong here. It is measured where the org is enumerated.
- It records, it does not fix. Findings first, remediation afterwards. Patching while measuring is how the inconsistency it detects was produced.
SKIPis never a pass — and never a severity either. A check that could not run says so and names why. But a skip is the ABSENCE of a verdict, so it cannot be the worst of a set that contains real ones: a repo'sstatusis the worst judged finding, andSKIPonly when nothing was judged (unregistered, or an empty finding set). It used to outrankOK, which made five repos with 0 ERROR, 0 WARN and a dozen OK headline as "skipped" in org-ops census 05 —okr, with the most OK in the org, read as unread. Fixing it in the status alone would have traded that for the worse direction, "skipped checks look clean", so coverage rides its own axis:notCheckedin--json, a· N not checkedqualifier on the summary line. AbsentnotCheckedmeans an older engine and prints the pre-0.7.0 line — it is not zero.- A skip is one of TWO facts, and merging them says the same wrong thing more
quietly.
byDesign(the check saw it and declined — out-of-repo links, fixture paths; it can never become a verdict, nobody has an action) versusnotRun(a re-run or an operator action resolves it). Measured across 19 clones, 35 of 39 skips were the first kind, which is whyportfolio-optimiser — OK · 11 not checkedread as eleven unread checks. The kind is set at the EMISSION site, never looked up from the code:VERSION-TAGis emitted atSKIPwith no tags and atERRORwith the wrong one, so a code→kind map has to re-derive a reason the emission site already had. Untagged falls tonotRun, the loud side. The summary line names onlynotRun; the deliberate ones keep their ownNOT JUDGEDheading, because the wanted side effect is exposure, not silence — landing the split in--jsonalone would have left the symptom untouched. The test that holds this is a scan of the engine SOURCE, not of a finding set: a corpus test only sees the sites it manages to trigger, and the next site added would take the reader's default invisibly. - "Nothing to judge" is an
OK, not a skip — and it is a THIRD shape, not a third bucket. A check that ran, saw everything, and found no subject has produced a verdict: there is nothing here to be wrong.checkReadmeLanguagealready answered that way for a README with no prose;VERSION-NONE(0.9.0, wasVERSION-UNAVAILABLE/notRun) does for a repo that claims no version. The deferral that kept it a skip for one release was recorded in the engine as "re-levelling moves a repo's status" — measured false: an addedOKcannot worsen the worst judged finding, and all three affected repos already readOK. Status moves only for a repo whose entire finding set was skips. The lesson is not about this code: a recorded decision that is wrong is worse than no record, because the next session re-derives the measurement instead of reading it. And the guard againstOKblessing a real gap is that the gap is caught elsewhere — no class requires a version file, and apluginmissing its manifest is an independentFILE-MISSINGERROR. Without that, this would have been "SKIP is never a pass" one level up. - When a check fires many times in one repo, suspect the CHECK. The first
link pass produced ~30 findings against
llm-securityand all were noise — regexes in code spans,file:URLs, relative paths resolved against the wrong directory. Strip code before scanning text; resolve links against the file they sit in. A gate that is wrong this often gets switched off. - Two axes on every finding. Level (
ERROR/WARN/SKIP/OK) and bucket (broken/missing/weakening). They are independent — aweakeningfinding can be anERROR. - Class is structural, traits are judgement. Class is read off the catalog
and the remotes. A trait (
security) says what the code does, which no remote knows, and the operator owns that list. - Who the reader is decides what is required. Contributor-facing documents
are required by no class — solo-maintained, and the published stance says so.
Consumer-facing ones (
SECURITY.md,LICENSE, non-goals, limitations) are untouched by that. This is not a rule against having the others. - No CI badge, because there is no CI. The forge has zero Actions runners registered (measured). The substitute is one command from a clean clone, said plainly. A static badge asserting a run is the anti-pattern this gate flags — and an early draft of this README carried one.
- Three outcomes on references. "No match" and "match on a known non-repo" must stay distinct findings. Collapsing them hides real loss inside correct text — the exact defect class this gate exists to catch.
- Two API calls per invocation, anonymous, with 429 retry. The org listing
(description + topics) is one; the catalog's
marketplace.jsonfor INSTALL-TRUTH is the other (added after this used to say "one call" — that line went stale and stayed stale until a 13-repo shell loop trusted it and tripped the rate limiter at 26 requests). Both go throughfetchWithRetry, which retries HTTP 429 rather than silently reporting SKIP. Both are anonymous — no token, confirmed no different with one — so the gate works for any reader, not only someone holding one. A sweep across every repo still does not belong here: it needs the listing fetched once, not once per invocation, which is a different shape of caller (org-ops), not a flag on this engine. The "13 calls in a loop" explanation was incomplete (2026-08-04): the forge's nginx never sendsRetry-Afteron its 429s (measured directly), sofetchWithRetryalways falls back to exponential backoff — theRetry-Afterbranch is live code with no live path yet. The limit is also smaller than "loop of 13" implied: 20 concurrent requests from one IP reproduced it directly, no loop needed, and a single well-formed 2-call invocation can still lose if something else on the same IP is calling the forge at the same moment (other repos' hooks, another session). The block is a leaky bucket, not a fixed ban — a 20-25 request burst took up to ~15s to fully drain.fetchWithRetrydefaults toretries: 5/maxDelayMs: 8000(23s worst case) to cover that. - Codepoints, not bytes, not UTF-16 units. Use
[...s].length. An em-dash exposes only the byte layer; astral characters expose the rest. - The reader decides a link's level, not just what is required. Root
documents are the shop window — a dead link there is an
ERROR. Below the root it is aWARN: that is where session plans, agent working files and path-traversal fixtures with deliberately invalid targets live. Measured, 30 of 43 findings were down there and all wereERRORs. - A fixture-path dead link is
SKIP, notWARN— and never silently dropped.test/,tests/,fixtures/(exact segment) and*golden*(substring) mark a path as presumed intentional; the finding still fires asLINK-INTERNAL-FIXTUREwith its file and line, it just isn't judged. Grounded innav-golden-escape/bundle/index.md's deliberate../../../../etc/passwdescape: the deep..pops the whole base path rather than resolving tonull, so it read as a genuineWARN— third tool in the org to hit this exact pattern, which is the signal the check was at fault. Measured before shipping: 16 findings before, 16 after, across all 20 local clones — every one converted 1:1, none disappeared. - A repo's name is its remote, not its directory.
catalog/holdsktg-plugin-marketplace. The basename left it unregistered with zero checks run, against the one repo every catalog rule depends on. - A decision needs somewhere to live, or the gate repeats itself forever.
The engine already held that a human README title is the operator's call —
and still warned about it every round, because a YES could not be recorded.
Six warnings, unchanged across censuses 05, 06 and 07.
titlesin the register is that record: set, the H1 matching it isOK; unset, theWARNstands. What the gate must never do is make "we decided this" and "nobody looked" the same output. The wanted side effect is exposure, not silence —ai-psychosisis deliberately unregistered so it stands alone. - An exemption is a finding, not a deletion.
readme_desc_match: falseturns off README-DESC equality for a class, and the check still emits anOKnaming why. An exception nobody can see reads exactly like a check that silently stopped running. - Class rules live in the register, never as a class name in the engine.
The
org-profileexemption is a flag on the class, notif (klass === 'org-profile'). Per-repo copies of a rule are the drift this plugin exists to remove; a class name hardcoded in a classifier is the same defect one level up. - The version names a file; only the sha names the code.
engineVersionwas added because a stale cache served an old engine silently — but a feature and its version bump are two commits, so a worktree carries new behaviour under the old number for a window. Measured: a sweep stamped 18 raw files0.4.0, four of them holding findings from a check that only exists in0.5.0.engineCommitcloses that, derived from the same checkout with no network call. It is present-and-nullwhen underivable, never absent — an absent key means an older engine,nullmeans this one ran without a HEAD. - No hook until the rule is precise. A blocking gate that fails a correct repository is the mechanism that gets gates switched off.
Commands
npm test # 170 tests
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
node scripts/repo-standard-check.mjs --offline # no network call
node scripts/repo-standard-check.mjs --json # machine output
node scripts/repo-standard-check.mjs --refresh # register vs. forge
Release
Run --refresh before every release. Register freshness is owned HERE, not
by the sweeps that read the register. Twice running, a newly published repo was
missing when a census ran, and the cost is not a gap — it is false ERRORs in a
different repo: portfolio-optimiser earned three LINK-DEADs against a repo
that existed, in the same round it fixed its three real ones, so its status line
did not move even though the work was done. A stale register makes the gate
lie about repos that are not even the stale one. One owner, no shared duty:
consumers of the register are told not to check freshness themselves.
Polyrepo rule: a version bump is not finished until the tag vX.Y.Z is pushed
and the catalog ref is bumped to it. Use release-plugin.mjs, never a
hand-edited ref.
check-versions.mjs reads the catalog README's per-plugin label, and a missing
entry is a silent null rather than an error — release-plugin.mjs rewrites an
existing heading but cannot create one. A new plugin's catalog README entry has
to be added by hand once.