open/repo-standard
1
0
Fork 0
Code Issues Releases 4 Activity

feat(engine): RELEASE-CURRENT — the version page a stranger actually reads

Browse source 
A release is not a ref, so `git ls-remote` cannot answer this one. It is
therefore the third API call per invocation, and the only new one the
acquisition model adds.

Both sides come from the forge, never the clone: comparing a local tag to a
published release would report portfolio-optimiser as stale when the real
defect is a tag that was never pushed — REMOTE-SYNC's subject, not this one.

Measured across all 22 registered repos before the rule was locked: 4 have no
tags, 2 tag without ever publishing a release, 11 are current, 5 lag. Those 2
are why zero releases is an OK and not a finding — nothing in a repo says which
of the two legitimate conventions it follows, the same measurement that
rejected VERSION-DRIFT. Lagging is a WARN because the remedy is safe, and
because this repo is the org's worst offender: an ERROR would have let the gate
settle an operator question by exiting 1 on its own author.

The shell that produced that baseline sorted tags lexically and put v0.9.0
above v0.10.0 — the exact defect compareTags exists to fix, reproduced in the
tool meant to validate it. Recorded as an invariant: derive "newest" with the
engine's own comparator or the measurement is fiction.

CLAUDE.md's API-call count moves from two to three in this same commit; that
line has now gone stale twice. The unmarked "429 at ~40" figure is marked
unusable — its endpoint class was never recorded, and org-ops measured ~110
raw reads at 0.4s with zero 429 the same day.

205 tests (was 196).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWc4piM4QW6Jxfky2Rw4Z8
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 22:57:35 +02:00
commit a563035feb
4 changed files with 275 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

54
CHANGELOG.md
View file

@ -4,6 +4,60 @@ All notable changes to this project are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- **`RELEASE-CURRENT` — does the release page show the version the code is on?**
A stranger who wants a version they can name lands on the forge's release
page, and it is the one surface refs cannot answer: a release is not a ref, so
`git ls-remote` has nothing to report. This is therefore the third API call
per invocation, and the only new one the acquisition model adds.
Both sides are read from the FORGE, never from the clone. Comparing a local
tag against a published release would report `portfolio-optimiser` as having a
stale release when the actual defect is a tag that was never pushed (v1.0.0
local, v0.1.0 published) — a different finding, owned by a different check.
Four outcomes, and three of them are `OK`:
| outcome | level | code |
|---|---|---|
| releases or refs unreadable | `SKIP`/`notRun` | `RELEASE-UNAVAILABLE` |
| no tags on the forge | `OK` | `RELEASE-NONE` |
| tags, no release published | `OK` | `RELEASE-TAGS-ONLY` |
| newest release is the newest tag | `OK` | `RELEASE-CURRENT` |
| newest release lags the newest tag | `WARN`/`weakening` | `RELEASE-STALE` |
The levels come from a measurement across all 22 registered repos
(2026-08-12): 4 have no tags, 2 tag without ever publishing a release, 11 are
current, 5 lag. Those 2 — `ktg-plugin-marketplace` and `llm-security-commons`
— are why zero releases is an `OK` and not a finding. Nothing in a repository
says which of the two legitimate conventions it follows, and a gate that fails
a correct repository is the mechanism that gets gates switched off. That is
the same measurement that rejected `VERSION-DRIFT` one check over.
Lagging is a `WARN` rather than an `ERROR` for two reasons. The remedy is
safe: publishing a release for a tag that already exists moves no published
ref, unlike the remedy `TAG-ANNOTATED` has to withhold from tag history. And
this repository is the worst offender in the org — v0.9.0 tagged, v0.3.0
published — so an `ERROR` would have let the gate settle an operator
question, whether this org publishes releases at all, by exiting 1 on its own
author.
### Changed
- **The API-call count in `CLAUDE.md` is three, updated in the same commit as
the check that changed it.** That line has now gone stale twice; the first
time it stayed stale until a 13-repo shell loop trusted it and tripped the
rate limiter.
- **The "429 at ~40 requests" figure is marked unusable.** The run that produced
it did not record which endpoint class it hit, and org-ops measured ~110
sequential raw reads at 0.4s with zero 429 the same day. Either those are two
different limiters or the older figure is wrong, and the note cannot tell them
apart. One API-axis figure this repo can stand behind was recorded instead: 22
sequential `/api/v1/` calls at 2s pacing, anonymous, zero 429.
## [0.9.0] — 2026-08-09
### Changed

49
CLAUDE.md
View file

@ -101,12 +101,16 @@ would recreate, in data, exactly the drift this plugin exists to remove.
- **Three outcomes on references.** "No match" and "match on a known non-repo"
must stay distinct findings. Collapsing them hides real loss inside correct
text — the exact defect class this gate exists to catch.
- **Two API calls per invocation, anonymous, with 429 retry.** The org listing
- **Three API calls per invocation, anonymous, with 429 retry.** The org listing
(description + topics) is one; the catalog's `marketplace.json` for
INSTALL-TRUTH is the other (added after this used to say "one call" — that
INSTALL-TRUTH is the second (added after this used to say "one call" — that
line went stale and stayed stale until a 13-repo shell loop trusted it and
tripped the rate limiter at 26 requests). Both go through `fetchWithRetry`,
which retries HTTP 429 rather than silently reporting SKIP. Both are
tripped the rate limiter at 26 requests); the repo's releases listing for
`RELEASE-CURRENT` is the third, and is the only subject the cheaper git
channel cannot cover, because a release is not a ref. This count is the line
that has now gone stale twice — it is updated in the SAME commit as the check
that changes it, never afterwards. All go through `fetchWithRetry`,
which retries HTTP 429 rather than silently reporting SKIP. All are
anonymous — no token, confirmed no different with one — so the gate works
for any reader, not only someone holding one. A sweep across every repo
still does not belong here: it needs the listing fetched once, not once per
@ -126,7 +130,15 @@ would recreate, in data, exactly the drift this plugin exists to remove.
**Burst is not the only way in** (2026-08-12): a *steady* 1 request/second
tripped 429 at around request 40, so pacing alone does not buy immunity —
only a smaller total does. Recovery measured at 35s drain plus 3s pacing:
12/12 through.
12/12 through. **That ~40 is unmarked and cannot be quoted as a forge
number**: the run did not record which endpoint class it hit, and org-ops
measured ~110 sequential `/raw/branch/main/` reads at 0.4s with zero 429 the
same day. Either they are two different limiters — the likelier reading —
or the ~40 is wrong, and nothing in the note distinguishes those. The one
API-axis figure this repo can stand behind: 22 sequential `/api/v1/` calls
at 2s pacing, anonymous, zero 429 (the `RELEASE-CURRENT` baseline run). That
is below the suspected knee, so it bounds nothing — it is a floor, not a
ceiling. Every future measurement records its endpoint class.
- **Two acquisition channels, and refs are not one of the API calls.** Four
queued checks need state the local clone does not have, and answering the
mechanism question once is the point — asked per check, it gets four
@ -145,6 +157,31 @@ would recreate, in data, exactly the drift this plugin exists to remove.
block locally, while the forge's "Verified" badge needs a key registered
there — so `TAG-SIGNED` answers *is it signed*, never *does the forge vouch
for it*.
- **A stale release page is a WARN, and "no releases" is not a finding at all.**
`RELEASE-CURRENT` compares the newest release the forge publishes against the
newest tag the forge lists — both sides from the FORGE, never the clone.
Comparing a local tag against a published release would report
portfolio-optimiser as stale when the real defect is a tag that was never
pushed (v1.0.0 local, v0.1.0 published) — REMOTE-SYNC's subject, not this
one. Measured across all 22 registered repos (2026-08-12): 4 have no tags,
2 tag without ever publishing a release, 11 are current, 5 lag. Those 2 —
`ktg-plugin-marketplace` and `llm-security-commons` — are why zero releases
is an `OK` (`RELEASE-TAGS-ONLY`) and not a finding: nothing in a repo says
which of the two legitimate conventions it follows, which is the measurement
that rejected `VERSION-DRIFT` one check over. Lagging is a `WARN` because
the remedy is safe — publishing a release for a tag that already exists
moves no published ref, unlike the remedy `TAG-ANNOTATED` has to withhold.
It is also `WARN` because this repo is the worst offender in the org
(v0.9.0 tagged, v0.3.0 published), and an `ERROR` would have let the gate
decide an operator question — whether this org publishes releases at all —
by exiting 1 on its own author.
- **Version order, in the measurement as well as in the code.** The shell that
measured this check's baseline sorted tags with `sort -t. -k1,1V` and put
`v0.9.0` above `v0.10.0` — the exact defect `compareTags` already exists to
fix, reproduced in the tool that was supposed to validate it. It named
`repo-mailbox`'s newest tag as v0.9.0 when it is v0.21.0. A baseline derived
with different ordering than the rule cannot validate the rule. Derive
"newest" with the engine's own comparator, or the measurement is fiction.
- **Codepoints, not bytes, not UTF-16 units.** Use `[...s].length`. An em-dash
exposes only the byte layer; astral characters expose the rest.
- **The reader decides a link's level, not just what is required.** Root
@ -225,7 +262,7 @@ would recreate, in data, exactly the drift this plugin exists to remove.
## Commands
```bash
npm test # 196 tests
npm test # 205 tests
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
node scripts/repo-standard-check.mjs --offline # no network call
node scripts/repo-standard-check.mjs --json # machine output

102
scripts/repo-standard-check.mjs
View file

@ -754,6 +754,72 @@ export function checkTagIntegrity({ tagObjects, name }, register) {
const CLAIM_WORD = /^(tests?|build|ci|coverage|passing)$/i;
const claimsARun = (s) => s.split(/[^a-z0-9]+/i).some((w) => CLAIM_WORD.test(w));
// The forge's release page is where a stranger lands when they want a version
// they can name, and it is the one surface refs cannot answer — a release is
// not a ref, so `git ls-remote` has nothing to report. That is why this is the
// THIRD API call and the only new one the acquisition model adds.
//
// Both sides come from the FORGE, never from the clone. Comparing a local tag
// against a published release would report portfolio-optimiser as having a
// stale release when the real finding is a tag that was never pushed (v1.0.0
// local, v0.1.0 published) — a different defect, owned by a different check.
//
// The levels come from a measurement across all 22 registered repos
// (2026-08-12): 4 have no tags at all, 2 tag without ever publishing a release,
// 11 are current, 5 lag their newest tag. Those 2 are why "no releases" is an
// OK and not a finding: nothing in a repo says which of the two legitimate
// conventions it follows, and a gate that fails a correct repository is the
// mechanism that gets gates switched off — the same measurement that rejected
// VERSION-DRIFT one check over. Lagging is a WARN rather than an ERROR because
// the remedy is safe: publishing a release for a tag that already exists moves
// no published ref, unlike the remedy TAG-ANNOTATED has to withhold.
export function checkReleaseCurrent({ forgeTagsSelf, releases }) {
if (releases === null || releases === undefined) {
return [{ level: 'SKIP', skip: 'notRun', code: 'RELEASE-UNAVAILABLE', msg: 'forge releases not available — check not run (offline, or the listing failed)' }];
}
// One side missing is not agreement. Refs and releases are acquired over two
// different channels, so either can fail alone.
if (forgeTagsSelf === null || forgeTagsSelf === undefined) {
return [{ level: 'SKIP', skip: 'notRun', code: 'RELEASE-UNAVAILABLE', msg: 'forge refs not readable — cannot tell whether the newest release is the newest tag' }];
}
const tags = [...forgeTagsSelf].sort(compareTags);
if (tags.length === 0) {
// The VERSION-NONE shape: the check ran, saw every tag the forge has, and
// found no subject. Nothing could have been released, so there is nothing
// here to be wrong — a verdict, not an absent one.
return [{ level: 'OK', code: 'RELEASE-NONE', msg: 'no tags on the forge — no release could exist' }];
}
const newestTag = tags[tags.length - 1];
if (releases.length === 0) {
return [{
level: 'OK',
code: 'RELEASE-TAGS-ONLY',
msg: `${tags.length} tag(s) and no release published — this repo tags without publishing releases, which is a convention this gate does not judge`,
}];
}
// Version order, not the order the API handed them over: the releases listing
// sorts by creation time, and a patch cut after a minor would read as newest.
const newestRelease = [...releases].sort(compareTags).pop();
if (compareTags(newestRelease, newestTag) < 0) {
return [{
level: 'WARN',
code: 'RELEASE-STALE',
bucket: 'weakening',
msg: `newest release is \`${newestRelease}\` but the newest tag is \`${newestTag}\` — the release page shows a version older than the code. Publish a release for \`${newestTag}\`.`,
}];
}
return [{
level: 'OK',
code: 'RELEASE-CURRENT',
msg: newestRelease === newestTag
? `newest release \`${newestRelease}\` is the newest tag`
: `newest release \`${newestRelease}\` is ahead of every tag the forge lists`,
}];
}
// Counting badges needs a NARROWER rule than detecting a dishonest one. The
// claim check reads any image, any host, on purpose. Here the opposite error
// matters: counting a screenshot or an architecture diagram as clutter would
@ -1191,7 +1257,7 @@ export function bucketsOf(findings) {
}
export function classifyRepo(
{ name, files, present, description, pluginVersion, readmeBadge, changelogTop, tags, tagObjects, catalogNames, forgeTagsByRepo },
{ name, files, present, description, pluginVersion, readmeBadge, changelogTop, tags, tagObjects, catalogNames, forgeTagsByRepo, forgeTagsSelf, releases },
register,
) {
const klass = register.repos?.[name];
@ -1230,6 +1296,7 @@ export function classifyRepo(
...checkBoilerplate({ files }),
...checkVersionConsistency({ pluginVersion, readmeBadge, changelogTop, tags }),
...checkTagIntegrity({ tagObjects, name }, register),
...checkReleaseCurrent({ forgeTagsSelf, releases }),
...checkDescription(description, register),
];
@ -1307,6 +1374,25 @@ async function fetchCatalogNames(register) {
}
}
// The third API call. Releases have no git equivalent — `ls-remote` reports
// refs, and a release is not a ref — so this is the one subject the cheaper
// channel cannot cover. Drafts are excluded: a draft is not published, so it is
// not a claim anyone can read. Pre-releases are kept; llm-ingestion-okf's
// v0.5.0a2 is its real newest release. Null on any failure, which reads as SKIP
// rather than as a pass.
async function fetchReleases(register, repo) {
const url = `${register.forge}/api/v1/repos/${register.org}/${repo}/releases?limit=50`;
try {
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
if (!res.ok) return null;
const json = JSON.parse(await res.text());
if (!Array.isArray(json)) return null;
return json.filter((r) => !r.draft).map((r) => r.tag_name).filter(Boolean);
} catch {
return null;
}
}
async function fetchOrgListing(register) {
const url = `${register.forge}/api/v1/orgs/${register.org}/repos?limit=50`;
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
@ -1447,7 +1533,7 @@ function gitTagObjects(dir) {
}
}
export function inspectRepo(dir, name, register, description, catalogNames = null, offline = false) {
export function inspectRepo(dir, name, register, description, catalogNames = null, offline = false, releases = null) {
const tracked = gitFiles(dir);
const present = (tracked ?? []).filter((f) => existsSync(join(dir, f)));
@ -1467,8 +1553,14 @@ export function inspectRepo(dir, name, register, description, catalogNames = nul
// Only the repos this README actually pins are fetched — one ref listing
// each, and nothing at all for the common case of no pins.
// This repo's own refs, from the forge rather than the clone — the side
// RELEASE-CURRENT compares a published release against, and the side
// REMOTE-SYNC will need next. Over the git protocol, so it costs nothing
// against the API budget.
let forgeTagsSelf = null;
let forgeTagsByRepo = null;
if (!offline) {
forgeTagsSelf = forgeTags(register, name);
forgeTagsByRepo = {};
for (const repo of new Set(extractInstallPins(readme, register).map((p) => p.repo))) {
const t = forgeTags(register, repo);
@ -1488,6 +1580,8 @@ export function inspectRepo(dir, name, register, description, catalogNames = nul
tags: gitTags(dir),
tagObjects: gitTagObjects(dir),
catalogNames,
forgeTagsSelf,
releases,
}, register);
}
@ -1596,8 +1690,10 @@ async function main(argv) {
let description = null;
let catalogNames = null;
let releases = null;
if (!argv.includes('--offline')) {
catalogNames = await fetchCatalogNames(register);
releases = await fetchReleases(register, name);
try {
const listing = await fetchOrgListing(register);
const row = listing.find((r) => r.name === name);
@ -1608,7 +1704,7 @@ async function main(argv) {
}
}
const result = inspectRepo(dir, name, register, description, catalogNames, argv.includes('--offline'));
const result = inspectRepo(dir, name, register, description, catalogNames, argv.includes('--offline'), releases);
const engineVersion = readEngineVersion();
const engineCommit = readEngineCommit();

79
scripts/repo-standard-check.test.mjs
View file

@ -41,6 +41,7 @@ import {
loadRegister,
groupSkips,
skipsOf,
checkReleaseCurrent,
} from './repo-standard-check.mjs';
const REGISTER = {
@ -2005,3 +2006,81 @@ test('headerLine still prints the pre-0.8.0 line for a result with no skip split
const line = headerLine({ name: 'okr', klass: 'plugin', traits: [], status: 'OK', notChecked: 1 }, '0.8.0');
assert.match(line, /1 not checked/);
});
// ---------------------------------------------------------- RELEASE-CURRENT
//
// Levels come from the corpus, not from taste. Measured 2026-08-12 across all
// 22 registered repos (forge releases API vs `git ls-remote --tags`):
// 4 have no tags at all, 2 have tags and publish no releases,
// 11 are current, 5 lag their newest tag.
// A rule that failed the 2 tag-only repos would fail a correct repository —
// the mechanism that got VERSION-DRIFT rejected one check over.
test('a release listing that could not be fetched is a SKIP, never a pass', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: ['v1.0.0'], releases: null });
assert.equal(f[0].level, 'SKIP');
assert.equal(f[0].skip, 'notRun');
});
test('refs that could not be read are a SKIP too — a missing side is not agreement', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: null, releases: ['v1.0.0'] });
assert.equal(f[0].level, 'SKIP');
assert.equal(f[0].skip, 'notRun');
});
test('no tags on the forge is an OK — the check ran and found no subject', () => {
// The VERSION-NONE / TAGS-NONE shape: nothing could have been released, so
// there is nothing here to be wrong. Measured on 4 of 22 repos.
const f = checkReleaseCurrent({ forgeTagsSelf: [], releases: [] });
assert.equal(f[0].level, 'OK');
assert.equal(f[0].code, 'RELEASE-NONE');
});
test('tags with no releases is an OK — tag-only is a convention, not a defect', () => {
// ktg-plugin-marketplace (9 tags) and llm-security-commons (8) publish no
// releases at all. Nothing in a clone says which convention a repo follows.
const f = checkReleaseCurrent({ forgeTagsSelf: ['v0.1.0', 'v0.4.3'], releases: [] });
assert.equal(f[0].level, 'OK');
assert.equal(f[0].code, 'RELEASE-TAGS-ONLY');
assert.match(f[0].msg, /no release/i);
});
test('the newest release matching the newest tag is current', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: ['v7.8.2', 'v7.8.3'], releases: ['v7.8.3'] });
assert.equal(f[0].level, 'OK');
assert.equal(f[0].code, 'RELEASE-CURRENT');
});
test('a release behind the newest tag is a WARN naming both versions', () => {
// repo-standard itself: v0.9.0 tagged, v0.3.0 published. The remedy — publish
// a release for a tag that already exists — is safe, unlike force-moving a
// published ref, so this is a WARN and not an ERROR.
const f = checkReleaseCurrent({ forgeTagsSelf: ['v0.3.0', 'v0.9.0'], releases: ['v0.2.2', 'v0.3.0'] });
assert.equal(f[0].level, 'WARN');
assert.equal(f[0].code, 'RELEASE-STALE');
assert.equal(f[0].bucket, 'weakening');
assert.match(f[0].msg, /v0\.9\.0/);
assert.match(f[0].msg, /v0\.3\.0/);
});
test('newest is version order, not lexical — v0.10.0 outranks v0.9.0', () => {
// The defect this repo already fixed once in compareTags, and reproduced a
// second time in the shell that MEASURED this check's baseline. okr is the
// live case: tag v1.10.0, release v1.9.0.
const stale = checkReleaseCurrent({ forgeTagsSelf: ['v1.9.0', 'v1.10.0'], releases: ['v1.9.0'] });
assert.equal(stale[0].code, 'RELEASE-STALE');
const current = checkReleaseCurrent({ forgeTagsSelf: ['v1.9.0', 'v1.10.0'], releases: ['v1.10.0'] });
assert.equal(current[0].code, 'RELEASE-CURRENT');
});
test('a pre-release tag published as a release is current', () => {
// llm-ingestion-okf sits on v0.5.0a2 in both places. A repo on a pre-release
// could never reach 0 findings if the suffix were truncated away.
const f = checkReleaseCurrent({ forgeTagsSelf: ['v0.4.0', 'v0.5.0a1', 'v0.5.0a2'], releases: ['v0.5.0a2'] });
assert.equal(f[0].code, 'RELEASE-CURRENT');
});
test('a release ahead of every known tag is not reported as stale', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: ['v1.0.0'], releases: ['v1.1.0'] });
assert.equal(f[0].level, 'OK');
});