open/repo-standard
1
0
Fork 0
Code Issues Releases 4 Activity

feat(engine): RELEASE-CURRENT — the version page a stranger actually reads

Browse source 
A release is not a ref, so `git ls-remote` cannot answer this one. It is
therefore the third API call per invocation, and the only new one the
acquisition model adds.

Both sides come from the forge, never the clone: comparing a local tag to a
published release would report portfolio-optimiser as stale when the real
defect is a tag that was never pushed — REMOTE-SYNC's subject, not this one.

Measured across all 22 registered repos before the rule was locked: 4 have no
tags, 2 tag without ever publishing a release, 11 are current, 5 lag. Those 2
are why zero releases is an OK and not a finding — nothing in a repo says which
of the two legitimate conventions it follows, the same measurement that
rejected VERSION-DRIFT. Lagging is a WARN because the remedy is safe, and
because this repo is the org's worst offender: an ERROR would have let the gate
settle an operator question by exiting 1 on its own author.

The shell that produced that baseline sorted tags lexically and put v0.9.0
above v0.10.0 — the exact defect compareTags exists to fix, reproduced in the
tool meant to validate it. Recorded as an invariant: derive "newest" with the
engine's own comparator or the measurement is fiction.

CLAUDE.md's API-call count moves from two to three in this same commit; that
line has now gone stale twice. The unmarked "429 at ~40" figure is marked
unusable — its endpoint class was never recorded, and org-ops measured ~110
raw reads at 0.4s with zero 429 the same day.

205 tests (was 196).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWc4piM4QW6Jxfky2Rw4Z8
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 22:57:35 +02:00
commit a563035feb
4 changed files with 275 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

102
scripts/repo-standard-check.mjs
View file

@ -754,6 +754,72 @@ export function checkTagIntegrity({ tagObjects, name }, register) {
const CLAIM_WORD = /^(tests?|build|ci|coverage|passing)$/i;
const claimsARun = (s) => s.split(/[^a-z0-9]+/i).some((w) => CLAIM_WORD.test(w));
// The forge's release page is where a stranger lands when they want a version
// they can name, and it is the one surface refs cannot answer — a release is
// not a ref, so `git ls-remote` has nothing to report. That is why this is the
// THIRD API call and the only new one the acquisition model adds.
//
// Both sides come from the FORGE, never from the clone. Comparing a local tag
// against a published release would report portfolio-optimiser as having a
// stale release when the real finding is a tag that was never pushed (v1.0.0
// local, v0.1.0 published) — a different defect, owned by a different check.
//
// The levels come from a measurement across all 22 registered repos
// (2026-08-12): 4 have no tags at all, 2 tag without ever publishing a release,
// 11 are current, 5 lag their newest tag. Those 2 are why "no releases" is an
// OK and not a finding: nothing in a repo says which of the two legitimate
// conventions it follows, and a gate that fails a correct repository is the
// mechanism that gets gates switched off — the same measurement that rejected
// VERSION-DRIFT one check over. Lagging is a WARN rather than an ERROR because
// the remedy is safe: publishing a release for a tag that already exists moves
// no published ref, unlike the remedy TAG-ANNOTATED has to withhold.
export function checkReleaseCurrent({ forgeTagsSelf, releases }) {
if (releases === null || releases === undefined) {
return [{ level: 'SKIP', skip: 'notRun', code: 'RELEASE-UNAVAILABLE', msg: 'forge releases not available — check not run (offline, or the listing failed)' }];
}
// One side missing is not agreement. Refs and releases are acquired over two
// different channels, so either can fail alone.
if (forgeTagsSelf === null || forgeTagsSelf === undefined) {
return [{ level: 'SKIP', skip: 'notRun', code: 'RELEASE-UNAVAILABLE', msg: 'forge refs not readable — cannot tell whether the newest release is the newest tag' }];
}
const tags = [...forgeTagsSelf].sort(compareTags);
if (tags.length === 0) {
// The VERSION-NONE shape: the check ran, saw every tag the forge has, and
// found no subject. Nothing could have been released, so there is nothing
// here to be wrong — a verdict, not an absent one.
return [{ level: 'OK', code: 'RELEASE-NONE', msg: 'no tags on the forge — no release could exist' }];
}
const newestTag = tags[tags.length - 1];
if (releases.length === 0) {
return [{
level: 'OK',
code: 'RELEASE-TAGS-ONLY',
msg: `${tags.length} tag(s) and no release published — this repo tags without publishing releases, which is a convention this gate does not judge`,
}];
}
// Version order, not the order the API handed them over: the releases listing
// sorts by creation time, and a patch cut after a minor would read as newest.
const newestRelease = [...releases].sort(compareTags).pop();
if (compareTags(newestRelease, newestTag) < 0) {
return [{
level: 'WARN',
code: 'RELEASE-STALE',
bucket: 'weakening',
msg: `newest release is \`${newestRelease}\` but the newest tag is \`${newestTag}\` — the release page shows a version older than the code. Publish a release for \`${newestTag}\`.`,
}];
}
return [{
level: 'OK',
code: 'RELEASE-CURRENT',
msg: newestRelease === newestTag
? `newest release \`${newestRelease}\` is the newest tag`
: `newest release \`${newestRelease}\` is ahead of every tag the forge lists`,
}];
}
// Counting badges needs a NARROWER rule than detecting a dishonest one. The
// claim check reads any image, any host, on purpose. Here the opposite error
// matters: counting a screenshot or an architecture diagram as clutter would
@ -1191,7 +1257,7 @@ export function bucketsOf(findings) {
}
export function classifyRepo(
{ name, files, present, description, pluginVersion, readmeBadge, changelogTop, tags, tagObjects, catalogNames, forgeTagsByRepo },
{ name, files, present, description, pluginVersion, readmeBadge, changelogTop, tags, tagObjects, catalogNames, forgeTagsByRepo, forgeTagsSelf, releases },
register,
) {
const klass = register.repos?.[name];
@ -1230,6 +1296,7 @@ export function classifyRepo(
...checkBoilerplate({ files }),
...checkVersionConsistency({ pluginVersion, readmeBadge, changelogTop, tags }),
...checkTagIntegrity({ tagObjects, name }, register),
...checkReleaseCurrent({ forgeTagsSelf, releases }),
...checkDescription(description, register),
];
@ -1307,6 +1374,25 @@ async function fetchCatalogNames(register) {
}
}
// The third API call. Releases have no git equivalent — `ls-remote` reports
// refs, and a release is not a ref — so this is the one subject the cheaper
// channel cannot cover. Drafts are excluded: a draft is not published, so it is
// not a claim anyone can read. Pre-releases are kept; llm-ingestion-okf's
// v0.5.0a2 is its real newest release. Null on any failure, which reads as SKIP
// rather than as a pass.
async function fetchReleases(register, repo) {
const url = `${register.forge}/api/v1/repos/${register.org}/${repo}/releases?limit=50`;
try {
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
if (!res.ok) return null;
const json = JSON.parse(await res.text());
if (!Array.isArray(json)) return null;
return json.filter((r) => !r.draft).map((r) => r.tag_name).filter(Boolean);
} catch {
return null;
}
}
async function fetchOrgListing(register) {
const url = `${register.forge}/api/v1/orgs/${register.org}/repos?limit=50`;
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
@ -1447,7 +1533,7 @@ function gitTagObjects(dir) {
}
}
export function inspectRepo(dir, name, register, description, catalogNames = null, offline = false) {
export function inspectRepo(dir, name, register, description, catalogNames = null, offline = false, releases = null) {
const tracked = gitFiles(dir);
const present = (tracked ?? []).filter((f) => existsSync(join(dir, f)));
@ -1467,8 +1553,14 @@ export function inspectRepo(dir, name, register, description, catalogNames = nul
// Only the repos this README actually pins are fetched — one ref listing
// each, and nothing at all for the common case of no pins.
// This repo's own refs, from the forge rather than the clone — the side
// RELEASE-CURRENT compares a published release against, and the side
// REMOTE-SYNC will need next. Over the git protocol, so it costs nothing
// against the API budget.
let forgeTagsSelf = null;
let forgeTagsByRepo = null;
if (!offline) {
forgeTagsSelf = forgeTags(register, name);
forgeTagsByRepo = {};
for (const repo of new Set(extractInstallPins(readme, register).map((p) => p.repo))) {
const t = forgeTags(register, repo);
@ -1488,6 +1580,8 @@ export function inspectRepo(dir, name, register, description, catalogNames = nul
tags: gitTags(dir),
tagObjects: gitTagObjects(dir),
catalogNames,
forgeTagsSelf,
releases,
}, register);
}
@ -1596,8 +1690,10 @@ async function main(argv) {
let description = null;
let catalogNames = null;
let releases = null;
if (!argv.includes('--offline')) {
catalogNames = await fetchCatalogNames(register);
releases = await fetchReleases(register, name);
try {
const listing = await fetchOrgListing(register);
const row = listing.find((r) => r.name === name);
@ -1608,7 +1704,7 @@ async function main(argv) {
}
}
const result = inspectRepo(dir, name, register, description, catalogNames, argv.includes('--offline'));
const result = inspectRepo(dir, name, register, description, catalogNames, argv.includes('--offline'), releases);
const engineVersion = readEngineVersion();
const engineCommit = readEngineCommit();

79
scripts/repo-standard-check.test.mjs
View file

@ -41,6 +41,7 @@ import {
loadRegister,
groupSkips,
skipsOf,
checkReleaseCurrent,
} from './repo-standard-check.mjs';
const REGISTER = {
@ -2005,3 +2006,81 @@ test('headerLine still prints the pre-0.8.0 line for a result with no skip split
const line = headerLine({ name: 'okr', klass: 'plugin', traits: [], status: 'OK', notChecked: 1 }, '0.8.0');
assert.match(line, /1 not checked/);
});
// ---------------------------------------------------------- RELEASE-CURRENT
//
// Levels come from the corpus, not from taste. Measured 2026-08-12 across all
// 22 registered repos (forge releases API vs `git ls-remote --tags`):
// 4 have no tags at all, 2 have tags and publish no releases,
// 11 are current, 5 lag their newest tag.
// A rule that failed the 2 tag-only repos would fail a correct repository —
// the mechanism that got VERSION-DRIFT rejected one check over.
test('a release listing that could not be fetched is a SKIP, never a pass', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: ['v1.0.0'], releases: null });
assert.equal(f[0].level, 'SKIP');
assert.equal(f[0].skip, 'notRun');
});
test('refs that could not be read are a SKIP too — a missing side is not agreement', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: null, releases: ['v1.0.0'] });
assert.equal(f[0].level, 'SKIP');
assert.equal(f[0].skip, 'notRun');
});
test('no tags on the forge is an OK — the check ran and found no subject', () => {
// The VERSION-NONE / TAGS-NONE shape: nothing could have been released, so
// there is nothing here to be wrong. Measured on 4 of 22 repos.
const f = checkReleaseCurrent({ forgeTagsSelf: [], releases: [] });
assert.equal(f[0].level, 'OK');
assert.equal(f[0].code, 'RELEASE-NONE');
});
test('tags with no releases is an OK — tag-only is a convention, not a defect', () => {
// ktg-plugin-marketplace (9 tags) and llm-security-commons (8) publish no
// releases at all. Nothing in a clone says which convention a repo follows.
const f = checkReleaseCurrent({ forgeTagsSelf: ['v0.1.0', 'v0.4.3'], releases: [] });
assert.equal(f[0].level, 'OK');
assert.equal(f[0].code, 'RELEASE-TAGS-ONLY');
assert.match(f[0].msg, /no release/i);
});
test('the newest release matching the newest tag is current', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: ['v7.8.2', 'v7.8.3'], releases: ['v7.8.3'] });
assert.equal(f[0].level, 'OK');
assert.equal(f[0].code, 'RELEASE-CURRENT');
});
test('a release behind the newest tag is a WARN naming both versions', () => {
// repo-standard itself: v0.9.0 tagged, v0.3.0 published. The remedy — publish
// a release for a tag that already exists — is safe, unlike force-moving a
// published ref, so this is a WARN and not an ERROR.
const f = checkReleaseCurrent({ forgeTagsSelf: ['v0.3.0', 'v0.9.0'], releases: ['v0.2.2', 'v0.3.0'] });
assert.equal(f[0].level, 'WARN');
assert.equal(f[0].code, 'RELEASE-STALE');
assert.equal(f[0].bucket, 'weakening');
assert.match(f[0].msg, /v0\.9\.0/);
assert.match(f[0].msg, /v0\.3\.0/);
});
test('newest is version order, not lexical — v0.10.0 outranks v0.9.0', () => {
// The defect this repo already fixed once in compareTags, and reproduced a
// second time in the shell that MEASURED this check's baseline. okr is the
// live case: tag v1.10.0, release v1.9.0.
const stale = checkReleaseCurrent({ forgeTagsSelf: ['v1.9.0', 'v1.10.0'], releases: ['v1.9.0'] });
assert.equal(stale[0].code, 'RELEASE-STALE');
const current = checkReleaseCurrent({ forgeTagsSelf: ['v1.9.0', 'v1.10.0'], releases: ['v1.10.0'] });
assert.equal(current[0].code, 'RELEASE-CURRENT');
});
test('a pre-release tag published as a release is current', () => {
// llm-ingestion-okf sits on v0.5.0a2 in both places. A repo on a pre-release
// could never reach 0 findings if the suffix were truncated away.
const f = checkReleaseCurrent({ forgeTagsSelf: ['v0.4.0', 'v0.5.0a1', 'v0.5.0a2'], releases: ['v0.5.0a2'] });
assert.equal(f[0].code, 'RELEASE-CURRENT');
});
test('a release ahead of every known tag is not reported as stale', () => {
const f = checkReleaseCurrent({ forgeTagsSelf: ['v1.0.0'], releases: ['v1.1.0'] });
assert.equal(f[0].level, 'OK');
});