feat(engine)!: "no version to compare against" is a verdict, not a skip

`VERSION-UNAVAILABLE` (SKIP/notRun) becomes `VERSION-NONE` (OK). A repo that
claims no version in any of the four places one can be written down has not
left a check un-run — the check ran, saw all four, and found no subject. That
is the shape `checkReadmeLanguage` has always answered with OK.

0.8.0 deferred this and recorded the reason in the engine: re-levelling "moves
a repo's status". Measured false across 19 clones — an added OK cannot worsen
the worst *judged* finding, and all three repos emitting the code already read
OK. Status moves only for a repo whose entire finding set was skips.

OK cannot bless a real gap here: no class requires a version file, and a
`plugin` missing its manifest is an independent FILE-MISSING ERROR.

Renamed, not just re-levelled: `-UNAVAILABLE` is this engine's naming for a
notRun skip, and one OK-level `-UNAVAILABLE` would mislead exactly the reader
this change serves. A source scan keeps the old name out.

A/B sweep, both engines offline, 19 clones: 223 judged findings, every status
and buckets identical; 3 conversions (notChecked 14→13, 3→2, 14→13), byDesign
untouched. 170 tests. `--refresh`: register 21 = forge 21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CTNGvxzCLZp3UXfibYG4kb
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 22:05:21 +02:00
commit 1dfd408ffc
8 changed files with 99 additions and 21 deletions

View file

@ -517,13 +517,15 @@ export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogT
const findings = [];
const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null;
if (!v) {
// `notRun` by operator decision 2026-08-09, not by default. "No package
// version to compare against" is arguably a third thing — no SUBJECT to
// judge, the shape checkReadmeLanguage answers with OK — but re-levelling
// it moves a repo's status, and that is a second behaviour change riding
// on this one. Recorded here so the next reader knows it was decided, not
// overlooked.
return [{ level: 'SKIP', skip: 'notRun', code: 'VERSION-UNAVAILABLE', msg: 'no package version found — nothing to compare against' }];
// Not a SKIP. This check ran, saw all four places a version can be written
// down, and found no version claimed in any of them — no SUBJECT to judge,
// the shape checkReadmeLanguage answers with OK. It sat at `SKIP`/`notRun`
// until 0.9.0, deferred once on the ground that re-levelling would move a
// repo's status. That was measured false: an added OK cannot worsen the
// worst *judged* finding, and all three repos that reach this line already
// read OK. Nor can OK bless a real gap — no class requires a version file,
// and a `plugin` missing its manifest is an independent FILE-MISSING ERROR.
return [{ level: 'OK', code: 'VERSION-NONE', msg: 'no version claimed anywhere — nothing to check' }];
}
if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) {

View file

@ -1646,13 +1646,24 @@ test('no tags yet is un-runnable — a release resolves it', () => {
assert.equal(f.find((x) => x.code === 'VERSION-TAG').skip, 'notRun');
});
// Kept in `notRun` deliberately (operator decision 2026-08-09). It is arguably
// a third thing — "no subject to judge", the shape `checkReadmeLanguage` calls
// OK rather than SKIP — but re-levelling it moves `status` for a repo and is a
// second behaviour change; it does not ride on this one.
test('no package version is un-runnable, not deliberate', () => {
// Was `SKIP`/`notRun` until 0.9.0, deferred once on the stated ground that
// re-levelling "moves a repo's status". Measured false: an added `OK` cannot
// worsen the worst *judged* finding, and all three repos that emit this already
// read `OK`. It is the `checkReadmeLanguage` shape — the check ran, saw
// everything, and found no subject.
test('no version claimed anywhere is nothing to judge, not a skip', () => {
const f = checkVersionConsistency({ pluginVersion: null, readmeBadge: null, changelogTop: null, tags: [] });
assert.equal(f.find((x) => x.code === 'VERSION-UNAVAILABLE').skip, 'notRun');
const hit = f.find((x) => x.code === 'VERSION-NONE');
assert.equal(hit.level, 'OK');
assert.equal(hit.skip, undefined);
});
// The rename is the point, not cosmetics: `-UNAVAILABLE` is this engine's
// naming for a `notRun` skip (`DESC-UNAVAILABLE`, `INSTALL-TRUTH`). One
// OK-level `-UNAVAILABLE` would mislead exactly the reader this change serves.
test('the un-runnable version codes keep the -UNAVAILABLE naming', () => {
const src = readFileSync(new URL('./repo-standard-check.mjs', import.meta.url), 'utf8');
assert.ok(!src.includes('VERSION-UNAVAILABLE'), 'VERSION-UNAVAILABLE was renamed to VERSION-NONE');
});
test('an unreachable catalog is un-runnable — a second run resolves it', () => {