feat(engine)!: "no version to compare against" is a verdict, not a skip

`VERSION-UNAVAILABLE` (SKIP/notRun) becomes `VERSION-NONE` (OK). A repo that
claims no version in any of the four places one can be written down has not
left a check un-run — the check ran, saw all four, and found no subject. That
is the shape `checkReadmeLanguage` has always answered with OK.

0.8.0 deferred this and recorded the reason in the engine: re-levelling "moves
a repo's status". Measured false across 19 clones — an added OK cannot worsen
the worst *judged* finding, and all three repos emitting the code already read
OK. Status moves only for a repo whose entire finding set was skips.

OK cannot bless a real gap here: no class requires a version file, and a
`plugin` missing its manifest is an independent FILE-MISSING ERROR.

Renamed, not just re-levelled: `-UNAVAILABLE` is this engine's naming for a
notRun skip, and one OK-level `-UNAVAILABLE` would mislead exactly the reader
this change serves. A source scan keeps the old name out.

A/B sweep, both engines offline, 19 clones: 223 judged findings, every status
and buckets identical; 3 conversions (notChecked 14→13, 3→2, 14→13), byDesign
untouched. 170 tests. `--refresh`: register 21 = forge 21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CTNGvxzCLZp3UXfibYG4kb
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 22:05:21 +02:00
commit e30f54bb64
8 changed files with 99 additions and 21 deletions

View file

@ -1,6 +1,6 @@
{ {
"name": "repo-standard", "name": "repo-standard",
"version": "0.8.1", "version": "0.9.0",
"description": "Per-repo gate for the open/ presentation standard: README first screen, install block, files required by the repo's class, and dead repo references.", "description": "Per-repo gate for the open/ presentation standard: README first screen, install block, files required by the repo's class, and dead repo references.",
"author": { "author": {
"name": "Kjell Tore Guttormsen" "name": "Kjell Tore Guttormsen"

View file

@ -4,6 +4,42 @@ All notable changes to this project are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html). versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.9.0] — 2026-08-09
### Changed
- **`VERSION-UNAVAILABLE` (`SKIP`/`notRun`) is now `VERSION-NONE` at `OK`.** A
repository that claims no version in any of the four places one can be written
down has not left a check un-run — the check ran, saw all four, and found no
subject. That is the shape `checkReadmeLanguage` has always answered with `OK`
("no running prose to judge"), and filing it as a skip told three
correctly-built repositories that something about them was unread.
0.8.0 deferred this deliberately and recorded the reason in the engine:
re-levelling "moves a repo's status". That was measured false. An added `OK`
cannot worsen the worst *judged* finding, so `status` moves only for a repo
whose entire finding set was skips — and all three repos that emit this code
(`ktg-plugin-marketplace`, `playground-design-system`,
`portfolio-optimiser-commons`) already read `OK`. The comment is gone with the
code it justified.
Nor can `OK` here bless a real gap: no class in the register requires a
version file, and a `plugin` missing its manifest is an independent
`FILE-MISSING` `ERROR`.
**The code was renamed, not just re-levelled.** `-UNAVAILABLE` is this
engine's naming for a `notRun` skip (`DESC-UNAVAILABLE`), and one `OK`-level
`-UNAVAILABLE` would mislead exactly the reader this change is for. A source
scan holds the old name out of the engine, so a re-introduction fails the
suite rather than the next census.
Verified before release: 170 tests; 19 clones swept with both engines offline,
223 judged findings, every `status` and `buckets` identical. Exactly three
findings converted — `notChecked` 14→13, 3→2, 14→13, `notRun` 3→2 in each, and
`byDesign` untouched. No repository's `status` moved. Remaining `notRun` across
the org is actionable by construction: `INSTALL-TRUTH` (12), `VERSION-TAG` (1),
plus the two that only appear under `--offline`.
## [0.8.1] — 2026-08-09 ## [0.8.1] — 2026-08-09
### Fixed ### Fixed

View file

@ -54,6 +54,21 @@ would recreate, in data, exactly the drift this plugin exists to remove.
the symptom untouched. The test that holds this is a scan of the engine the symptom untouched. The test that holds this is a scan of the engine
SOURCE, not of a finding set: a corpus test only sees the sites it manages to SOURCE, not of a finding set: a corpus test only sees the sites it manages to
trigger, and the next site added would take the reader's default invisibly. trigger, and the next site added would take the reader's default invisibly.
- **"Nothing to judge" is an `OK`, not a skip — and it is a THIRD shape, not a
third bucket.** A check that ran, saw everything, and found no subject has
produced a verdict: there is nothing here to be wrong. `checkReadmeLanguage`
already answered that way for a README with no prose; `VERSION-NONE` (0.9.0,
was `VERSION-UNAVAILABLE`/`notRun`) does for a repo that claims no version.
The deferral that kept it a skip for one release was recorded in the engine as
"re-levelling moves a repo's status" — measured false: an added `OK` cannot
worsen the worst *judged* finding, and all three affected repos already read
`OK`. Status moves only for a repo whose *entire* finding set was skips. The
lesson is not about this code: **a recorded decision that is wrong is worse
than no record**, because the next session re-derives the measurement instead
of reading it. And the guard against `OK` blessing a real gap is that the gap
is caught elsewhere — no class requires a version file, and a `plugin` missing
its manifest is an independent `FILE-MISSING` `ERROR`. Without that, this
would have been "SKIP is never a pass" one level up.
- **When a check fires many times in one repo, suspect the CHECK.** The first - **When a check fires many times in one repo, suspect the CHECK.** The first
link pass produced ~30 findings against `llm-security` and all were noise — link pass produced ~30 findings against `llm-security` and all were noise —
regexes in code spans, `file:` URLs, relative paths resolved against the wrong regexes in code spans, `file:` URLs, relative paths resolved against the wrong
@ -149,7 +164,7 @@ would recreate, in data, exactly the drift this plugin exists to remove.
## Commands ## Commands
```bash ```bash
npm test # 169 tests npm test # 170 tests
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
node scripts/repo-standard-check.mjs --offline # no network call node scripts/repo-standard-check.mjs --offline # no network call
node scripts/repo-standard-check.mjs --json # machine output node scripts/repo-standard-check.mjs --json # machine output

View file

@ -10,7 +10,7 @@ checks that surface in one repository and reports what it finds.
*AI-generated: all code produced by Claude Code through dialog-driven development.* *AI-generated: all code produced by Claude Code through dialog-driven development.*
![Version](https://img.shields.io/badge/version-0.8.1-blue) ![Version](https://img.shields.io/badge/version-0.9.0-blue)
![Platform](https://img.shields.io/badge/platform-Claude_Code_Plugin-purple) ![Platform](https://img.shields.io/badge/platform-Claude_Code_Plugin-purple)
![Skills](https://img.shields.io/badge/skills-1-orange) ![Skills](https://img.shields.io/badge/skills-1-orange)
![License](https://img.shields.io/badge/license-MIT-lightgrey) ![License](https://img.shields.io/badge/license-MIT-lightgrey)
@ -103,12 +103,20 @@ Merging them made clean repositories read as unread. Measured across 19 clones,
checked` announced eleven unread checks when all eleven were links the gate checked` announced eleven unread checks when all eleven were links the gate
declines to judge on purpose. declines to judge on purpose.
A third shape is not a skip at all and must not be filed as one: the check ran,
saw everything, and found **no subject** to judge. A repository that claims no
version anywhere is `VERSION-NONE` at `OK` since 0.9.0 — the same answer
`LANGUAGE` gives a README with no running prose. It had been `notRun`, which
told three correctly-built repositories that something about them was still
unread. What remains under `notRun` is genuinely actionable: an unreachable
catalog, an unregistered repo, a repo that has not been tagged yet.
The summary line names only what someone has an action on: The summary line names only what someone has an action on:
``` ```
✓ portfolio-optimiser [standalone] — OK (repo-standard v0.8.1 @<sha>) ✓ portfolio-optimiser [standalone] — OK (repo-standard v0.9.0 @<sha>)
✗ some-plugin [plugin] — ERROR · 2 not run (repo-standard v0.8.1 @<sha>) ✗ some-plugin [plugin] — ERROR · 2 not run (repo-standard v0.9.0 @<sha>)
· stranger — SKIP · 1 not run (repo-standard v0.8.1 @<sha>) · stranger — SKIP · 1 not run (repo-standard v0.9.0 @<sha>)
``` ```
The deliberate ones are not silenced: they keep their own `NOT JUDGED` heading The deliberate ones are not silenced: they keep their own `NOT JUDGED` heading
@ -259,7 +267,7 @@ distinct from "an older engine that never tried".
npm test npm test
``` ```
169 tests over the pure classifiers. The reference fixtures are measured false 170 tests over the pure classifiers. The reference fixtures are measured false
positives, each with its expected verdict — the six that produced the positives, each with its expected verdict — the six that produced the
three-outcome reference rule, plus the noise sources found by running the gate three-outcome reference rule, plus the noise sources found by running the gate
against a real repository: regexes inside code spans that are markdown links to against a real repository: regexes inside code spans that are markdown links to

View file

@ -1,6 +1,6 @@
{ {
"name": "repo-standard", "name": "repo-standard",
"version": "0.8.1", "version": "0.9.0",
"private": true, "private": true,
"type": "module", "type": "module",
"engines": { "engines": {

View file

@ -517,13 +517,15 @@ export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogT
const findings = []; const findings = [];
const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null; const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null;
if (!v) { if (!v) {
// `notRun` by operator decision 2026-08-09, not by default. "No package // Not a SKIP. This check ran, saw all four places a version can be written
// version to compare against" is arguably a third thing — no SUBJECT to // down, and found no version claimed in any of them — no SUBJECT to judge,
// judge, the shape checkReadmeLanguage answers with OK — but re-levelling // the shape checkReadmeLanguage answers with OK. It sat at `SKIP`/`notRun`
// it moves a repo's status, and that is a second behaviour change riding // until 0.9.0, deferred once on the ground that re-levelling would move a
// on this one. Recorded here so the next reader knows it was decided, not // repo's status. That was measured false: an added OK cannot worsen the
// overlooked. // worst *judged* finding, and all three repos that reach this line already
return [{ level: 'SKIP', skip: 'notRun', code: 'VERSION-UNAVAILABLE', msg: 'no package version found — nothing to compare against' }]; // read OK. Nor can OK bless a real gap — no class requires a version file,
// and a `plugin` missing its manifest is an independent FILE-MISSING ERROR.
return [{ level: 'OK', code: 'VERSION-NONE', msg: 'no version claimed anywhere — nothing to check' }];
} }
if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) { if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) {

View file

@ -1646,13 +1646,24 @@ test('no tags yet is un-runnable — a release resolves it', () => {
assert.equal(f.find((x) => x.code === 'VERSION-TAG').skip, 'notRun'); assert.equal(f.find((x) => x.code === 'VERSION-TAG').skip, 'notRun');
}); });
// Kept in `notRun` deliberately (operator decision 2026-08-09). It is arguably // Was `SKIP`/`notRun` until 0.9.0, deferred once on the stated ground that
// a third thing — "no subject to judge", the shape `checkReadmeLanguage` calls // re-levelling "moves a repo's status". Measured false: an added `OK` cannot
// OK rather than SKIP — but re-levelling it moves `status` for a repo and is a // worsen the worst *judged* finding, and all three repos that emit this already
// second behaviour change; it does not ride on this one. // read `OK`. It is the `checkReadmeLanguage` shape — the check ran, saw
test('no package version is un-runnable, not deliberate', () => { // everything, and found no subject.
test('no version claimed anywhere is nothing to judge, not a skip', () => {
const f = checkVersionConsistency({ pluginVersion: null, readmeBadge: null, changelogTop: null, tags: [] }); const f = checkVersionConsistency({ pluginVersion: null, readmeBadge: null, changelogTop: null, tags: [] });
assert.equal(f.find((x) => x.code === 'VERSION-UNAVAILABLE').skip, 'notRun'); const hit = f.find((x) => x.code === 'VERSION-NONE');
assert.equal(hit.level, 'OK');
assert.equal(hit.skip, undefined);
});
// The rename is the point, not cosmetics: `-UNAVAILABLE` is this engine's
// naming for a `notRun` skip (`DESC-UNAVAILABLE`, `INSTALL-TRUTH`). One
// OK-level `-UNAVAILABLE` would mislead exactly the reader this change serves.
test('the un-runnable version codes keep the -UNAVAILABLE naming', () => {
const src = readFileSync(new URL('./repo-standard-check.mjs', import.meta.url), 'utf8');
assert.ok(!src.includes('VERSION-UNAVAILABLE'), 'VERSION-UNAVAILABLE was renamed to VERSION-NONE');
}); });
test('an unreachable catalog is un-runnable — a second run resolves it', () => { test('an unreachable catalog is un-runnable — a second run resolves it', () => {

View file

@ -43,6 +43,12 @@ not interchangeable:
Nobody has an action. Reporting these as outstanding work is how a clean repo Nobody has an action. Reporting these as outstanding work is how a clean repo
gets rewritten for nothing. gets rewritten for nothing.
The inverse error is reporting an `OK` as an un-run check. A check that ran and
found **no subject** is a verdict, not a skip: `VERSION-NONE` ("no version
claimed anywhere") and `LANGUAGE` ("no running prose to judge") are `OK` and
carry no action. `VERSION-NONE` was `VERSION-UNAVAILABLE`/`notRun` before 0.9.0
— if you see the old code, the engine is stale; see the version note below.
That applies hardest when the repo's own status is green. Since 0.7.0 a `SKIP` That applies hardest when the repo's own status is green. Since 0.7.0 a `SKIP`
no longer outranks an `OK` in the status line, so a repo reads `OK · 3 not run` no longer outranks an `OK` in the status line, so a repo reads `OK · 3 not run`
rather than `SKIP`. The summary line carries only the `notRun` count — a repo rather than `SKIP`. The summary line carries only the `notRun` count — a repo