feat(engine)!: "no version to compare against" is a verdict, not a skip

`VERSION-UNAVAILABLE` (SKIP/notRun) becomes `VERSION-NONE` (OK). A repo that
claims no version in any of the four places one can be written down has not
left a check un-run — the check ran, saw all four, and found no subject. That
is the shape `checkReadmeLanguage` has always answered with OK.

0.8.0 deferred this and recorded the reason in the engine: re-levelling "moves
a repo's status". Measured false across 19 clones — an added OK cannot worsen
the worst *judged* finding, and all three repos emitting the code already read
OK. Status moves only for a repo whose entire finding set was skips.

OK cannot bless a real gap here: no class requires a version file, and a
`plugin` missing its manifest is an independent FILE-MISSING ERROR.

Renamed, not just re-levelled: `-UNAVAILABLE` is this engine's naming for a
notRun skip, and one OK-level `-UNAVAILABLE` would mislead exactly the reader
this change serves. A source scan keeps the old name out.

A/B sweep, both engines offline, 19 clones: 223 judged findings, every status
and buckets identical; 3 conversions (notChecked 14→13, 3→2, 14→13), byDesign
untouched. 170 tests. `--refresh`: register 21 = forge 21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CTNGvxzCLZp3UXfibYG4kb
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 22:05:21 +02:00
commit 1dfd408ffc
8 changed files with 99 additions and 21 deletions

View file

@ -1,6 +1,6 @@
{
"name": "repo-standard",
"version": "0.8.1",
"version": "0.9.0",
"description": "Per-repo gate for the open/ presentation standard: README first screen, install block, files required by the repo's class, and dead repo references.",
"author": {
"name": "Kjell Tore Guttormsen"

View file

@ -4,6 +4,42 @@ All notable changes to this project are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.9.0] — 2026-08-09
### Changed
- **`VERSION-UNAVAILABLE` (`SKIP`/`notRun`) is now `VERSION-NONE` at `OK`.** A
repository that claims no version in any of the four places one can be written
down has not left a check un-run — the check ran, saw all four, and found no
subject. That is the shape `checkReadmeLanguage` has always answered with `OK`
("no running prose to judge"), and filing it as a skip told three
correctly-built repositories that something about them was unread.
0.8.0 deferred this deliberately and recorded the reason in the engine:
re-levelling "moves a repo's status". That was measured false. An added `OK`
cannot worsen the worst *judged* finding, so `status` moves only for a repo
whose entire finding set was skips — and all three repos that emit this code
(`ktg-plugin-marketplace`, `playground-design-system`,
`portfolio-optimiser-commons`) already read `OK`. The comment is gone with the
code it justified.
Nor can `OK` here bless a real gap: no class in the register requires a
version file, and a `plugin` missing its manifest is an independent
`FILE-MISSING` `ERROR`.
**The code was renamed, not just re-levelled.** `-UNAVAILABLE` is this
engine's naming for a `notRun` skip (`DESC-UNAVAILABLE`), and one `OK`-level
`-UNAVAILABLE` would mislead exactly the reader this change is for. A source
scan holds the old name out of the engine, so a re-introduction fails the
suite rather than the next census.
Verified before release: 170 tests; 19 clones swept with both engines offline,
223 judged findings, every `status` and `buckets` identical. Exactly three
findings converted — `notChecked` 14→13, 3→2, 14→13, `notRun` 3→2 in each, and
`byDesign` untouched. No repository's `status` moved. Remaining `notRun` across
the org is actionable by construction: `INSTALL-TRUTH` (12), `VERSION-TAG` (1),
plus the two that only appear under `--offline`.
## [0.8.1] — 2026-08-09
### Fixed

View file

@ -54,6 +54,21 @@ would recreate, in data, exactly the drift this plugin exists to remove.
the symptom untouched. The test that holds this is a scan of the engine
SOURCE, not of a finding set: a corpus test only sees the sites it manages to
trigger, and the next site added would take the reader's default invisibly.
- **"Nothing to judge" is an `OK`, not a skip — and it is a THIRD shape, not a
third bucket.** A check that ran, saw everything, and found no subject has
produced a verdict: there is nothing here to be wrong. `checkReadmeLanguage`
already answered that way for a README with no prose; `VERSION-NONE` (0.9.0,
was `VERSION-UNAVAILABLE`/`notRun`) does for a repo that claims no version.
The deferral that kept it a skip for one release was recorded in the engine as
"re-levelling moves a repo's status" — measured false: an added `OK` cannot
worsen the worst *judged* finding, and all three affected repos already read
`OK`. Status moves only for a repo whose *entire* finding set was skips. The
lesson is not about this code: **a recorded decision that is wrong is worse
than no record**, because the next session re-derives the measurement instead
of reading it. And the guard against `OK` blessing a real gap is that the gap
is caught elsewhere — no class requires a version file, and a `plugin` missing
its manifest is an independent `FILE-MISSING` `ERROR`. Without that, this
would have been "SKIP is never a pass" one level up.
- **When a check fires many times in one repo, suspect the CHECK.** The first
link pass produced ~30 findings against `llm-security` and all were noise —
regexes in code spans, `file:` URLs, relative paths resolved against the wrong
@ -149,7 +164,7 @@ would recreate, in data, exactly the drift this plugin exists to remove.
## Commands
```bash
npm test # 169 tests
npm test # 170 tests
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
node scripts/repo-standard-check.mjs --offline # no network call
node scripts/repo-standard-check.mjs --json # machine output

View file

@ -10,7 +10,7 @@ checks that surface in one repository and reports what it finds.
*AI-generated: all code produced by Claude Code through dialog-driven development.*
![Version](https://img.shields.io/badge/version-0.8.1-blue)
![Version](https://img.shields.io/badge/version-0.9.0-blue)
![Platform](https://img.shields.io/badge/platform-Claude_Code_Plugin-purple)
![Skills](https://img.shields.io/badge/skills-1-orange)
![License](https://img.shields.io/badge/license-MIT-lightgrey)
@ -103,12 +103,20 @@ Merging them made clean repositories read as unread. Measured across 19 clones,
checked` announced eleven unread checks when all eleven were links the gate
declines to judge on purpose.
A third shape is not a skip at all and must not be filed as one: the check ran,
saw everything, and found **no subject** to judge. A repository that claims no
version anywhere is `VERSION-NONE` at `OK` since 0.9.0 — the same answer
`LANGUAGE` gives a README with no running prose. It had been `notRun`, which
told three correctly-built repositories that something about them was still
unread. What remains under `notRun` is genuinely actionable: an unreachable
catalog, an unregistered repo, a repo that has not been tagged yet.
The summary line names only what someone has an action on:
```
✓ portfolio-optimiser [standalone] — OK (repo-standard v0.8.1 @<sha>)
✗ some-plugin [plugin] — ERROR · 2 not run (repo-standard v0.8.1 @<sha>)
· stranger — SKIP · 1 not run (repo-standard v0.8.1 @<sha>)
✓ portfolio-optimiser [standalone] — OK (repo-standard v0.9.0 @<sha>)
✗ some-plugin [plugin] — ERROR · 2 not run (repo-standard v0.9.0 @<sha>)
· stranger — SKIP · 1 not run (repo-standard v0.9.0 @<sha>)
```
The deliberate ones are not silenced: they keep their own `NOT JUDGED` heading
@ -259,7 +267,7 @@ distinct from "an older engine that never tried".
npm test
```
169 tests over the pure classifiers. The reference fixtures are measured false
170 tests over the pure classifiers. The reference fixtures are measured false
positives, each with its expected verdict — the six that produced the
three-outcome reference rule, plus the noise sources found by running the gate
against a real repository: regexes inside code spans that are markdown links to

View file

@ -1,6 +1,6 @@
{
"name": "repo-standard",
"version": "0.8.1",
"version": "0.9.0",
"private": true,
"type": "module",
"engines": {

View file

@ -517,13 +517,15 @@ export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogT
const findings = [];
const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null;
if (!v) {
// `notRun` by operator decision 2026-08-09, not by default. "No package
// version to compare against" is arguably a third thing — no SUBJECT to
// judge, the shape checkReadmeLanguage answers with OK — but re-levelling
// it moves a repo's status, and that is a second behaviour change riding
// on this one. Recorded here so the next reader knows it was decided, not
// overlooked.
return [{ level: 'SKIP', skip: 'notRun', code: 'VERSION-UNAVAILABLE', msg: 'no package version found — nothing to compare against' }];
// Not a SKIP. This check ran, saw all four places a version can be written
// down, and found no version claimed in any of them — no SUBJECT to judge,
// the shape checkReadmeLanguage answers with OK. It sat at `SKIP`/`notRun`
// until 0.9.0, deferred once on the ground that re-levelling would move a
// repo's status. That was measured false: an added OK cannot worsen the
// worst *judged* finding, and all three repos that reach this line already
// read OK. Nor can OK bless a real gap — no class requires a version file,
// and a `plugin` missing its manifest is an independent FILE-MISSING ERROR.
return [{ level: 'OK', code: 'VERSION-NONE', msg: 'no version claimed anywhere — nothing to check' }];
}
if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) {

View file

@ -1646,13 +1646,24 @@ test('no tags yet is un-runnable — a release resolves it', () => {
assert.equal(f.find((x) => x.code === 'VERSION-TAG').skip, 'notRun');
});
// Kept in `notRun` deliberately (operator decision 2026-08-09). It is arguably
// a third thing — "no subject to judge", the shape `checkReadmeLanguage` calls
// OK rather than SKIP — but re-levelling it moves `status` for a repo and is a
// second behaviour change; it does not ride on this one.
test('no package version is un-runnable, not deliberate', () => {
// Was `SKIP`/`notRun` until 0.9.0, deferred once on the stated ground that
// re-levelling "moves a repo's status". Measured false: an added `OK` cannot
// worsen the worst *judged* finding, and all three repos that emit this already
// read `OK`. It is the `checkReadmeLanguage` shape — the check ran, saw
// everything, and found no subject.
test('no version claimed anywhere is nothing to judge, not a skip', () => {
const f = checkVersionConsistency({ pluginVersion: null, readmeBadge: null, changelogTop: null, tags: [] });
assert.equal(f.find((x) => x.code === 'VERSION-UNAVAILABLE').skip, 'notRun');
const hit = f.find((x) => x.code === 'VERSION-NONE');
assert.equal(hit.level, 'OK');
assert.equal(hit.skip, undefined);
});
// The rename is the point, not cosmetics: `-UNAVAILABLE` is this engine's
// naming for a `notRun` skip (`DESC-UNAVAILABLE`, `INSTALL-TRUTH`). One
// OK-level `-UNAVAILABLE` would mislead exactly the reader this change serves.
test('the un-runnable version codes keep the -UNAVAILABLE naming', () => {
const src = readFileSync(new URL('./repo-standard-check.mjs', import.meta.url), 'utf8');
assert.ok(!src.includes('VERSION-UNAVAILABLE'), 'VERSION-UNAVAILABLE was renamed to VERSION-NONE');
});
test('an unreachable catalog is un-runnable — a second run resolves it', () => {

View file

@ -43,6 +43,12 @@ not interchangeable:
Nobody has an action. Reporting these as outstanding work is how a clean repo
gets rewritten for nothing.
The inverse error is reporting an `OK` as an un-run check. A check that ran and
found **no subject** is a verdict, not a skip: `VERSION-NONE` ("no version
claimed anywhere") and `LANGUAGE` ("no running prose to judge") are `OK` and
carry no action. `VERSION-NONE` was `VERSION-UNAVAILABLE`/`notRun` before 0.9.0
— if you see the old code, the engine is stale; see the version note below.
That applies hardest when the repo's own status is green. Since 0.7.0 a `SKIP`
no longer outranks an `OK` in the status line, so a repo reads `OK · 3 not run`
rather than `SKIP`. The summary line carries only the `notRun` count — a repo