feat(engine)!: "no version to compare against" is a verdict, not a skip
`VERSION-UNAVAILABLE` (SKIP/notRun) becomes `VERSION-NONE` (OK). A repo that claims no version in any of the four places one can be written down has not left a check un-run — the check ran, saw all four, and found no subject. That is the shape `checkReadmeLanguage` has always answered with OK. 0.8.0 deferred this and recorded the reason in the engine: re-levelling "moves a repo's status". Measured false across 19 clones — an added OK cannot worsen the worst *judged* finding, and all three repos emitting the code already read OK. Status moves only for a repo whose entire finding set was skips. OK cannot bless a real gap here: no class requires a version file, and a `plugin` missing its manifest is an independent FILE-MISSING ERROR. Renamed, not just re-levelled: `-UNAVAILABLE` is this engine's naming for a notRun skip, and one OK-level `-UNAVAILABLE` would mislead exactly the reader this change serves. A source scan keeps the old name out. A/B sweep, both engines offline, 19 clones: 223 judged findings, every status and buckets identical; 3 conversions (notChecked 14→13, 3→2, 14→13), byDesign untouched. 170 tests. `--refresh`: register 21 = forge 21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CTNGvxzCLZp3UXfibYG4kb
This commit is contained in:
parent
9a566240e8
commit
1dfd408ffc
8 changed files with 99 additions and 21 deletions
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "repo-standard",
|
||||
"version": "0.8.1",
|
||||
"version": "0.9.0",
|
||||
"description": "Per-repo gate for the open/ presentation standard: README first screen, install block, files required by the repo's class, and dead repo references.",
|
||||
"author": {
|
||||
"name": "Kjell Tore Guttormsen"
|
||||
|
|
|
|||
36
CHANGELOG.md
36
CHANGELOG.md
|
|
@ -4,6 +4,42 @@ All notable changes to this project are documented here.
|
|||
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
|
||||
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [0.9.0] — 2026-08-09
|
||||
|
||||
### Changed
|
||||
|
||||
- **`VERSION-UNAVAILABLE` (`SKIP`/`notRun`) is now `VERSION-NONE` at `OK`.** A
|
||||
repository that claims no version in any of the four places one can be written
|
||||
down has not left a check un-run — the check ran, saw all four, and found no
|
||||
subject. That is the shape `checkReadmeLanguage` has always answered with `OK`
|
||||
("no running prose to judge"), and filing it as a skip told three
|
||||
correctly-built repositories that something about them was unread.
|
||||
|
||||
0.8.0 deferred this deliberately and recorded the reason in the engine:
|
||||
re-levelling "moves a repo's status". That was measured false. An added `OK`
|
||||
cannot worsen the worst *judged* finding, so `status` moves only for a repo
|
||||
whose entire finding set was skips — and all three repos that emit this code
|
||||
(`ktg-plugin-marketplace`, `playground-design-system`,
|
||||
`portfolio-optimiser-commons`) already read `OK`. The comment is gone with the
|
||||
code it justified.
|
||||
|
||||
Nor can `OK` here bless a real gap: no class in the register requires a
|
||||
version file, and a `plugin` missing its manifest is an independent
|
||||
`FILE-MISSING` `ERROR`.
|
||||
|
||||
**The code was renamed, not just re-levelled.** `-UNAVAILABLE` is this
|
||||
engine's naming for a `notRun` skip (`DESC-UNAVAILABLE`), and one `OK`-level
|
||||
`-UNAVAILABLE` would mislead exactly the reader this change is for. A source
|
||||
scan holds the old name out of the engine, so a re-introduction fails the
|
||||
suite rather than the next census.
|
||||
|
||||
Verified before release: 170 tests; 19 clones swept with both engines offline,
|
||||
223 judged findings, every `status` and `buckets` identical. Exactly three
|
||||
findings converted — `notChecked` 14→13, 3→2, 14→13, `notRun` 3→2 in each, and
|
||||
`byDesign` untouched. No repository's `status` moved. Remaining `notRun` across
|
||||
the org is actionable by construction: `INSTALL-TRUTH` (12), `VERSION-TAG` (1),
|
||||
plus the two that only appear under `--offline`.
|
||||
|
||||
## [0.8.1] — 2026-08-09
|
||||
|
||||
### Fixed
|
||||
|
|
|
|||
17
CLAUDE.md
17
CLAUDE.md
|
|
@ -54,6 +54,21 @@ would recreate, in data, exactly the drift this plugin exists to remove.
|
|||
the symptom untouched. The test that holds this is a scan of the engine
|
||||
SOURCE, not of a finding set: a corpus test only sees the sites it manages to
|
||||
trigger, and the next site added would take the reader's default invisibly.
|
||||
- **"Nothing to judge" is an `OK`, not a skip — and it is a THIRD shape, not a
|
||||
third bucket.** A check that ran, saw everything, and found no subject has
|
||||
produced a verdict: there is nothing here to be wrong. `checkReadmeLanguage`
|
||||
already answered that way for a README with no prose; `VERSION-NONE` (0.9.0,
|
||||
was `VERSION-UNAVAILABLE`/`notRun`) does for a repo that claims no version.
|
||||
The deferral that kept it a skip for one release was recorded in the engine as
|
||||
"re-levelling moves a repo's status" — measured false: an added `OK` cannot
|
||||
worsen the worst *judged* finding, and all three affected repos already read
|
||||
`OK`. Status moves only for a repo whose *entire* finding set was skips. The
|
||||
lesson is not about this code: **a recorded decision that is wrong is worse
|
||||
than no record**, because the next session re-derives the measurement instead
|
||||
of reading it. And the guard against `OK` blessing a real gap is that the gap
|
||||
is caught elsewhere — no class requires a version file, and a `plugin` missing
|
||||
its manifest is an independent `FILE-MISSING` `ERROR`. Without that, this
|
||||
would have been "SKIP is never a pass" one level up.
|
||||
- **When a check fires many times in one repo, suspect the CHECK.** The first
|
||||
link pass produced ~30 findings against `llm-security` and all were noise —
|
||||
regexes in code spans, `file:` URLs, relative paths resolved against the wrong
|
||||
|
|
@ -149,7 +164,7 @@ would recreate, in data, exactly the drift this plugin exists to remove.
|
|||
## Commands
|
||||
|
||||
```bash
|
||||
npm test # 169 tests
|
||||
npm test # 170 tests
|
||||
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
|
||||
node scripts/repo-standard-check.mjs --offline # no network call
|
||||
node scripts/repo-standard-check.mjs --json # machine output
|
||||
|
|
|
|||
18
README.md
18
README.md
|
|
@ -10,7 +10,7 @@ checks that surface in one repository and reports what it finds.
|
|||
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development.*
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
|
@ -103,12 +103,20 @@ Merging them made clean repositories read as unread. Measured across 19 clones,
|
|||
checked` announced eleven unread checks when all eleven were links the gate
|
||||
declines to judge on purpose.
|
||||
|
||||
A third shape is not a skip at all and must not be filed as one: the check ran,
|
||||
saw everything, and found **no subject** to judge. A repository that claims no
|
||||
version anywhere is `VERSION-NONE` at `OK` since 0.9.0 — the same answer
|
||||
`LANGUAGE` gives a README with no running prose. It had been `notRun`, which
|
||||
told three correctly-built repositories that something about them was still
|
||||
unread. What remains under `notRun` is genuinely actionable: an unreachable
|
||||
catalog, an unregistered repo, a repo that has not been tagged yet.
|
||||
|
||||
The summary line names only what someone has an action on:
|
||||
|
||||
```
|
||||
✓ portfolio-optimiser [standalone] — OK (repo-standard v0.8.1 @<sha>)
|
||||
✗ some-plugin [plugin] — ERROR · 2 not run (repo-standard v0.8.1 @<sha>)
|
||||
· stranger — SKIP · 1 not run (repo-standard v0.8.1 @<sha>)
|
||||
✓ portfolio-optimiser [standalone] — OK (repo-standard v0.9.0 @<sha>)
|
||||
✗ some-plugin [plugin] — ERROR · 2 not run (repo-standard v0.9.0 @<sha>)
|
||||
· stranger — SKIP · 1 not run (repo-standard v0.9.0 @<sha>)
|
||||
```
|
||||
|
||||
The deliberate ones are not silenced: they keep their own `NOT JUDGED` heading
|
||||
|
|
@ -259,7 +267,7 @@ distinct from "an older engine that never tried".
|
|||
npm test
|
||||
```
|
||||
|
||||
169 tests over the pure classifiers. The reference fixtures are measured false
|
||||
170 tests over the pure classifiers. The reference fixtures are measured false
|
||||
positives, each with its expected verdict — the six that produced the
|
||||
three-outcome reference rule, plus the noise sources found by running the gate
|
||||
against a real repository: regexes inside code spans that are markdown links to
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "repo-standard",
|
||||
"version": "0.8.1",
|
||||
"version": "0.9.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"engines": {
|
||||
|
|
|
|||
|
|
@ -517,13 +517,15 @@ export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogT
|
|||
const findings = [];
|
||||
const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null;
|
||||
if (!v) {
|
||||
// `notRun` by operator decision 2026-08-09, not by default. "No package
|
||||
// version to compare against" is arguably a third thing — no SUBJECT to
|
||||
// judge, the shape checkReadmeLanguage answers with OK — but re-levelling
|
||||
// it moves a repo's status, and that is a second behaviour change riding
|
||||
// on this one. Recorded here so the next reader knows it was decided, not
|
||||
// overlooked.
|
||||
return [{ level: 'SKIP', skip: 'notRun', code: 'VERSION-UNAVAILABLE', msg: 'no package version found — nothing to compare against' }];
|
||||
// Not a SKIP. This check ran, saw all four places a version can be written
|
||||
// down, and found no version claimed in any of them — no SUBJECT to judge,
|
||||
// the shape checkReadmeLanguage answers with OK. It sat at `SKIP`/`notRun`
|
||||
// until 0.9.0, deferred once on the ground that re-levelling would move a
|
||||
// repo's status. That was measured false: an added OK cannot worsen the
|
||||
// worst *judged* finding, and all three repos that reach this line already
|
||||
// read OK. Nor can OK bless a real gap — no class requires a version file,
|
||||
// and a `plugin` missing its manifest is an independent FILE-MISSING ERROR.
|
||||
return [{ level: 'OK', code: 'VERSION-NONE', msg: 'no version claimed anywhere — nothing to check' }];
|
||||
}
|
||||
|
||||
if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) {
|
||||
|
|
|
|||
|
|
@ -1646,13 +1646,24 @@ test('no tags yet is un-runnable — a release resolves it', () => {
|
|||
assert.equal(f.find((x) => x.code === 'VERSION-TAG').skip, 'notRun');
|
||||
});
|
||||
|
||||
// Kept in `notRun` deliberately (operator decision 2026-08-09). It is arguably
|
||||
// a third thing — "no subject to judge", the shape `checkReadmeLanguage` calls
|
||||
// OK rather than SKIP — but re-levelling it moves `status` for a repo and is a
|
||||
// second behaviour change; it does not ride on this one.
|
||||
test('no package version is un-runnable, not deliberate', () => {
|
||||
// Was `SKIP`/`notRun` until 0.9.0, deferred once on the stated ground that
|
||||
// re-levelling "moves a repo's status". Measured false: an added `OK` cannot
|
||||
// worsen the worst *judged* finding, and all three repos that emit this already
|
||||
// read `OK`. It is the `checkReadmeLanguage` shape — the check ran, saw
|
||||
// everything, and found no subject.
|
||||
test('no version claimed anywhere is nothing to judge, not a skip', () => {
|
||||
const f = checkVersionConsistency({ pluginVersion: null, readmeBadge: null, changelogTop: null, tags: [] });
|
||||
assert.equal(f.find((x) => x.code === 'VERSION-UNAVAILABLE').skip, 'notRun');
|
||||
const hit = f.find((x) => x.code === 'VERSION-NONE');
|
||||
assert.equal(hit.level, 'OK');
|
||||
assert.equal(hit.skip, undefined);
|
||||
});
|
||||
|
||||
// The rename is the point, not cosmetics: `-UNAVAILABLE` is this engine's
|
||||
// naming for a `notRun` skip (`DESC-UNAVAILABLE`, `INSTALL-TRUTH`). One
|
||||
// OK-level `-UNAVAILABLE` would mislead exactly the reader this change serves.
|
||||
test('the un-runnable version codes keep the -UNAVAILABLE naming', () => {
|
||||
const src = readFileSync(new URL('./repo-standard-check.mjs', import.meta.url), 'utf8');
|
||||
assert.ok(!src.includes('VERSION-UNAVAILABLE'), 'VERSION-UNAVAILABLE was renamed to VERSION-NONE');
|
||||
});
|
||||
|
||||
test('an unreachable catalog is un-runnable — a second run resolves it', () => {
|
||||
|
|
|
|||
|
|
@ -43,6 +43,12 @@ not interchangeable:
|
|||
Nobody has an action. Reporting these as outstanding work is how a clean repo
|
||||
gets rewritten for nothing.
|
||||
|
||||
The inverse error is reporting an `OK` as an un-run check. A check that ran and
|
||||
found **no subject** is a verdict, not a skip: `VERSION-NONE` ("no version
|
||||
claimed anywhere") and `LANGUAGE` ("no running prose to judge") are `OK` and
|
||||
carry no action. `VERSION-NONE` was `VERSION-UNAVAILABLE`/`notRun` before 0.9.0
|
||||
— if you see the old code, the engine is stale; see the version note below.
|
||||
|
||||
That applies hardest when the repo's own status is green. Since 0.7.0 a `SKIP`
|
||||
no longer outranks an `OK` in the status line, so a repo reads `OK · 3 not run`
|
||||
rather than `SKIP`. The summary line carries only the `notRun` count — a repo
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue