feat(engine)!: a skip says which of the two things it is

`notChecked` merged "deliberately not judged" with "could not run". 0.7.0 took
SKIP out of the severity lattice, which fixed "clean repos look skipped"; this
fixes the same defect one level down, where it was saying so more quietly.
Measured across 19 clones, 35 of 39 skips are deliberate — so
`portfolio-optimiser — OK · 11 not checked` announced eleven unread checks when
all eleven were fixture and out-of-repo links.

Every SKIP finding now carries `skip: 'byDesign' | 'notRun'`, set at the
emission site rather than looked up from its code: VERSION-TAG is emitted at
SKIP with no tags and at ERROR with the wrong one, so a code→kind map has to
re-derive a reason the emitter already had. Untagged falls to `notRun`, the
loud side. Results carry `skips: { byDesign, notRun }`; `notChecked` stays a
number and stays the total, because a consumer doing `notChecked > 0` against
an object gets a silent false.

The summary line names only `notRun` — the deliberate ones keep their own
`NOT JUDGED` heading in the detail output. Landing the split in `--json` alone
would have left the symptom exactly where it was.

VERSION-UNAVAILABLE stays `notRun` by decision, not by default: it is arguably
"no subject to judge", the shape checkReadmeLanguage answers with OK, but
re-levelling it moves a repo's status and is a second behaviour change.

Verified: 169 tests. 19 clones swept with both engines offline — 220 judged
findings, every status, notChecked and buckets identical, no finding differing
beyond the new tag. The exhaustiveness test scans the engine SOURCE, not a
finding set: a corpus test only sees the sites it triggers, and the next
emission site added would take the reader's default invisibly.

BREAKING CHANGE: the summary line's coverage qualifier reads `· N not run` and
counts only un-runnable skips, where it read `· N not checked` over all of
them. `--json` gains `skips`; `notChecked` is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DHaQTcaD3w2C6PqVsAmrcv
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 21:52:00 +02:00
commit f4017bcfe1
8 changed files with 346 additions and 38 deletions

View file

@ -4,6 +4,63 @@ All notable changes to this project are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.8.0] — 2026-08-09
### Added
- **Skips split into the two facts they always were.** Every `SKIP` finding now
carries `skip: 'byDesign' | 'notRun'` beside its level, and results carry
`skips: { byDesign, notRun }` beside `status`. `byDesign` is a check that saw
the thing and declined — a link leaving the repository, a dead link in a
fixture path; it can never become a verdict and nobody has an action.
`notRun` is one a re-run or an operator action resolves — an unreachable
catalog, an unregistered repo, a repo with no tags.
0.7.0 took `SKIP` out of the severity lattice, which fixed "clean repos look
skipped". It left both kinds inside one number, which said the same thing
more quietly: measured across 19 clones, 35 of 39 skips were deliberate, so
`portfolio-optimiser — OK · 11 not checked` announced eleven unread checks
when all eleven were fixture and out-of-repo links. org-ops named the split
in its census-05 report without a name for it.
The kind is set where the finding is emitted, never looked up from its code:
`VERSION-TAG` is emitted at `SKIP` with no tags and at `ERROR` with the wrong
one, so a code→kind map would have to re-derive a reason the emission site
already had. An untagged skip counts as `notRun` — the loud side, because a
skip of unknown kind must not inherit "deliberate, nothing to see".
### Changed
- **The summary line names only what someone has an action on.** A repo whose
every skip was deliberate prints an unqualified verdict; `· N not run`
appears only when N > 0. The deliberate ones are not silenced — the detail
output gained a `NOT JUDGED — deliberately outside what this gate decides`
heading beside `NOT CHECKED`. Landing the split in `--json` alone would have
left the human-facing symptom exactly where it was.
- `notChecked` is untouched: still present, still a **number**, still the total
of both kinds. A consumer doing `notChecked > 0` against an object would get
a silent false — the same class of quiet wrong answer this axis exists to
remove. Absent `skips` means an engine older than 0.8.0; absent `notChecked`
one older than 0.7.0. Neither absence is zero.
- `VERSION-UNAVAILABLE` stays `notRun` by decision, not by default. "No package
version to compare against" is arguably a third thing — no *subject* to
judge, the shape `checkReadmeLanguage` answers with `OK` rather than `SKIP`
but re-levelling it moves a repo's `status`, and that is a second behaviour
change. Recorded in the engine so the next reader knows it was decided.
- README example output cites `@<sha>` instead of a commit hash. Three releases
running, those examples went stale against the behaviour they demonstrate; a
placeholder cannot.
Verified before release: 169 tests; 19 clones swept with both engines offline,
220 judged findings and every `status`, `notChecked` and `buckets` identical,
with no finding differing beyond the new tag. The exhaustiveness test scans the
engine source rather than the finding set — a corpus test only sees the sites it
manages to trigger, and the next emission site added would inherit the reader's
default invisibly.
## [0.7.1] — 2026-08-09
### Fixed