feat(engine)!: a skip says which of the two things it is
`notChecked` merged "deliberately not judged" with "could not run". 0.7.0 took
SKIP out of the severity lattice, which fixed "clean repos look skipped"; this
fixes the same defect one level down, where it was saying so more quietly.
Measured across 19 clones, 35 of 39 skips are deliberate — so
`portfolio-optimiser — OK · 11 not checked` announced eleven unread checks when
all eleven were fixture and out-of-repo links.
Every SKIP finding now carries `skip: 'byDesign' | 'notRun'`, set at the
emission site rather than looked up from its code: VERSION-TAG is emitted at
SKIP with no tags and at ERROR with the wrong one, so a code→kind map has to
re-derive a reason the emitter already had. Untagged falls to `notRun`, the
loud side. Results carry `skips: { byDesign, notRun }`; `notChecked` stays a
number and stays the total, because a consumer doing `notChecked > 0` against
an object gets a silent false.
The summary line names only `notRun` — the deliberate ones keep their own
`NOT JUDGED` heading in the detail output. Landing the split in `--json` alone
would have left the symptom exactly where it was.
VERSION-UNAVAILABLE stays `notRun` by decision, not by default: it is arguably
"no subject to judge", the shape checkReadmeLanguage answers with OK, but
re-levelling it moves a repo's status and is a second behaviour change.
Verified: 169 tests. 19 clones swept with both engines offline — 220 judged
findings, every status, notChecked and buckets identical, no finding differing
beyond the new tag. The exhaustiveness test scans the engine SOURCE, not a
finding set: a corpus test only sees the sites it triggers, and the next
emission site added would take the reader's default invisibly.
BREAKING CHANGE: the summary line's coverage qualifier reads `· N not run` and
counts only un-runnable skips, where it read `· N not checked` over all of
them. `--json` gains `skips`; `notChecked` is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DHaQTcaD3w2C6PqVsAmrcv
This commit is contained in:
parent
1a647a591f
commit
f4017bcfe1
8 changed files with 346 additions and 38 deletions
57
CHANGELOG.md
57
CHANGELOG.md
|
|
@ -4,6 +4,63 @@ All notable changes to this project are documented here.
|
|||
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
|
||||
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [0.8.0] — 2026-08-09
|
||||
|
||||
### Added
|
||||
|
||||
- **Skips split into the two facts they always were.** Every `SKIP` finding now
|
||||
carries `skip: 'byDesign' | 'notRun'` beside its level, and results carry
|
||||
`skips: { byDesign, notRun }` beside `status`. `byDesign` is a check that saw
|
||||
the thing and declined — a link leaving the repository, a dead link in a
|
||||
fixture path; it can never become a verdict and nobody has an action.
|
||||
`notRun` is one a re-run or an operator action resolves — an unreachable
|
||||
catalog, an unregistered repo, a repo with no tags.
|
||||
|
||||
0.7.0 took `SKIP` out of the severity lattice, which fixed "clean repos look
|
||||
skipped". It left both kinds inside one number, which said the same thing
|
||||
more quietly: measured across 19 clones, 35 of 39 skips were deliberate, so
|
||||
`portfolio-optimiser — OK · 11 not checked` announced eleven unread checks
|
||||
when all eleven were fixture and out-of-repo links. org-ops named the split
|
||||
in its census-05 report without a name for it.
|
||||
|
||||
The kind is set where the finding is emitted, never looked up from its code:
|
||||
`VERSION-TAG` is emitted at `SKIP` with no tags and at `ERROR` with the wrong
|
||||
one, so a code→kind map would have to re-derive a reason the emission site
|
||||
already had. An untagged skip counts as `notRun` — the loud side, because a
|
||||
skip of unknown kind must not inherit "deliberate, nothing to see".
|
||||
|
||||
### Changed
|
||||
|
||||
- **The summary line names only what someone has an action on.** A repo whose
|
||||
every skip was deliberate prints an unqualified verdict; `· N not run`
|
||||
appears only when N > 0. The deliberate ones are not silenced — the detail
|
||||
output gained a `NOT JUDGED — deliberately outside what this gate decides`
|
||||
heading beside `NOT CHECKED`. Landing the split in `--json` alone would have
|
||||
left the human-facing symptom exactly where it was.
|
||||
|
||||
- `notChecked` is untouched: still present, still a **number**, still the total
|
||||
of both kinds. A consumer doing `notChecked > 0` against an object would get
|
||||
a silent false — the same class of quiet wrong answer this axis exists to
|
||||
remove. Absent `skips` means an engine older than 0.8.0; absent `notChecked`
|
||||
one older than 0.7.0. Neither absence is zero.
|
||||
|
||||
- `VERSION-UNAVAILABLE` stays `notRun` by decision, not by default. "No package
|
||||
version to compare against" is arguably a third thing — no *subject* to
|
||||
judge, the shape `checkReadmeLanguage` answers with `OK` rather than `SKIP` —
|
||||
but re-levelling it moves a repo's `status`, and that is a second behaviour
|
||||
change. Recorded in the engine so the next reader knows it was decided.
|
||||
|
||||
- README example output cites `@<sha>` instead of a commit hash. Three releases
|
||||
running, those examples went stale against the behaviour they demonstrate; a
|
||||
placeholder cannot.
|
||||
|
||||
Verified before release: 169 tests; 19 clones swept with both engines offline,
|
||||
220 judged findings and every `status`, `notChecked` and `buckets` identical,
|
||||
with no finding differing beyond the new tag. The exhaustiveness test scans the
|
||||
engine source rather than the finding set — a corpus test only sees the sites it
|
||||
manages to trigger, and the next emission site added would inherit the reader's
|
||||
default invisibly.
|
||||
|
||||
## [0.7.1] — 2026-08-09
|
||||
|
||||
### Fixed
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue