feat(engine)!: a skip says which of the two things it is
`notChecked` merged "deliberately not judged" with "could not run". 0.7.0 took
SKIP out of the severity lattice, which fixed "clean repos look skipped"; this
fixes the same defect one level down, where it was saying so more quietly.
Measured across 19 clones, 35 of 39 skips are deliberate — so
`portfolio-optimiser — OK · 11 not checked` announced eleven unread checks when
all eleven were fixture and out-of-repo links.
Every SKIP finding now carries `skip: 'byDesign' | 'notRun'`, set at the
emission site rather than looked up from its code: VERSION-TAG is emitted at
SKIP with no tags and at ERROR with the wrong one, so a code→kind map has to
re-derive a reason the emitter already had. Untagged falls to `notRun`, the
loud side. Results carry `skips: { byDesign, notRun }`; `notChecked` stays a
number and stays the total, because a consumer doing `notChecked > 0` against
an object gets a silent false.
The summary line names only `notRun` — the deliberate ones keep their own
`NOT JUDGED` heading in the detail output. Landing the split in `--json` alone
would have left the symptom exactly where it was.
VERSION-UNAVAILABLE stays `notRun` by decision, not by default: it is arguably
"no subject to judge", the shape checkReadmeLanguage answers with OK, but
re-levelling it moves a repo's status and is a second behaviour change.
Verified: 169 tests. 19 clones swept with both engines offline — 220 judged
findings, every status, notChecked and buckets identical, no finding differing
beyond the new tag. The exhaustiveness test scans the engine SOURCE, not a
finding set: a corpus test only sees the sites it triggers, and the next
emission site added would take the reader's default invisibly.
BREAKING CHANGE: the summary line's coverage qualifier reads `· N not run` and
counts only un-runnable skips, where it read `· N not checked` over all of
them. `--json` gains `skips`; `notChecked` is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DHaQTcaD3w2C6PqVsAmrcv
This commit is contained in:
parent
1a647a591f
commit
f4017bcfe1
8 changed files with 346 additions and 38 deletions
|
|
@ -30,16 +30,25 @@ could not run), `OK`. Exit 1 on any ERROR. Add `--offline` to skip the two
|
|||
network calls, `--json` for machine output, `--refresh` to compare the bundled
|
||||
register against the live org listing.
|
||||
|
||||
**Never report a `SKIP` as a pass.** A SKIP means the gate could not see enough
|
||||
to judge — an unreachable forge, an untagged repo, a link leaving the
|
||||
repository. Say which.
|
||||
**Never report a `SKIP` as a pass.** A SKIP means the gate did not judge the
|
||||
thing. Say which, and say which KIND — since 0.8.0 there are two, and they are
|
||||
not interchangeable:
|
||||
|
||||
- **`NOT CHECKED` (`skips.notRun`)** — the gate could not see enough. An
|
||||
unreachable forge or catalog, an unregistered repo, a repo with no tags. A
|
||||
re-run or an operator action turns it into a verdict, so it belongs in your
|
||||
report even when the status is green.
|
||||
- **`NOT JUDGED` (`skips.byDesign`)** — the gate saw it and declined on
|
||||
purpose. A link leaving the repository, a dead link inside a fixture path.
|
||||
Nobody has an action. Reporting these as outstanding work is how a clean repo
|
||||
gets rewritten for nothing.
|
||||
|
||||
That applies hardest when the repo's own status is green. Since 0.7.0 a `SKIP`
|
||||
no longer outranks an `OK` in the status line, so a repo reads
|
||||
`OK · 3 not checked` rather than `SKIP`. The number is the qualifier: three
|
||||
findings nobody judged. Report the status and the number together, never the
|
||||
status alone, and name what went unjudged — the detail output lists them under
|
||||
`NOT CHECKED`.
|
||||
no longer outranks an `OK` in the status line, so a repo reads `OK · 3 not run`
|
||||
rather than `SKIP`. The summary line carries only the `notRun` count — a repo
|
||||
whose every skip was deliberate prints an unqualified verdict. **The line's
|
||||
silence is not proof there was nothing to say:** read the `NOT JUDGED` block, or
|
||||
`skips.byDesign` in `--json`, before you tell anyone a repo was fully judged.
|
||||
|
||||
**Confirm the version before you trust a green run.** Measured 2026-08-04:
|
||||
`${CLAUDE_PLUGIN_ROOT}` resolved to a cached `0.1.1` in a live session while
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue