docs(catalog): re-measure badge coverage at pinned refs (27 gated / 15 ungated across 12 plugins)

The 2026-08-02 reading (24 gated / 15 ungated across 7 of 11 plugins) predated
repo-standard entering the catalog and repo-mailbox badging its two remaining
axes. Re-measured 2026-08-04 at every pinned ref with the gate's own parsers:
42 catalog axis-claims, 27 badge-covered, 15 badge-less across 7 of 12 plugins,
source=ref for 12/12. The two readings reconcile exactly (24+3, 15-2+2).

Two premisses corrected while measuring:

- graceful-handoff has NOT lost its tests/hooks badges. Both badge-removing
  commits are unreleased (after v3.1.0), so at the pinned ref the catalog's
  "30 tests" is gated. Recorded at pickStatSource as the opposite failure
  direction to the llm-security case: reading the worktree misreports COVERAGE,
  not just values, and neither example alone would have caught it.

- the "251 selftest checks" correction is not settled: CLAUDE.md said 370,
  check-versions.mjs said 374, and repo-mailbox's own README at v0.19.0 implies
  390. Moot for the gate (badged and green at v0.20.2) and not re-measured, so
  the disagreement is flagged rather than laundered into one number.

Badge-less VALUES were not re-audited this session and do not need to be: only
two refs have moved since their audit, so all 15 stand measured at the ref they
are pinned to. Re-measurement is triggered by a ref moving, not by the calendar.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PtnSwCLpUR3Hk8SHkPoxX4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-04 21:43:52 +02:00
commit e6b734c4fb
2 changed files with 35 additions and 12 deletions

View file

@ -42,13 +42,25 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
compares every number on that line against the plugin's own shields badge for the same axis, and
ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not
per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list
to maintain. Measured 2026-08-02: **24 axis-pairs badge-covered · 15 axes badge-less across 7 of
the 11 plugins** and therefore ungated. All 15 have now been hand-audited at their pinned tags —
**14 were exact**; the two defects found were repo-mailbox (`6 CLI scripts` / `251 selftest checks`
against a true 8 / 370) and voyage (`500+ tests` against a measured 832). `N+` in the catalog is
read as a lower bound, not an equality, so `500+` was never gate-visible — ungated axes rot in
silence and need a periodic re-run of this pass. **Never hand-edit a stat line to silence the
gate** — the plugin's badge is the source for every stat number; fix the catalog to match it.
to maintain. **Re-measured 2026-08-04 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
27 badge-covered · 15 badge-less across 7 of the 12 plugins** and therefore ungated. That supersedes
the 2026-08-02 reading (`24 · 15 across 7 of 11`); only two entries moved in between and both
reconcile exactly — repo-mailbox now badges the two axes it did not (`+2` gated, `-2` ungated) and
repo-standard entered the catalog (`+1` gated, `+2` ungated). The 15 ungated axes were hand-audited
at their pinned tags — 13 on 2026-08-02 (**14 of that pass's 15 were exact**; the defects were
repo-mailbox `6 CLI scripts` (true 8) / `251 selftest checks`, and voyage `500+ tests` against a
measured 832), and repo-standard's two on 2026-08-04. Since no ungated plugin's ref has moved
since its audit, every ungated value currently stands measured at the ref it is pinned to.
**⚠️ The `251 selftest checks` correction is itself unresolved and must not be cited as settled:**
three records disagree on the true value at v0.19.0 — this file said 370, `check-versions.mjs`
said 374, and repo-mailbox's own README at that tag implies 390 (`183 + 134 + 73`). It is moot for
the gate (repo-mailbox badges the axis now, 398 at v0.20.2, green) and was **not** re-measured
2026-08-04. It stands as the standing proof of the cost: an ungated number rots, and so does the
record of having fixed it.
`N+` in the catalog is read as a lower bound, not an equality, so `500+` was never gate-visible —
ungated axes rot in silence and need a re-run of this pass **whenever a ref moves**. **Never
hand-edit a stat line to silence the gate** — the plugin's badge is the source for every stat
number; fix the catalog to match it.
- **Counting rules for a badge-less axis (calibrated against the badged plugins, 2026-08-02).** When
the catalog must count an axis itself, count it the way the badges do, or the numbers stop being
comparable across plugin blocks: **hooks = hook COMMAND entries** in `hooks/hooks.json` (not

View file

@ -48,11 +48,15 @@ export function extractCatalogLabel(readmeText, name) {
//
// The rule is PER-AXIS, not per-plugin. An axis is gated only when the plugin
// carries a badge for it; a badge-less axis is skipped silently. That is not a
// concession to voyage (the one plugin with no stat badges at all) — 14 axes
// across 8 of the 11 plugins are badge-less, so a per-plugin exception list
// would have had to name most of the marketplace and be edited by hand forever.
// The cost is stated plainly: those 14 numbers stay ungated and need a human
// pass (repo-mailbox's two were 6 and 251 against a true 8 and 374 when measured).
// concession to voyage (the one plugin with no stat badges at all) — re-measured
// 2026-08-04 at the pinned refs, 15 of the 42 axis-claims across 7 of the 12
// plugins are badge-less, so a per-plugin exception list would have had to name
// most of the marketplace and be edited by hand forever. The cost is stated
// plainly: those 15 numbers stay ungated and need a human pass whenever a ref
// moves — repo-mailbox's `6 CLI scripts` was 8 when measured, and it badges both
// of its axes now, so they are gated as of v0.20.2. (Its selftest-check figure is
// the standing proof that ungated numbers rot: the catalog said 251 while three
// records of the true value disagree — see CLAUDE.md. Not re-measured.)
const AXIS_SYNONYM = { 'reference doc': 'reference', 'knowledge doc': 'reference', doc: 'reference' };
@ -105,6 +109,13 @@ function readStatSegment(segment, out) {
// while the catalog was correct about v7.8.3. Fall back to the working tree only when
// the ref cannot be read at all — a ref that resolves to nothing is already its own
// ERROR (check 1), so this fallback never hides a dangling ref.
//
// Reading the ref decides COVERAGE too, not just values. Measured 2026-08-04:
// graceful-handoff had dropped its `tests-30` and `hooks-0` badges on main (13e2972,
// 5e17409) without releasing, so the working tree says the catalog's `30 tests` is
// ungated while at v3.1.0 it is badged and gated. The failure directions are opposite
// — the llm-security case above makes a correct catalog look stale — so neither one
// alone would have caught this.
export function pickStatSource({ atRef, atWorktree }) {
return atRef !== null && atRef !== undefined ? atRef : (atWorktree ?? null);
}