docs(catalog): re-measure badge coverage at pinned refs (27 gated / 15 ungated across 12 plugins)

The 2026-08-02 reading (24 gated / 15 ungated across 7 of 11 plugins) predated
repo-standard entering the catalog and repo-mailbox badging its two remaining
axes. Re-measured 2026-08-04 at every pinned ref with the gate's own parsers:
42 catalog axis-claims, 27 badge-covered, 15 badge-less across 7 of 12 plugins,
source=ref for 12/12. The two readings reconcile exactly (24+3, 15-2+2).

Two premisses corrected while measuring:

- graceful-handoff has NOT lost its tests/hooks badges. Both badge-removing
  commits are unreleased (after v3.1.0), so at the pinned ref the catalog's
  "30 tests" is gated. Recorded at pickStatSource as the opposite failure
  direction to the llm-security case: reading the worktree misreports COVERAGE,
  not just values, and neither example alone would have caught it.

- the "251 selftest checks" correction is not settled: CLAUDE.md said 370,
  check-versions.mjs said 374, and repo-mailbox's own README at v0.19.0 implies
  390. Moot for the gate (badged and green at v0.20.2) and not re-measured, so
  the disagreement is flagged rather than laundered into one number.

Badge-less VALUES were not re-audited this session and do not need to be: only
two refs have moved since their audit, so all 15 stand measured at the ref they
are pinned to. Re-measurement is triggered by a ref moving, not by the calendar.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PtnSwCLpUR3Hk8SHkPoxX4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-04 21:43:52 +02:00
commit e6b734c4fb
2 changed files with 35 additions and 12 deletions

View file

@ -42,13 +42,25 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
compares every number on that line against the plugin's own shields badge for the same axis, and compares every number on that line against the plugin's own shields badge for the same axis, and
ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not
per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list
to maintain. Measured 2026-08-02: **24 axis-pairs badge-covered · 15 axes badge-less across 7 of to maintain. **Re-measured 2026-08-04 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
the 11 plugins** and therefore ungated. All 15 have now been hand-audited at their pinned tags — 27 badge-covered · 15 badge-less across 7 of the 12 plugins** and therefore ungated. That supersedes
**14 were exact**; the two defects found were repo-mailbox (`6 CLI scripts` / `251 selftest checks` the 2026-08-02 reading (`24 · 15 across 7 of 11`); only two entries moved in between and both
against a true 8 / 370) and voyage (`500+ tests` against a measured 832). `N+` in the catalog is reconcile exactly — repo-mailbox now badges the two axes it did not (`+2` gated, `-2` ungated) and
read as a lower bound, not an equality, so `500+` was never gate-visible — ungated axes rot in repo-standard entered the catalog (`+1` gated, `+2` ungated). The 15 ungated axes were hand-audited
silence and need a periodic re-run of this pass. **Never hand-edit a stat line to silence the at their pinned tags — 13 on 2026-08-02 (**14 of that pass's 15 were exact**; the defects were
gate** — the plugin's badge is the source for every stat number; fix the catalog to match it. repo-mailbox `6 CLI scripts` (true 8) / `251 selftest checks`, and voyage `500+ tests` against a
measured 832), and repo-standard's two on 2026-08-04. Since no ungated plugin's ref has moved
since its audit, every ungated value currently stands measured at the ref it is pinned to.
**⚠️ The `251 selftest checks` correction is itself unresolved and must not be cited as settled:**
three records disagree on the true value at v0.19.0 — this file said 370, `check-versions.mjs`
said 374, and repo-mailbox's own README at that tag implies 390 (`183 + 134 + 73`). It is moot for
the gate (repo-mailbox badges the axis now, 398 at v0.20.2, green) and was **not** re-measured
2026-08-04. It stands as the standing proof of the cost: an ungated number rots, and so does the
record of having fixed it.
`N+` in the catalog is read as a lower bound, not an equality, so `500+` was never gate-visible —
ungated axes rot in silence and need a re-run of this pass **whenever a ref moves**. **Never
hand-edit a stat line to silence the gate** — the plugin's badge is the source for every stat
number; fix the catalog to match it.
- **Counting rules for a badge-less axis (calibrated against the badged plugins, 2026-08-02).** When - **Counting rules for a badge-less axis (calibrated against the badged plugins, 2026-08-02).** When
the catalog must count an axis itself, count it the way the badges do, or the numbers stop being the catalog must count an axis itself, count it the way the badges do, or the numbers stop being
comparable across plugin blocks: **hooks = hook COMMAND entries** in `hooks/hooks.json` (not comparable across plugin blocks: **hooks = hook COMMAND entries** in `hooks/hooks.json` (not

View file

@ -48,11 +48,15 @@ export function extractCatalogLabel(readmeText, name) {
// //
// The rule is PER-AXIS, not per-plugin. An axis is gated only when the plugin // The rule is PER-AXIS, not per-plugin. An axis is gated only when the plugin
// carries a badge for it; a badge-less axis is skipped silently. That is not a // carries a badge for it; a badge-less axis is skipped silently. That is not a
// concession to voyage (the one plugin with no stat badges at all) — 14 axes // concession to voyage (the one plugin with no stat badges at all) — re-measured
// across 8 of the 11 plugins are badge-less, so a per-plugin exception list // 2026-08-04 at the pinned refs, 15 of the 42 axis-claims across 7 of the 12
// would have had to name most of the marketplace and be edited by hand forever. // plugins are badge-less, so a per-plugin exception list would have had to name
// The cost is stated plainly: those 14 numbers stay ungated and need a human // most of the marketplace and be edited by hand forever. The cost is stated
// pass (repo-mailbox's two were 6 and 251 against a true 8 and 374 when measured). // plainly: those 15 numbers stay ungated and need a human pass whenever a ref
// moves — repo-mailbox's `6 CLI scripts` was 8 when measured, and it badges both
// of its axes now, so they are gated as of v0.20.2. (Its selftest-check figure is
// the standing proof that ungated numbers rot: the catalog said 251 while three
// records of the true value disagree — see CLAUDE.md. Not re-measured.)
const AXIS_SYNONYM = { 'reference doc': 'reference', 'knowledge doc': 'reference', doc: 'reference' }; const AXIS_SYNONYM = { 'reference doc': 'reference', 'knowledge doc': 'reference', doc: 'reference' };
@ -105,6 +109,13 @@ function readStatSegment(segment, out) {
// while the catalog was correct about v7.8.3. Fall back to the working tree only when // while the catalog was correct about v7.8.3. Fall back to the working tree only when
// the ref cannot be read at all — a ref that resolves to nothing is already its own // the ref cannot be read at all — a ref that resolves to nothing is already its own
// ERROR (check 1), so this fallback never hides a dangling ref. // ERROR (check 1), so this fallback never hides a dangling ref.
//
// Reading the ref decides COVERAGE too, not just values. Measured 2026-08-04:
// graceful-handoff had dropped its `tests-30` and `hooks-0` badges on main (13e2972,
// 5e17409) without releasing, so the working tree says the catalog's `30 tests` is
// ungated while at v3.1.0 it is badged and gated. The failure directions are opposite
// — the llm-security case above makes a correct catalog look stale — so neither one
// alone would have caught this.
export function pickStatSource({ atRef, atWorktree }) { export function pickStatSource({ atRef, atWorktree }) {
return atRef !== null && atRef !== undefined ? atRef : (atWorktree ?? null); return atRef !== null && atRef !== undefined ? atRef : (atWorktree ?? null);
} }