docs(catalog): re-measure badge coverage at every pinned ref — 24 gated, 18 ungated

The 2026-08-04 count (27 gated / 15 ungated across 7 plugins) was correct and is
not overturned: re-running the measurement against catalog commit c7fbbd3
reproduces it exactly. Coverage fell to 24/18 across 8 because three axes lost
their badge when a ref moved — okr v1.8.2 -> v1.10.0 dropped agents-7 and
hooks-3, graceful-handoff v3.1.0 -> v3.2.1 dropped tests-30. 27 - 3 = 24.

All 18 ungated values are now measured at their refs in the same pass rather
than inherited: 16 exact, 1 correct-but-split (voyage's 6 commands + 1 helper is
the plugin's own split, 7 files), 1 defect (repo-standard states 170 tests,
measures 243 — the catalog faithfully mirrors the plugin's own stale README, so
the fix belongs in repo-standard first). repo-standard's checks axis stays open:
catalog bullet says twelve, stat line says 14, plugin README says twelve,
measurement finds 19 exported check functions — ask, do not guess.

Coverage is now read by importing check-versions.mjs's own extractStatBadges and
extractCatalogStats rather than a second parser that could drift from the gate.

Also records two measurement defects found in the pass itself: git ls-tree quotes
non-ASCII paths, which silently dropped okr's two Norwegian-named commands and
read as a defect in okr; and a zsh loop whose set -- did not word-split reported
13/13 MATCH by comparing empty strings.

No ref, tag or stat line was touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wF4LZtSrfBFZr2h5ZPVHu
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 21:06:55 +02:00
commit aff807c101

View file

@ -58,18 +58,35 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
compares every number on that line against the plugin's own shields badge for the same axis, and
ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not
per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list
to maintain. **Re-measured 2026-08-04 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
27 badge-covered · 15 badge-less across 7 of the 12 plugins** and therefore ungated. That supersedes
the 2026-08-02 reading (`24 · 15 across 7 of 11`); only two entries moved in between and both
reconcile exactly — repo-mailbox now badges the two axes it did not (`+2` gated, `-2` ungated) and
repo-standard entered the catalog (`+1` gated, `+2` ungated). The 15 ungated axes were hand-audited
at their pinned tags — 13 on 2026-08-02 (**14 of that pass's 15 were exact**; the defects were
repo-mailbox `6 CLI scripts` (true 8) / `251 selftest checks`, and voyage `500+ tests` against a
measured 832), and repo-standard's two on 2026-08-04. Every ungated value currently stands measured
at the ref it is pinned to — verified on **both** sides 2026-08-04, because a ref that has not moved
only proves the *badge source* is unchanged and catalog prose is itself ungated: no ungated plugin's
ref has moved since its audit, **and** diffing `extractCatalogStats` against the 08-02 README shows
the only ungated numbers the catalog has restated since are repo-standard's two (absent then).
to maintain. **Re-measured 2026-08-13 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
24 badge-covered · 18 badge-less across 8 of the 12 plugins** and therefore ungated, with **0
disagreements** on the gated set. The measurement imports `check-versions.mjs`'s OWN exported
`extractStatBadges` / `extractCatalogStats` and feeds them `git show <ref>:README.md`, so coverage is
read by the same code that gates it rather than by a second parser that could drift.
**The prior `27 · 15 across 7` (2026-08-04) was correct and is not overturned** — re-running the
measurement against catalog commit `c7fbbd3` reproduces `27 · 15 across 7` exactly. Coverage FELL
because three axes lost their badge when a ref moved, each verified at both tags: `okr` v1.8.2 →
v1.10.0 dropped `agents-7`, `hooks-3` and `references-17` (the catalog restates the first two →
**2 gated**), and `graceful-handoff` v3.1.0 → v3.2.1 dropped `tests-30` and `hooks-0` (the catalog
restates only `test`**1 gated**). 27 3 = 24, 15 + 3 = 18, +1 ungated plugin (`okr`). This is
the CLAUDE.md-documented graceful-handoff badge-drop reaching a *release*: it was visible on `main`
on 2026-08-04 and is now what installs.
**All 18 ungated values were measured at their refs in the same pass (2026-08-13) — 16 exact, 1
correct-but-split, 1 defect.** Exact: voyage `24 agents` / `7 hooks` / `832 tests`, linkedin-studio
`6 skills`, graceful-handoff `1 pipeline` / `48 tests`, ai-psychosis `1 skill` / `1 command`,
ms-ai-architect `29 commands` / `5 skills` / `2 hooks`, okr `7 agents` / `3 hooks`, claude-design
`5 tests`. Split: voyage `6 commands (+1 helper)` measures 7 command files, and the split is the
plugin's own — its README at `v5.9.1` reads "6 slash commands (…) + trekendsession helper", and that
file's `description:` calls itself a helper. **Defect: repo-standard `170 tests` measures 243** at
`v0.11.1` (`node --test scripts/*.test.mjs`, the version its `package.json` declares). The catalog is
faithfully mirroring repo-standard's own README, which says "170 tests over the pure classifiers" at
that tag — so the plugin's prose is the stale source and the fix belongs there first. Understating,
not overstating, which is why nothing screamed.
**Also unresolved at `v0.11.1`: repo-standard's `14 checks`.** The catalog block contradicts itself —
the bullet says "**Twelve** checks", the stat line says **14** — and the plugin's own README says
"twelve checks". Measured: **19 exported `check*` functions** in `scripts/repo-standard-check.mjs`.
Three numbers, no agreement; "exported function" is not established as the unit a *user-visible*
check is counted in. **Do not pick one — ask repo-standard**, and carry the measured 19 into the ask.
**⚠️ The `251 selftest checks` correction is not fully settled**, though the weight is on 370: this
file *and* the catalog's own 08-02 stat line both record the axis as corrected to 370,
`check-versions.mjs` said 374 (most likely a transcription slip), and repo-mailbox's README at
@ -86,6 +103,23 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
matchers — the three diverge), and **tests = ` tests` from `node --test`**, not ` pass`
(config-audit's badge 1398 is its `tests` count; `pass` was 1375). Measure in an extraction of the
tag (`git archive <ref> | tar -x -C <tmp>`), never the sibling working tree.
- **Re-calibrated 2026-08-13 — the file-counting rules now have a measured referent, 11/11.**
`agents` = `agents/**/*.md` (4/4: llm-security 6, config-audit 7, ms-ai-architect 12,
linkedin-studio 20), `commands` = `commands/**/*.md` (3/3: config-audit 21, okr 16,
linkedin-studio 30), `skills` = `skills/*/SKILL.md` (4/4: repo-mailbox 3, claude-design 1,
graceful-handoff 1, repo-standard 1). `tests = tests` re-confirmed against config-audit
(` tests 1398` == badge 1398, `pass` 1374 — count `tests`, and note the suite need not be green
for the census to be valid). **`scanners` is NOT calibrated:** neither files-under-`scanners/`
(llm-security 55, config-audit 61) nor top-level `.mjs` (27 / 32) reproduces the badges (23 / 16),
so the axis counts something the tree does not name. Harmless today — **both** scanner claims are
gated and there is no ungated one to count — but do not invent a rule if that changes.
- **⚠️ `git ls-tree` QUOTES non-ASCII paths, and a naive count silently drops them.** okr's
`commands/innføring.md` and `commands/møter.md` come back as `"commands/innf\303\270ring.md"`, so a
`^commands/.+\.md$` match counted 14 against a badge of 16 and read as a defect in okr. It was a
defect in the measurement. **Always pass `-c core.quotePath=false`.** The same pass also produced a
fully bogus "13/13 MATCH" from a zsh loop where `set -- $spec` did not word-split, leaving every
field empty so `"" == ""` passed — **a verification that cannot fail has verified nothing.** Count
in Node, not in a shell loop.
- **`type: prompt` entries COUNT (decided 2026-08-13).** The rule read "hook COMMAND entries" until
now. That wording was calibrated in 2026-08 against plugins that predate prompt-hooks, so it was
never a ruling on them — it had no case to rule on. A prompt entry fires on the same event and
@ -100,8 +134,12 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
- **The `agents` axis IS badge-covered — 4 of the 6 plugins that claim it (measured 2026-08-13).**
`llm-security` `agents-6`, `config-audit` `agents-7`, `ms-ai-architect` `agents-12`,
`linkedin-studio` `agents-20`; all four match the catalog exactly and are gated. Ungated:
`voyage` (`24 agents`) and `okr` (`7 agents`) — badge-less, so those two numbers are unverified
until measured at their refs. Do **not** record this axis as uncalibrated: it has a referent.
`voyage` (`24 agents`) and `okr` (`7 agents`) — badge-less. **Both measured 2026-08-13 at their
refs: 24 and 7, exact.** Do **not** record this axis as uncalibrated: it has a referent.
- **A badge whose value is not an integer does not gate the axis.** `extractStatBadges` keeps only
integer-valued badges, so graceful-handoff's `Pipeline`/`STATE--helper-deterministic` badge leaves
`1 pipeline` ungated even though a badge for the axis visibly exists. When asking "is this axis
gated?", read the badge's VALUE, not its label.
- **The stat mirror reads the plugin README AT THE PINNED `ref`, never the sibling working tree.**
The catalog documents what *installs*, and that is the tag. A plugin that commits past its tag
without bumping its version — measured 2026-08-02 on both llm-security (scanners 23→22, tests