docs(catalog): re-measure badge coverage at every pinned ref — 24 gated, 18 ungated

The 2026-08-04 count (27 gated / 15 ungated across 7 plugins) was correct and is
not overturned: re-running the measurement against catalog commit c7fbbd3
reproduces it exactly. Coverage fell to 24/18 across 8 because three axes lost
their badge when a ref moved — okr v1.8.2 -> v1.10.0 dropped agents-7 and
hooks-3, graceful-handoff v3.1.0 -> v3.2.1 dropped tests-30. 27 - 3 = 24.

All 18 ungated values are now measured at their refs in the same pass rather
than inherited: 16 exact, 1 correct-but-split (voyage's 6 commands + 1 helper is
the plugin's own split, 7 files), 1 defect (repo-standard states 170 tests,
measures 243 — the catalog faithfully mirrors the plugin's own stale README, so
the fix belongs in repo-standard first). repo-standard's checks axis stays open:
catalog bullet says twelve, stat line says 14, plugin README says twelve,
measurement finds 19 exported check functions — ask, do not guess.

Coverage is now read by importing check-versions.mjs's own extractStatBadges and
extractCatalogStats rather than a second parser that could drift from the gate.

Also records two measurement defects found in the pass itself: git ls-tree quotes
non-ASCII paths, which silently dropped okr's two Norwegian-named commands and
read as a defect in okr; and a zsh loop whose set -- did not word-split reported
13/13 MATCH by comparing empty strings.

No ref, tag or stat line was touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wF4LZtSrfBFZr2h5ZPVHu
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 21:06:55 +02:00
commit aff807c101

View file

@ -58,18 +58,35 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
compares every number on that line against the plugin's own shields badge for the same axis, and compares every number on that line against the plugin's own shields badge for the same axis, and
ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not
per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list
to maintain. **Re-measured 2026-08-04 at the pinned refs, all 12 plugins: 42 catalog axis-claims — to maintain. **Re-measured 2026-08-13 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
27 badge-covered · 15 badge-less across 7 of the 12 plugins** and therefore ungated. That supersedes 24 badge-covered · 18 badge-less across 8 of the 12 plugins** and therefore ungated, with **0
the 2026-08-02 reading (`24 · 15 across 7 of 11`); only two entries moved in between and both disagreements** on the gated set. The measurement imports `check-versions.mjs`'s OWN exported
reconcile exactly — repo-mailbox now badges the two axes it did not (`+2` gated, `-2` ungated) and `extractStatBadges` / `extractCatalogStats` and feeds them `git show <ref>:README.md`, so coverage is
repo-standard entered the catalog (`+1` gated, `+2` ungated). The 15 ungated axes were hand-audited read by the same code that gates it rather than by a second parser that could drift.
at their pinned tags — 13 on 2026-08-02 (**14 of that pass's 15 were exact**; the defects were **The prior `27 · 15 across 7` (2026-08-04) was correct and is not overturned** — re-running the
repo-mailbox `6 CLI scripts` (true 8) / `251 selftest checks`, and voyage `500+ tests` against a measurement against catalog commit `c7fbbd3` reproduces `27 · 15 across 7` exactly. Coverage FELL
measured 832), and repo-standard's two on 2026-08-04. Every ungated value currently stands measured because three axes lost their badge when a ref moved, each verified at both tags: `okr` v1.8.2 →
at the ref it is pinned to — verified on **both** sides 2026-08-04, because a ref that has not moved v1.10.0 dropped `agents-7`, `hooks-3` and `references-17` (the catalog restates the first two →
only proves the *badge source* is unchanged and catalog prose is itself ungated: no ungated plugin's **−2 gated**), and `graceful-handoff` v3.1.0 → v3.2.1 dropped `tests-30` and `hooks-0` (the catalog
ref has moved since its audit, **and** diffing `extractCatalogStats` against the 08-02 README shows restates only `test` → **−1 gated**). 27 − 3 = 24, 15 + 3 = 18, +1 ungated plugin (`okr`). This is
the only ungated numbers the catalog has restated since are repo-standard's two (absent then). the CLAUDE.md-documented graceful-handoff badge-drop reaching a *release*: it was visible on `main`
on 2026-08-04 and is now what installs.
**All 18 ungated values were measured at their refs in the same pass (2026-08-13) — 16 exact, 1
correct-but-split, 1 defect.** Exact: voyage `24 agents` / `7 hooks` / `832 tests`, linkedin-studio
`6 skills`, graceful-handoff `1 pipeline` / `48 tests`, ai-psychosis `1 skill` / `1 command`,
ms-ai-architect `29 commands` / `5 skills` / `2 hooks`, okr `7 agents` / `3 hooks`, claude-design
`5 tests`. Split: voyage `6 commands (+1 helper)` measures 7 command files, and the split is the
plugin's own — its README at `v5.9.1` reads "6 slash commands (…) + trekendsession helper", and that
file's `description:` calls itself a helper. **Defect: repo-standard `170 tests` measures 243** at
`v0.11.1` (`node --test scripts/*.test.mjs`, the version its `package.json` declares). The catalog is
faithfully mirroring repo-standard's own README, which says "170 tests over the pure classifiers" at
that tag — so the plugin's prose is the stale source and the fix belongs there first. Understating,
not overstating, which is why nothing screamed.
**Also unresolved at `v0.11.1`: repo-standard's `14 checks`.** The catalog block contradicts itself —
the bullet says "**Twelve** checks", the stat line says **14** — and the plugin's own README says
"twelve checks". Measured: **19 exported `check*` functions** in `scripts/repo-standard-check.mjs`.
Three numbers, no agreement; "exported function" is not established as the unit a *user-visible*
check is counted in. **Do not pick one — ask repo-standard**, and carry the measured 19 into the ask.
**⚠️ The `251 selftest checks` correction is not fully settled**, though the weight is on 370: this **⚠️ The `251 selftest checks` correction is not fully settled**, though the weight is on 370: this
file *and* the catalog's own 08-02 stat line both record the axis as corrected to 370, file *and* the catalog's own 08-02 stat line both record the axis as corrected to 370,
`check-versions.mjs` said 374 (most likely a transcription slip), and repo-mailbox's README at `check-versions.mjs` said 374 (most likely a transcription slip), and repo-mailbox's README at
@ -86,6 +103,23 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
matchers — the three diverge), and **tests = `ℹ tests` from `node --test`**, not `ℹ pass` matchers — the three diverge), and **tests = `ℹ tests` from `node --test`**, not `ℹ pass`
(config-audit's badge 1398 is its `tests` count; `pass` was 1375). Measure in an extraction of the (config-audit's badge 1398 is its `tests` count; `pass` was 1375). Measure in an extraction of the
tag (`git archive <ref> | tar -x -C <tmp>`), never the sibling working tree. tag (`git archive <ref> | tar -x -C <tmp>`), never the sibling working tree.
- **Re-calibrated 2026-08-13 — the file-counting rules now have a measured referent, 11/11.**
`agents` = `agents/**/*.md` (4/4: llm-security 6, config-audit 7, ms-ai-architect 12,
linkedin-studio 20), `commands` = `commands/**/*.md` (3/3: config-audit 21, okr 16,
linkedin-studio 30), `skills` = `skills/*/SKILL.md` (4/4: repo-mailbox 3, claude-design 1,
graceful-handoff 1, repo-standard 1). `tests = ℹ tests` re-confirmed against config-audit
(`ℹ tests 1398` == badge 1398, `pass` 1374 — count `tests`, and note the suite need not be green
for the census to be valid). **`scanners` is NOT calibrated:** neither files-under-`scanners/`
(llm-security 55, config-audit 61) nor top-level `.mjs` (27 / 32) reproduces the badges (23 / 16),
so the axis counts something the tree does not name. Harmless today — **both** scanner claims are
gated and there is no ungated one to count — but do not invent a rule if that changes.
- **⚠️ `git ls-tree` QUOTES non-ASCII paths, and a naive count silently drops them.** okr's
`commands/innføring.md` and `commands/møter.md` come back as `"commands/innf\303\270ring.md"`, so a
`^commands/.+\.md$` match counted 14 against a badge of 16 and read as a defect in okr. It was a
defect in the measurement. **Always pass `-c core.quotePath=false`.** The same pass also produced a
fully bogus "13/13 MATCH" from a zsh loop where `set -- $spec` did not word-split, leaving every
field empty so `"" == ""` passed — **a verification that cannot fail has verified nothing.** Count
in Node, not in a shell loop.
- **`type: prompt` entries COUNT (decided 2026-08-13).** The rule read "hook COMMAND entries" until - **`type: prompt` entries COUNT (decided 2026-08-13).** The rule read "hook COMMAND entries" until
now. That wording was calibrated in 2026-08 against plugins that predate prompt-hooks, so it was now. That wording was calibrated in 2026-08 against plugins that predate prompt-hooks, so it was
never a ruling on them — it had no case to rule on. A prompt entry fires on the same event and never a ruling on them — it had no case to rule on. A prompt entry fires on the same event and
@ -100,8 +134,12 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
- **The `agents` axis IS badge-covered — 4 of the 6 plugins that claim it (measured 2026-08-13).** - **The `agents` axis IS badge-covered — 4 of the 6 plugins that claim it (measured 2026-08-13).**
`llm-security` `agents-6`, `config-audit` `agents-7`, `ms-ai-architect` `agents-12`, `llm-security` `agents-6`, `config-audit` `agents-7`, `ms-ai-architect` `agents-12`,
`linkedin-studio` `agents-20`; all four match the catalog exactly and are gated. Ungated: `linkedin-studio` `agents-20`; all four match the catalog exactly and are gated. Ungated:
`voyage` (`24 agents`) and `okr` (`7 agents`) — badge-less, so those two numbers are unverified `voyage` (`24 agents`) and `okr` (`7 agents`) — badge-less. **Both measured 2026-08-13 at their
until measured at their refs. Do **not** record this axis as uncalibrated: it has a referent. refs: 24 and 7, exact.** Do **not** record this axis as uncalibrated: it has a referent.
- **A badge whose value is not an integer does not gate the axis.** `extractStatBadges` keeps only
integer-valued badges, so graceful-handoff's `Pipeline`/`STATE--helper-deterministic` badge leaves
`1 pipeline` ungated even though a badge for the axis visibly exists. When asking "is this axis
gated?", read the badge's VALUE, not its label.
- **The stat mirror reads the plugin README AT THE PINNED `ref`, never the sibling working tree.** - **The stat mirror reads the plugin README AT THE PINNED `ref`, never the sibling working tree.**
The catalog documents what *installs*, and that is the tag. A plugin that commits past its tag The catalog documents what *installs*, and that is the tag. A plugin that commits past its tag
without bumping its version — measured 2026-08-02 on both llm-security (scanners 23→22, tests without bumping its version — measured 2026-08-02 on both llm-security (scanners 23→22, tests