docs(catalog): re-measure badge coverage at every pinned ref — 24 gated, 18 ungated
The 2026-08-04 count (27 gated / 15 ungated across 7 plugins) was correct and is
not overturned: re-running the measurement against catalog commit c7fbbd3
reproduces it exactly. Coverage fell to 24/18 across 8 because three axes lost
their badge when a ref moved — okr v1.8.2 -> v1.10.0 dropped agents-7 and
hooks-3, graceful-handoff v3.1.0 -> v3.2.1 dropped tests-30. 27 - 3 = 24.
All 18 ungated values are now measured at their refs in the same pass rather
than inherited: 16 exact, 1 correct-but-split (voyage's 6 commands + 1 helper is
the plugin's own split, 7 files), 1 defect (repo-standard states 170 tests,
measures 243 — the catalog faithfully mirrors the plugin's own stale README, so
the fix belongs in repo-standard first). repo-standard's checks axis stays open:
catalog bullet says twelve, stat line says 14, plugin README says twelve,
measurement finds 19 exported check functions — ask, do not guess.
Coverage is now read by importing check-versions.mjs's own extractStatBadges and
extractCatalogStats rather than a second parser that could drift from the gate.
Also records two measurement defects found in the pass itself: git ls-tree quotes
non-ASCII paths, which silently dropped okr's two Norwegian-named commands and
read as a defect in okr; and a zsh loop whose set -- did not word-split reported
13/13 MATCH by comparing empty strings.
No ref, tag or stat line was touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wF4LZtSrfBFZr2h5ZPVHu
This commit is contained in:
parent
e17229fb78
commit
aff807c101
1 changed files with 52 additions and 14 deletions
66
CLAUDE.md
66
CLAUDE.md
|
|
@ -58,18 +58,35 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
|
|||
compares every number on that line against the plugin's own shields badge for the same axis, and
|
||||
ERRORs when they disagree — the catalog must not overstate a plugin. **The rule is per-AXIS, not
|
||||
per-plugin:** an axis the plugin does not badge is skipped silently, so there is no exception list
|
||||
to maintain. **Re-measured 2026-08-04 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
|
||||
27 badge-covered · 15 badge-less across 7 of the 12 plugins** and therefore ungated. That supersedes
|
||||
the 2026-08-02 reading (`24 · 15 across 7 of 11`); only two entries moved in between and both
|
||||
reconcile exactly — repo-mailbox now badges the two axes it did not (`+2` gated, `-2` ungated) and
|
||||
repo-standard entered the catalog (`+1` gated, `+2` ungated). The 15 ungated axes were hand-audited
|
||||
at their pinned tags — 13 on 2026-08-02 (**14 of that pass's 15 were exact**; the defects were
|
||||
repo-mailbox `6 CLI scripts` (true 8) / `251 selftest checks`, and voyage `500+ tests` against a
|
||||
measured 832), and repo-standard's two on 2026-08-04. Every ungated value currently stands measured
|
||||
at the ref it is pinned to — verified on **both** sides 2026-08-04, because a ref that has not moved
|
||||
only proves the *badge source* is unchanged and catalog prose is itself ungated: no ungated plugin's
|
||||
ref has moved since its audit, **and** diffing `extractCatalogStats` against the 08-02 README shows
|
||||
the only ungated numbers the catalog has restated since are repo-standard's two (absent then).
|
||||
to maintain. **Re-measured 2026-08-13 at the pinned refs, all 12 plugins: 42 catalog axis-claims —
|
||||
24 badge-covered · 18 badge-less across 8 of the 12 plugins** and therefore ungated, with **0
|
||||
disagreements** on the gated set. The measurement imports `check-versions.mjs`'s OWN exported
|
||||
`extractStatBadges` / `extractCatalogStats` and feeds them `git show <ref>:README.md`, so coverage is
|
||||
read by the same code that gates it rather than by a second parser that could drift.
|
||||
**The prior `27 · 15 across 7` (2026-08-04) was correct and is not overturned** — re-running the
|
||||
measurement against catalog commit `c7fbbd3` reproduces `27 · 15 across 7` exactly. Coverage FELL
|
||||
because three axes lost their badge when a ref moved, each verified at both tags: `okr` v1.8.2 →
|
||||
v1.10.0 dropped `agents-7`, `hooks-3` and `references-17` (the catalog restates the first two →
|
||||
**−2 gated**), and `graceful-handoff` v3.1.0 → v3.2.1 dropped `tests-30` and `hooks-0` (the catalog
|
||||
restates only `test` → **−1 gated**). 27 − 3 = 24, 15 + 3 = 18, +1 ungated plugin (`okr`). This is
|
||||
the CLAUDE.md-documented graceful-handoff badge-drop reaching a *release*: it was visible on `main`
|
||||
on 2026-08-04 and is now what installs.
|
||||
**All 18 ungated values were measured at their refs in the same pass (2026-08-13) — 16 exact, 1
|
||||
correct-but-split, 1 defect.** Exact: voyage `24 agents` / `7 hooks` / `832 tests`, linkedin-studio
|
||||
`6 skills`, graceful-handoff `1 pipeline` / `48 tests`, ai-psychosis `1 skill` / `1 command`,
|
||||
ms-ai-architect `29 commands` / `5 skills` / `2 hooks`, okr `7 agents` / `3 hooks`, claude-design
|
||||
`5 tests`. Split: voyage `6 commands (+1 helper)` measures 7 command files, and the split is the
|
||||
plugin's own — its README at `v5.9.1` reads "6 slash commands (…) + trekendsession helper", and that
|
||||
file's `description:` calls itself a helper. **Defect: repo-standard `170 tests` measures 243** at
|
||||
`v0.11.1` (`node --test scripts/*.test.mjs`, the version its `package.json` declares). The catalog is
|
||||
faithfully mirroring repo-standard's own README, which says "170 tests over the pure classifiers" at
|
||||
that tag — so the plugin's prose is the stale source and the fix belongs there first. Understating,
|
||||
not overstating, which is why nothing screamed.
|
||||
**Also unresolved at `v0.11.1`: repo-standard's `14 checks`.** The catalog block contradicts itself —
|
||||
the bullet says "**Twelve** checks", the stat line says **14** — and the plugin's own README says
|
||||
"twelve checks". Measured: **19 exported `check*` functions** in `scripts/repo-standard-check.mjs`.
|
||||
Three numbers, no agreement; "exported function" is not established as the unit a *user-visible*
|
||||
check is counted in. **Do not pick one — ask repo-standard**, and carry the measured 19 into the ask.
|
||||
**⚠️ The `251 selftest checks` correction is not fully settled**, though the weight is on 370: this
|
||||
file *and* the catalog's own 08-02 stat line both record the axis as corrected to 370,
|
||||
`check-versions.mjs` said 374 (most likely a transcription slip), and repo-mailbox's README at
|
||||
|
|
@ -86,6 +103,23 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
|
|||
matchers — the three diverge), and **tests = `ℹ tests` from `node --test`**, not `ℹ pass`
|
||||
(config-audit's badge 1398 is its `tests` count; `pass` was 1375). Measure in an extraction of the
|
||||
tag (`git archive <ref> | tar -x -C <tmp>`), never the sibling working tree.
|
||||
- **Re-calibrated 2026-08-13 — the file-counting rules now have a measured referent, 11/11.**
|
||||
`agents` = `agents/**/*.md` (4/4: llm-security 6, config-audit 7, ms-ai-architect 12,
|
||||
linkedin-studio 20), `commands` = `commands/**/*.md` (3/3: config-audit 21, okr 16,
|
||||
linkedin-studio 30), `skills` = `skills/*/SKILL.md` (4/4: repo-mailbox 3, claude-design 1,
|
||||
graceful-handoff 1, repo-standard 1). `tests = ℹ tests` re-confirmed against config-audit
|
||||
(`ℹ tests 1398` == badge 1398, `pass` 1374 — count `tests`, and note the suite need not be green
|
||||
for the census to be valid). **`scanners` is NOT calibrated:** neither files-under-`scanners/`
|
||||
(llm-security 55, config-audit 61) nor top-level `.mjs` (27 / 32) reproduces the badges (23 / 16),
|
||||
so the axis counts something the tree does not name. Harmless today — **both** scanner claims are
|
||||
gated and there is no ungated one to count — but do not invent a rule if that changes.
|
||||
- **⚠️ `git ls-tree` QUOTES non-ASCII paths, and a naive count silently drops them.** okr's
|
||||
`commands/innføring.md` and `commands/møter.md` come back as `"commands/innf\303\270ring.md"`, so a
|
||||
`^commands/.+\.md$` match counted 14 against a badge of 16 and read as a defect in okr. It was a
|
||||
defect in the measurement. **Always pass `-c core.quotePath=false`.** The same pass also produced a
|
||||
fully bogus "13/13 MATCH" from a zsh loop where `set -- $spec` did not word-split, leaving every
|
||||
field empty so `"" == ""` passed — **a verification that cannot fail has verified nothing.** Count
|
||||
in Node, not in a shell loop.
|
||||
- **`type: prompt` entries COUNT (decided 2026-08-13).** The rule read "hook COMMAND entries" until
|
||||
now. That wording was calibrated in 2026-08 against plugins that predate prompt-hooks, so it was
|
||||
never a ruling on them — it had no case to rule on. A prompt entry fires on the same event and
|
||||
|
|
@ -100,8 +134,12 @@ their own Forgejo repositories under `https://git.fromaitochitta.com/open/`.
|
|||
- **The `agents` axis IS badge-covered — 4 of the 6 plugins that claim it (measured 2026-08-13).**
|
||||
`llm-security` `agents-6`, `config-audit` `agents-7`, `ms-ai-architect` `agents-12`,
|
||||
`linkedin-studio` `agents-20`; all four match the catalog exactly and are gated. Ungated:
|
||||
`voyage` (`24 agents`) and `okr` (`7 agents`) — badge-less, so those two numbers are unverified
|
||||
until measured at their refs. Do **not** record this axis as uncalibrated: it has a referent.
|
||||
`voyage` (`24 agents`) and `okr` (`7 agents`) — badge-less. **Both measured 2026-08-13 at their
|
||||
refs: 24 and 7, exact.** Do **not** record this axis as uncalibrated: it has a referent.
|
||||
- **A badge whose value is not an integer does not gate the axis.** `extractStatBadges` keeps only
|
||||
integer-valued badges, so graceful-handoff's `Pipeline`/`STATE--helper-deterministic` badge leaves
|
||||
`1 pipeline` ungated even though a badge for the axis visibly exists. When asking "is this axis
|
||||
gated?", read the badge's VALUE, not its label.
|
||||
- **The stat mirror reads the plugin README AT THE PINNED `ref`, never the sibling working tree.**
|
||||
The catalog documents what *installs*, and that is the tag. A plugin that commits past its tag
|
||||
without bumping its version — measured 2026-08-02 on both llm-security (scanners 23→22, tests
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue