Measured 2026-08-04: two repos (llm-security, config-audit) independently
proved the /repo-standard skill had resolved ${CLAUDE_PLUGIN_ROOT} to a
cached 0.1.1 while 0.2.0 was installed and the catalog pinned it — same
instruction, same variable, wrong engine. 0.1.1 has neither BADGE-COUNT nor
README-LANGUAGE, so a broadcast recommending the skill produced clean-looking
runs that could not have found what they were sent to find.
The header line and --json output now carry `repo-standard v<version>`
(headerLine/withEngineVersion, both pure and unit tested). SKILL.md tells the
reader to confirm it against the catalog pin before trusting green. The
stale-cache resolution itself is the harness's, not this repo's — not fixed
here, only made visible instead of silent.
TDD: 4 new tests. 111/111 green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01496ZWasKPnA627crFBXWhe
238 lines
14 KiB
Markdown
238 lines
14 KiB
Markdown
# Changelog
|
|
|
|
All notable changes to this project are documented here.
|
|
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
|
|
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
|
## [0.2.2] — 2026-08-04
|
|
|
|
### Fixed
|
|
|
|
- The engine never printed its own version, which is how a stale plugin cache
|
|
went unnoticed: two repos independently proved the `/repo-standard` skill
|
|
had resolved `${CLAUDE_PLUGIN_ROOT}` to a cached `0.1.1` while `0.2.0` was
|
|
installed and the catalog pinned it — same instruction, wrong engine, and
|
|
`0.1.1` has neither `BADGE-COUNT` nor `README-LANGUAGE`, so the run looked
|
|
clean instead of incomplete. The header line and `--json` output now both
|
|
carry `repo-standard v<version>`; `SKILL.md` tells the reader to confirm it
|
|
against the catalog pin before trusting a green result. The cache
|
|
resolution itself is the harness's, not this repo's, and is not fixed here.
|
|
|
|
## [0.2.1] — 2026-08-04
|
|
|
|
### Fixed
|
|
|
|
- The gate makes two anonymous forge calls per invocation (org listing +
|
|
catalog `marketplace.json`), not one — this repo's own `CLAUDE.md` said
|
|
"one" from before `INSTALL-TRUTH` added the second, and stayed wrong long
|
|
enough that a 13-repo shell loop trusted the count and tripped an HTTP 429
|
|
at 26 requests. Both calls now go through `fetchWithRetry`, which honors
|
|
`Retry-After` and retries instead of silently reporting `SKIP` on a
|
|
transient rate limit. `CLAUDE.md` corrected to match.
|
|
|
|
## [0.2.0] — 2026-08-04
|
|
|
|
### Added
|
|
|
|
Two presentation checks, both from adversarially-verified research rather than
|
|
taste — `docs/presentation-research-2026-08-03.md` records what the evidence
|
|
supports and, more usefully, what it refuses to support.
|
|
|
|
- `BADGE-COUNT` (`WARN`) — more than five badges. Trockman et al., ICSE 2018
|
|
(n=294,941 npm packages) measured a non-linear relationship with popularity
|
|
inflecting at five, motivated by surveyed maintainers calling over-badged
|
|
READMEs cluttered and "trying too hard". `WARN` and never `ERROR`: the
|
|
coefficient sits in an appendix without CI or p-value, so it carries "more is
|
|
not better" and cannot carry a hard limit. Counting uses a narrower rule than
|
|
the existing claim check — a screenshot or architecture diagram must not be
|
|
counted as clutter. Measured: 8 of 18 `open/` repos are past it.
|
|
- `README-LANGUAGE` (`WARN`) — the prose is not in the language this repo's
|
|
readers were declared to speak, via a new `locales` axis in the register.
|
|
English is the default; `ms-ai-architect` and `okr` are declared `nb` as
|
|
Norway-only in audience. Detection is a stopword-frequency comparison over
|
|
prose with code stripped, so a Norwegian flag name in a shell example cannot
|
|
decide the document. Evenly bilingual prose is a `SKIP` — the question is
|
|
live and unanswered. No running prose is an `OK`: nothing claims a language,
|
|
and a thin README is `checkFirstScreen`'s business. Measured: fires on
|
|
exactly the two declared repos, silent on all sixteen English ones.
|
|
|
|
Deliberately **not** built, because the evidence does not reach: any rule about
|
|
images, diagrams, screenshots or terminal recordings (every such claim was
|
|
refuted 0-3); a README length bound (no evidence-based target exists); a section
|
|
count (would fire on 9 of 18 — textbook "suspect the CHECK"); Mermaid source
|
|
length and `#gh-dark-mode-only` (zero occurrences, and the instance's actual
|
|
limit is not readable via the API, so any threshold would be a guess).
|
|
|
|
### Changed
|
|
|
|
- The skill now carries diagram guidance, deliberately as judgement and not as a
|
|
check. Mermaid is verified to render on this forge; that is capability, not
|
|
justification, and every claim that visuals improve a README was refuted. What
|
|
the guidance covers is a diagram that already exists — height, node labels,
|
|
the silent degradation past `MERMAID_MAX_SOURCE_CHARACTERS`, and the fact that
|
|
`#gh-dark-mode-only` is unavailable below Gitea 1.26.0. The org's own measured
|
|
example renders at 1,459 px and wraps node text mid-word while its neighbour
|
|
sits at 191 px: rendering and rendering well are separate questions.
|
|
|
|
## [0.1.3] — 2026-08-03
|
|
|
|
### Fixed
|
|
|
|
Three more measured false positives, all reported via coord the same day
|
|
0.1.2 shipped, from repos running the gate for the first time.
|
|
|
|
- `extractChangelogTop` truncated a PEP 440 pre-release (`0.5.0a2`) to `0.5.0`,
|
|
so `VERSION-CHANGELOG` disagreed with `VERSION-TAG` — which compares the
|
|
untruncated tag and does not have this problem — over a version the manifest
|
|
and the tag already agreed on. Reported by `llm-ingestion-okf` with a
|
|
repro. An affected repo could reach 0 ERROR only by lying about its version;
|
|
now the pre-release suffix is kept.
|
|
- `BADGE-STATIC-CLAIM` treated a bare `status` badge (`status-alpha`,
|
|
`status-experimental`) as a run claim, the same as `tests-34-passing`.
|
|
A self-declared maturity label asserts no run — the same class as version,
|
|
licence and platform, already exempt. Reported by
|
|
`llm-ingestion-pipeline-security`. `build`/`ci`/`passing` still catch the
|
|
run-asserting compounds ("build status", "CI status"); only the bare word
|
|
is dropped.
|
|
- `BOILERPLATE` flagged a bare `FIXME` even when it named the *convention*
|
|
rather than an instance of it — a scanner's own docs describing what it
|
|
detects ("Flag TODO/FIXME markers…", a table row listing "TODO/FIXME
|
|
comments" as a finding type) read the same as a forgotten marker. Reported
|
|
by `config-audit`, whose product literally is a marker scanner. "TODO/FIXME"
|
|
named together is now read as the convention; a lone `FIXME` is still
|
|
caught.
|
|
|
|
## [0.1.2] — 2026-08-03
|
|
|
|
### Fixed
|
|
|
|
- `checkInternalLinks` compared a link's resolved target only against
|
|
`present` — the set of tracked *files* — so a link to a directory
|
|
(`[x](dir/)`) was reported `LINK-INTERNAL-MISSING` even when every file
|
|
under that directory was tracked. A directory was never a member of that
|
|
set to begin with. Reported by `portfolio-optimiser-claude` (coord
|
|
20260803T194933Z) with a minimal repro; confirmed and traced to the same
|
|
defect in `voyage`, `linkedin-studio` and `portfolio-optimiser` — 12 of the
|
|
71 ERROR findings measured across the org on 2026-08-03 were this one check,
|
|
not twelve separate repo problems. Fixed by deriving the set of directories
|
|
that actually contain a tracked file from `present` itself, and letting a
|
|
link that resolves to one of them pass.
|
|
|
|
## [0.1.1] — 2026-08-03
|
|
|
|
Documentation only. No behaviour change: `scripts/` and `register/` are
|
|
unchanged from 0.1.0 — this release carries version strings and the correction
|
|
below.
|
|
|
|
### Fixed
|
|
|
|
- The 0.1.0 entry said the gate has **eleven** checks. It has twelve, which is
|
|
what both the code and the README table already said. The entry is corrected
|
|
in place, and this release exists so the catalog's pinned tag serves the
|
|
corrected text instead of the wrong count. The count is drift of exactly the
|
|
kind this plugin was built to catch, and it was sitting in the plugin's own
|
|
repository.
|
|
|
|
## [0.1.0] — 2026-07-27
|
|
|
|
First release. Covers the checks that a single repository can answer on its own.
|
|
|
|
### Added
|
|
|
|
- **The gate** (`scripts/repo-standard-check.mjs`) — pure classifiers with all
|
|
I/O resolved into their input, findings tagged `ERROR`/`WARN`/`SKIP`/`OK`,
|
|
exit 1 on `ERROR`. Twelve checks: README first screen, install block, install
|
|
truth, required headings, files required by the repo's class and traits,
|
|
`open/<name>` references, relative links, licence claim, badges, boilerplate,
|
|
version consistency and description length.
|
|
- **Taxonomy register** (`register/repos.json`) — one central file mapping each
|
|
repository to its class, plus the per-class file and install requirements.
|
|
`--refresh` compares it against the live org listing.
|
|
- **Three-outcome reference check** — "matches no repository" (`ERROR`) is a
|
|
separate finding from "matches something that is deliberately not a
|
|
repository" (`WARN`). Sharing an outcome would let real dead links hide among
|
|
correct text. Only names in URL position are treated as references; the `.git`
|
|
suffix is normalised first.
|
|
- **Two-axis findings** — every `ERROR`/`WARN` carries a **bucket** alongside its
|
|
level: `broken` (a stranger is blocked or misled), `missing` (an expected
|
|
artefact is absent), `weakening` (present, but it reads as amateur). Output is
|
|
grouped by bucket, because that is the order findings get acted on.
|
|
- **Traits, a second axis beside class** — class is structural and readable off
|
|
the catalog; a trait is about what the code does. `security` attaches a
|
|
`SECURITY.md` requirement and a `## Known limitations` section.
|
|
- **Required README headings per class** — `## Install`, `## Non-goals`,
|
|
`## Changelog`. A heading present at the wrong depth is its own finding, not
|
|
"missing".
|
|
- **In-repo version consistency** — manifest, README version badge, newest
|
|
CHANGELOG entry and the git tag must agree. An untagged repo `SKIP`s the tag
|
|
comparison rather than failing it.
|
|
- **Badge honesty** — a static badge asserting a test, build or coverage run is
|
|
a claim dressed as evidence. Version, licence and platform badges assert no
|
|
run and are fine.
|
|
- **Install truth, not just install syntax** — the marketplace URL in the install
|
|
block must be the real one (offline, against the register), and the plugin must
|
|
actually be pinned in the catalog (one call; `SKIP` if unreachable). A perfectly
|
|
formed `claude plugin install x@mkt` fails silently when `x` was never pinned,
|
|
and that is the single most disqualifying thing a repo can publish.
|
|
- **Boilerplate, licence-claim and relative-link checks.**
|
|
- **The skill** (`skills/repo-standard/`) — the judgement the script cannot
|
|
encode: what a description should say, why the summary card must never be used
|
|
to verify one, what not to retrofit, and where the per-repo boundary is.
|
|
- **Link level follows the reader.** A dead relative link in a root document —
|
|
README, CHANGELOG, SECURITY — is an `ERROR`: it is in the shop window. The same
|
|
link below the root is a `WARN`. Measured across seventeen repositories: 30 of
|
|
43 findings sat in `shared/`, `docs/plan/` and `.claude/` — session plans,
|
|
agent working files, and path-traversal test fixtures whose targets are invalid
|
|
on purpose. All 30 were `ERROR`s, which is how a gate gets switched off. Only
|
|
the level moves; every finding is still reported with its file and line.
|
|
- **The repo name comes from the remote, not the directory.** `catalog/` is the
|
|
working directory of the repo named `ktg-plugin-marketplace`. Reading the
|
|
basename left it `REPO-UNREGISTERED` with zero checks run — against the one
|
|
repo every catalog rule depends on. The scp form the forge's clone button hands
|
|
out is handled, and a bare host is not a repo name.
|
|
- 84 tests over the pure classifiers, using measured false positives as
|
|
reference fixtures.
|
|
|
|
### Notes
|
|
|
|
- Descriptions are measured in **codepoints** — not bytes, and not UTF-16 units.
|
|
The same string measures 248 / 249 / 253 across those three yardsticks when it
|
|
contains an astral character.
|
|
- `enabledPlugins` in `settings.json` is treated as a legitimate second install
|
|
form. The gate requires a CLI command *as well*, and never reports the JSON
|
|
form as the defect.
|
|
- Two network calls, both anonymous: the org listing and the catalog manifest.
|
|
Per-repo fetching trips the forge's rate limiter, so neither is ever done per
|
|
repository. `--offline` skips both, and the checks that needed them report
|
|
`SKIP` rather than passing.
|
|
- `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md` and `MAINTAINERS.md` are required by
|
|
no class: the project is solo-maintained and publishes that stance. This is
|
|
not a rule against having them — files already present are a cleanup decision,
|
|
not a gate finding. `SECURITY.md` is unaffected; it exists for the outsider who
|
|
finds a hole.
|
|
- There is no CI badge because there is no CI: the forge has no Actions runner
|
|
registered. The substitute is one command from a clean clone, stated honestly.
|
|
A badge asserting the test count would be the very anti-pattern the gate flags,
|
|
and an earlier draft of this README carried exactly that.
|
|
- The link and boilerplate checks ignore fenced blocks and inline code spans.
|
|
Documentation about regexes and placeholders is full of text that is a markdown
|
|
link to a naive scanner — the first version produced about thirty findings
|
|
against a real repository and every one was noise.
|
|
- Relative links resolve against the file they appear in, not the repo root.
|
|
- Indented text counts as a code block only when a blank line opens it, and then
|
|
continues while the indent holds. The first rule alone made links inside nested
|
|
list items invisible; the second alone let everything after a block's first line
|
|
leak back in. Both were silent false passes, which is worse than noise — the
|
|
second was caught by the gate on this plugin's own SKILL.md.
|
|
- A README H1 that differs from the repo name is a `WARN`, not an `ERROR`. The
|
|
thread that has to hold is description == catalog == opening line, and the H1
|
|
is none of those three; a human title is a naming choice the operator owns.
|
|
A missing H1 is still an `ERROR`.
|
|
- No hook ships in this release. A blocking gate has to be precise enough not to
|
|
fail a correct repository first.
|
|
|
|
[0.2.0]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.2.0
|
|
[0.1.3]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.3
|
|
[0.1.2]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.2
|
|
[0.1.1]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.1
|
|
[0.1.0]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.0
|