Kjell Tore Guttormsen
dc386d4471
Adversarial deep research (25 sources, 124 claims extracted, 25 verified: 11 confirmed / 14 refuted) plus direct measurement of all 18 cloned open/ repos. The useful half of the result is what it REFUSED to support, so the research is recorded in docs/presentation-research-2026-08-03.md rather than being spent and forgotten. - BADGE-COUNT (WARN) — past five badges. Trockman et al., ICSE 2018 (n=294,941 npm packages) measured a non-linear relationship with popularity inflecting at five, motivated by surveyed maintainers calling over-badged READMEs cluttered and "trying too hard". WARN and never ERROR: the coefficient sits in an appendix with no CI or p-value. Counting deliberately uses a NARROWER rule than the existing claim check, so a screenshot or an architecture diagram is never counted as clutter. Fires on 8 of 18. - README-LANGUAGE (WARN) — prose not in the language this repo's readers were declared to speak, via a new `locales` axis in the register. Class is structural, a trait is what the code DOES, a locale is who it is FOR — the standard's own "who the reader is decides what is required". English is the default; ms-ai-architect and okr are declared nb, named by the operator as Norway-only in audience. Stopword-frequency comparison over prose with code stripped: a Norwegian flag name in a shell example cannot decide the document. Fires on exactly those two, silent on all sixteen English repos. One design correction found mid-implementation: the first version returned SKIP when a README had too little prose to judge, which broke a passing fixture and would have stopped any terse repo from ever reaching OK. SKIP is for a check that could not RUN; this one ran, saw everything and found no prose to be in the wrong language — the same shape as "no licence claim to back". Insufficient prose is now OK, and evenly bilingual prose is the SKIP, because there the question is live and unanswered. Deliberately NOT built, because the evidence does not reach: any rule about images, diagrams or terminal recordings (every such claim refuted 0-3); a README length bound (no evidence-based target exists); a section count (would fire on 9 of 18 — textbook "suspect the CHECK"); Mermaid source length and #gh-dark-mode-only (zero occurrences, and the instance limit is not readable via the API, so any threshold would be a guess). Verified live against the operator's own forge (15.0.6+gitea-1.22.0): Mermaid DOES render in README.md — two div.mermaid-block iframes carrying real SVG — while #gh-dark-mode-only landed only in Gitea 1.26.0 and is unavailable here. 103 tests green, up from 92. No version bump: the catalog ref still trails at v0.1.1 against 0.1.3, and starting a second release chain over that is the operator's call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCAGKZT8h9F46ygzSkhEee
90 lines
4.9 KiB
Markdown
90 lines
4.9 KiB
Markdown
# repo-standard
|
|
|
|
Per-repo gate for the `open/` presentation standard, packaged as a marketplace
|
|
plugin.
|
|
|
|
## Context
|
|
|
|
Two components, one boundary:
|
|
|
|
- **Engine (`scripts/repo-standard-check.mjs`)** — pure classifiers with all I/O
|
|
resolved into their input, mirroring the marketplace's `check-versions.mjs`.
|
|
Findings are `ERROR`/`WARN`/`SKIP`/`OK`; exit 1 on `ERROR`. Pinned by
|
|
`scripts/repo-standard-check.test.mjs` (`npm test`).
|
|
- **Skill (`skills/repo-standard/`)** — the judgement the script cannot encode.
|
|
No checking logic lives here; it calls the engine.
|
|
|
|
`register/repos.json` is the single taxonomy register (name → class, per-class
|
|
requirements, and the known non-repo names). Central by design: per-repo copies
|
|
would recreate, in data, exactly the drift this plugin exists to remove.
|
|
|
|
## Invariants
|
|
|
|
- **This repo has a PUBLIC remote.** `STATE.md` is LOCAL-ONLY and gitignored
|
|
from the first commit. Repos on a private remote track theirs — do not carry
|
|
that habit across in either direction.
|
|
- **The gate sees ONE repo.** Anything needing a view across the whole org
|
|
(topic coverage, competing install forms, catalog-vs-forge divergence) does
|
|
not belong here. It is measured where the org is enumerated.
|
|
- **It records, it does not fix.** Findings first, remediation afterwards.
|
|
Patching while measuring is how the inconsistency it detects was produced.
|
|
- **`SKIP` is never a pass.** A check that could not run says so and names why.
|
|
- **When a check fires many times in one repo, suspect the CHECK.** The first
|
|
link pass produced ~30 findings against `llm-security` and all were noise —
|
|
regexes in code spans, `file:` URLs, relative paths resolved against the wrong
|
|
directory. Strip code before scanning text; resolve links against the file
|
|
they sit in. A gate that is wrong this often gets switched off.
|
|
- **Two axes on every finding.** Level (`ERROR`/`WARN`/`SKIP`/`OK`) and bucket
|
|
(`broken`/`missing`/`weakening`). They are independent — a `weakening` finding
|
|
can be an `ERROR`.
|
|
- **Class is structural, traits are judgement.** Class is read off the catalog
|
|
and the remotes. A trait (`security`) says what the code does, which no remote
|
|
knows, and the operator owns that list.
|
|
- **Who the reader is decides what is required.** Contributor-facing documents
|
|
are required by no class — solo-maintained, and the published stance says so.
|
|
Consumer-facing ones (`SECURITY.md`, `LICENSE`, non-goals, limitations) are
|
|
untouched by that. This is not a rule against having the others.
|
|
- **No CI badge, because there is no CI.** The forge has zero Actions runners
|
|
registered (measured). The substitute is one command from a clean clone, said
|
|
plainly. A static badge asserting a run is the anti-pattern this gate flags —
|
|
and an early draft of this README carried one.
|
|
- **Three outcomes on references.** "No match" and "match on a known non-repo"
|
|
must stay distinct findings. Collapsing them hides real loss inside correct
|
|
text — the exact defect class this gate exists to catch.
|
|
- **One API call, anonymous.** The org listing carries description and topics
|
|
already; per-repo fetching trips the rate limiter (HTTP 429). It reads without
|
|
a token, so the gate works for any reader — a public plugin whose documented
|
|
check only runs for its author is a broken plugin.
|
|
- **Codepoints, not bytes, not UTF-16 units.** Use `[...s].length`. An em-dash
|
|
exposes only the byte layer; astral characters expose the rest.
|
|
- **The reader decides a link's level, not just what is required.** Root
|
|
documents are the shop window — a dead link there is an `ERROR`. Below the
|
|
root it is a `WARN`: that is where session plans, agent working files and
|
|
path-traversal fixtures with deliberately invalid targets live. Measured, 30
|
|
of 43 findings were down there and all were `ERROR`s.
|
|
- **A repo's name is its remote, not its directory.** `catalog/` holds
|
|
`ktg-plugin-marketplace`. The basename left it unregistered with zero checks
|
|
run, against the one repo every catalog rule depends on.
|
|
- **No hook until the rule is precise.** A blocking gate that fails a correct
|
|
repository is the mechanism that gets gates switched off.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
npm test # 103 tests
|
|
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
|
|
node scripts/repo-standard-check.mjs --offline # no network call
|
|
node scripts/repo-standard-check.mjs --json # machine output
|
|
node scripts/repo-standard-check.mjs --refresh # register vs. forge
|
|
```
|
|
|
|
## Release
|
|
|
|
Polyrepo rule: a version bump is not finished until the tag `vX.Y.Z` is pushed
|
|
**and** the catalog `ref` is bumped to it. Use `release-plugin.mjs`, never a
|
|
hand-edited `ref`.
|
|
|
|
`check-versions.mjs` reads the catalog README's per-plugin label, and a **missing**
|
|
entry is a silent `null` rather than an error — `release-plugin.mjs` rewrites an
|
|
existing heading but cannot create one. A new plugin's catalog README entry has
|
|
to be added by hand once.
|