Kjell Tore Guttormsen dc386d4471 feat(gate): two presentation checks the evidence actually supports

Adversarial deep research (25 sources, 124 claims extracted, 25 verified:
11 confirmed / 14 refuted) plus direct measurement of all 18 cloned open/
repos. The useful half of the result is what it REFUSED to support, so the
research is recorded in docs/presentation-research-2026-08-03.md rather than
being spent and forgotten.

- BADGE-COUNT (WARN) — past five badges. Trockman et al., ICSE 2018
  (n=294,941 npm packages) measured a non-linear relationship with popularity
  inflecting at five, motivated by surveyed maintainers calling over-badged
  READMEs cluttered and "trying too hard". WARN and never ERROR: the
  coefficient sits in an appendix with no CI or p-value. Counting deliberately
  uses a NARROWER rule than the existing claim check, so a screenshot or an
  architecture diagram is never counted as clutter. Fires on 8 of 18.
- README-LANGUAGE (WARN) — prose not in the language this repo's readers were
  declared to speak, via a new `locales` axis in the register. Class is
  structural, a trait is what the code DOES, a locale is who it is FOR — the
  standard's own "who the reader is decides what is required". English is the
  default; ms-ai-architect and okr are declared nb, named by the operator as
  Norway-only in audience. Stopword-frequency comparison over prose with code
  stripped: a Norwegian flag name in a shell example cannot decide the
  document. Fires on exactly those two, silent on all sixteen English repos.

One design correction found mid-implementation: the first version returned
SKIP when a README had too little prose to judge, which broke a passing
fixture and would have stopped any terse repo from ever reaching OK. SKIP is
for a check that could not RUN; this one ran, saw everything and found no
prose to be in the wrong language — the same shape as "no licence claim to
back". Insufficient prose is now OK, and evenly bilingual prose is the SKIP,
because there the question is live and unanswered.

Deliberately NOT built, because the evidence does not reach: any rule about
images, diagrams or terminal recordings (every such claim refuted 0-3); a
README length bound (no evidence-based target exists); a section count (would
fire on 9 of 18 — textbook "suspect the CHECK"); Mermaid source length and
#gh-dark-mode-only (zero occurrences, and the instance limit is not readable
via the API, so any threshold would be a guess).

Verified live against the operator's own forge (15.0.6+gitea-1.22.0): Mermaid
DOES render in README.md — two div.mermaid-block iframes carrying real SVG —
while #gh-dark-mode-only landed only in Gitea 1.26.0 and is unavailable here.

103 tests green, up from 92. No version bump: the catalog ref still trails at
v0.1.1 against 0.1.3, and starting a second release chain over that is the
operator's call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TCAGKZT8h9F46ygzSkhEee

2026-08-04 09:41:53 +02:00

4.9 KiB

Raw Blame History

repo-standard

Per-repo gate for the open/ presentation standard, packaged as a marketplace plugin.

Context

Two components, one boundary:

Engine (scripts/repo-standard-check.mjs) — pure classifiers with all I/O resolved into their input, mirroring the marketplace's check-versions.mjs. Findings are ERROR/WARN/SKIP/OK; exit 1 on ERROR. Pinned by scripts/repo-standard-check.test.mjs (npm test).
Skill (skills/repo-standard/) — the judgement the script cannot encode. No checking logic lives here; it calls the engine.

register/repos.json is the single taxonomy register (name → class, per-class requirements, and the known non-repo names). Central by design: per-repo copies would recreate, in data, exactly the drift this plugin exists to remove.

Invariants

This repo has a PUBLIC remote. STATE.md is LOCAL-ONLY and gitignored from the first commit. Repos on a private remote track theirs — do not carry that habit across in either direction.
The gate sees ONE repo. Anything needing a view across the whole org (topic coverage, competing install forms, catalog-vs-forge divergence) does not belong here. It is measured where the org is enumerated.
It records, it does not fix. Findings first, remediation afterwards. Patching while measuring is how the inconsistency it detects was produced.
SKIP is never a pass. A check that could not run says so and names why.
When a check fires many times in one repo, suspect the CHECK. The first link pass produced ~30 findings against llm-security and all were noise — regexes in code spans, file: URLs, relative paths resolved against the wrong directory. Strip code before scanning text; resolve links against the file they sit in. A gate that is wrong this often gets switched off.
Two axes on every finding. Level (ERROR/WARN/SKIP/OK) and bucket (broken/missing/weakening). They are independent — a weakening finding can be an ERROR.
Class is structural, traits are judgement. Class is read off the catalog and the remotes. A trait (security) says what the code does, which no remote knows, and the operator owns that list.
Who the reader is decides what is required. Contributor-facing documents are required by no class — solo-maintained, and the published stance says so. Consumer-facing ones (SECURITY.md, LICENSE, non-goals, limitations) are untouched by that. This is not a rule against having the others.
No CI badge, because there is no CI. The forge has zero Actions runners registered (measured). The substitute is one command from a clean clone, said plainly. A static badge asserting a run is the anti-pattern this gate flags — and an early draft of this README carried one.
Three outcomes on references. "No match" and "match on a known non-repo" must stay distinct findings. Collapsing them hides real loss inside correct text — the exact defect class this gate exists to catch.
One API call, anonymous. The org listing carries description and topics already; per-repo fetching trips the rate limiter (HTTP 429). It reads without a token, so the gate works for any reader — a public plugin whose documented check only runs for its author is a broken plugin.
Codepoints, not bytes, not UTF-16 units. Use [...s].length. An em-dash exposes only the byte layer; astral characters expose the rest.
The reader decides a link's level, not just what is required. Root documents are the shop window — a dead link there is an ERROR. Below the root it is a WARN: that is where session plans, agent working files and path-traversal fixtures with deliberately invalid targets live. Measured, 30 of 43 findings were down there and all were ERRORs.
A repo's name is its remote, not its directory. catalog/ holds ktg-plugin-marketplace. The basename left it unregistered with zero checks run, against the one repo every catalog rule depends on.
No hook until the rule is precise. A blocking gate that fails a correct repository is the mechanism that gets gates switched off.

Commands

npm test                                                  # 103 tests
node scripts/repo-standard-check.mjs --dir "$PWD"         # gate one repo
node scripts/repo-standard-check.mjs --offline            # no network call
node scripts/repo-standard-check.mjs --json               # machine output
node scripts/repo-standard-check.mjs --refresh            # register vs. forge

Release

Polyrepo rule: a version bump is not finished until the tag vX.Y.Z is pushed and the catalog ref is bumped to it. Use release-plugin.mjs, never a hand-edited ref.

check-versions.mjs reads the catalog README's per-plugin label, and a missing entry is a silent null rather than an error — release-plugin.mjs rewrites an existing heading but cannot create one. A new plugin's catalog README entry has to be added by hand once.