Kjell Tore Guttormsen 720850a9ad feat(gate): buckets, traits, and the checks the brief calls load-bearing

Measured this build against a documentation brief for public repos. The
five original checks covered roughly one of its ten sections, so this
adds what a single repo can answer on its own.

New: required README headings per class (Non-goals is the cheapest
trust-builder there is), in-repo version consistency across manifest /
badge / CHANGELOG / tag, badge honesty, boilerplate, licence-claim,
and relative links. Findings now carry a BUCKET beside the level -
broken / missing / weakening - and output is grouped by it, because
that is the order the work gets done in.

Traits are a second axis beside class: class is structural and readable
off the catalog, a trait says what the code does. `security` attaches
SECURITY.md and a Known limitations section. The two names carrying it
are proposed, not measured - that list is the operator's.

Solo-maintained settles a category: CONTRIBUTING, CODE_OF_CONDUCT and
MAINTAINERS are required by no class. Consumer-facing documents are
untouched by that; SECURITY.md exists for the stranger who finds a hole.

Three bugs found by running against llm-security, not by reading:
- ~30 link findings, all noise. Regexes inside code spans are
  `[...](...)` to a naive scanner. Strip code first.
- `file:` and other schemes were treated as repo-relative paths.
- Relative links were resolved against the repo root instead of the
  file they sit in, calling two files missing that sat next to the
  README linking them.
Same fix applied to the boilerplate check: a document ABOUT placeholder
detection was tripping the placeholder detector.

Also removed this repo's own static tests badge. There is no CI - the
forge has zero Actions runners registered - so it could never become
real, and it is the exact anti-pattern the gate now flags.

67 tests. Against llm-security every remaining finding is real and
matches the census's independent hand-measurement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WYJ3FHLtVgzFXMZ6UF598h

2026-07-27 16:06:33 +02:00

4.4 KiB

Raw Blame History

repo-standard

Per-repo gate for the open/ presentation standard, packaged as a marketplace plugin.

Context

Two components, one boundary:

Engine (scripts/repo-standard-check.mjs) — pure classifiers with all I/O resolved into their input, mirroring the marketplace's check-versions.mjs. Findings are ERROR/WARN/SKIP/OK; exit 1 on ERROR. Pinned by scripts/repo-standard-check.test.mjs (npm test).
Skill (skills/repo-standard/) — the judgement the script cannot encode. No checking logic lives here; it calls the engine.

register/repos.json is the single taxonomy register (name → class, per-class requirements, and the known non-repo names). Central by design: per-repo copies would recreate, in data, exactly the drift this plugin exists to remove.

Invariants

This repo has a PUBLIC remote. STATE.md is LOCAL-ONLY and gitignored from the first commit. Repos on a private remote track theirs — do not carry that habit across in either direction.
The gate sees ONE repo. Anything needing a view across the whole org (topic coverage, competing install forms, catalog-vs-forge divergence) does not belong here. It is measured where the org is enumerated.
It records, it does not fix. Findings first, remediation afterwards. Patching while measuring is how the inconsistency it detects was produced.
SKIP is never a pass. A check that could not run says so and names why.
When a check fires many times in one repo, suspect the CHECK. The first link pass produced ~30 findings against llm-security and all were noise — regexes in code spans, file: URLs, relative paths resolved against the wrong directory. Strip code before scanning text; resolve links against the file they sit in. A gate that is wrong this often gets switched off.
Two axes on every finding. Level (ERROR/WARN/SKIP/OK) and bucket (broken/missing/weakening). They are independent — a weakening finding can be an ERROR.
Class is structural, traits are judgement. Class is read off the catalog and the remotes. A trait (security) says what the code does, which no remote knows, and the operator owns that list.
Who the reader is decides what is required. Contributor-facing documents are required by no class — solo-maintained, and the published stance says so. Consumer-facing ones (SECURITY.md, LICENSE, non-goals, limitations) are untouched by that. This is not a rule against having the others.
No CI badge, because there is no CI. The forge has zero Actions runners registered (measured). The substitute is one command from a clean clone, said plainly. A static badge asserting a run is the anti-pattern this gate flags — and an early draft of this README carried one.
Three outcomes on references. "No match" and "match on a known non-repo" must stay distinct findings. Collapsing them hides real loss inside correct text — the exact defect class this gate exists to catch.
One API call, anonymous. The org listing carries description and topics already; per-repo fetching trips the rate limiter (HTTP 429). It reads without a token, so the gate works for any reader — a public plugin whose documented check only runs for its author is a broken plugin.
Codepoints, not bytes, not UTF-16 units. Use [...s].length. An em-dash exposes only the byte layer; astral characters expose the rest.
No hook until the rule is precise. A blocking gate that fails a correct repository is the mechanism that gets gates switched off.

Commands

npm test                                                  # 67 tests
node scripts/repo-standard-check.mjs --dir "$PWD"         # gate one repo
node scripts/repo-standard-check.mjs --offline            # no network call
node scripts/repo-standard-check.mjs --json               # machine output
node scripts/repo-standard-check.mjs --refresh            # register vs. forge

Release

Polyrepo rule: a version bump is not finished until the tag vX.Y.Z is pushed and the catalog ref is bumped to it. Use release-plugin.mjs, never a hand-edited ref.

check-versions.mjs reads the catalog README's per-plugin label, and a missing entry is a silent null rather than an error — release-plugin.mjs rewrites an existing heading but cannot create one. A new plugin's catalog README entry has to be added by hand once.