Kjell Tore Guttormsen
a563035feb
A release is not a ref, so `git ls-remote` cannot answer this one. It is therefore the third API call per invocation, and the only new one the acquisition model adds. Both sides come from the forge, never the clone: comparing a local tag to a published release would report portfolio-optimiser as stale when the real defect is a tag that was never pushed — REMOTE-SYNC's subject, not this one. Measured across all 22 registered repos before the rule was locked: 4 have no tags, 2 tag without ever publishing a release, 11 are current, 5 lag. Those 2 are why zero releases is an OK and not a finding — nothing in a repo says which of the two legitimate conventions it follows, the same measurement that rejected VERSION-DRIFT. Lagging is a WARN because the remedy is safe, and because this repo is the org's worst offender: an ERROR would have let the gate settle an operator question by exiting 1 on its own author. The shell that produced that baseline sorted tags lexically and put v0.9.0 above v0.10.0 — the exact defect compareTags exists to fix, reproduced in the tool meant to validate it. Recorded as an invariant: derive "newest" with the engine's own comparator or the measurement is fiction. CLAUDE.md's API-call count moves from two to three in this same commit; that line has now gone stale twice. The unmarked "429 at ~40" figure is marked unusable — its endpoint class was never recorded, and org-ops measured ~110 raw reads at 0.4s with zero 429 the same day. 205 tests (was 196). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WWc4piM4QW6Jxfky2Rw4Z8
1721 lines
78 KiB
JavaScript
1721 lines
78 KiB
JavaScript
#!/usr/bin/env node
|
|
// repo-standard — the per-repo gate.
|
|
//
|
|
// Checks ONE repository against the standard for its class:
|
|
// - README first screen: H1 is the repo name, next line IS the forge description
|
|
// - Install block complete, in the form its class actually uses
|
|
// - Required files present for its class
|
|
// - Every `open/<name>` reference in URL position resolves
|
|
// - Description within the length bound, measured in codepoints
|
|
//
|
|
// What it deliberately does NOT do: anything that needs to see all repos at once.
|
|
// Divergence across the org (0/18 topics, three competing install forms, README
|
|
// release notes duplicating a CHANGELOG that 16 of 18 repos have) is invisible
|
|
// from inside one repo. Those checks live in org-ops, not here.
|
|
//
|
|
// Structure mirrors the marketplace's check-versions.mjs on purpose: pure
|
|
// classifiers with all I/O resolved into their input, findings tagged
|
|
// ERROR/WARN/SKIP/OK, exit 1 on ERROR. This is a gate, not a checklist —
|
|
// the catalog's eleven descriptions are good because a gate runs on them; the
|
|
// forge's nine were empty. Same care, different outcome.
|
|
//
|
|
// Usage:
|
|
// node scripts/repo-standard-check.mjs [--dir <path>] [--name <repo>] [--offline] [--json]
|
|
// node scripts/repo-standard-check.mjs --refresh # register vs. live org listing
|
|
import { readFileSync, existsSync } from 'node:fs';
|
|
import { execFileSync } from 'node:child_process';
|
|
import { join, dirname, basename } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
const REGISTER_PATH = join(HERE, '..', 'register', 'repos.json');
|
|
const PACKAGE_PATH = join(HERE, '..', 'package.json');
|
|
|
|
// This engine's own version, not the target repo's — distinct from
|
|
// readPackageVersion(dir) below, which reads the REPO BEING CHECKED.
|
|
export function readEngineVersion(path = PACKAGE_PATH) {
|
|
return JSON.parse(readFileSync(path, 'utf8')).version;
|
|
}
|
|
|
|
// The version names a FILE; only the sha names the CODE. A sweep once stamped
|
|
// 18 raw files `0.4.0` while four of them carried findings from a check that
|
|
// only exists in 0.5.0 — the feature and the version bump are two commits, so
|
|
// the worktree held new code under an old number for a window. Derived from
|
|
// this checkout, no network call; null outside a git checkout (a vendored copy
|
|
// or a tarball has no HEAD, and that is not a crash).
|
|
export function readEngineCommit(dir = join(HERE, '..')) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'rev-parse', 'HEAD'], {
|
|
encoding: 'utf8',
|
|
stdio: ['ignore', 'pipe', 'ignore'],
|
|
}).trim() || null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
// The JUDGEMENT lattice, and `SKIP` is deliberately not in it. A skip is not a
|
|
// severity — it is the absence of a verdict, so it cannot be the worst of a set
|
|
// that contains real ones. It used to sit between OK and WARN here, which meant
|
|
// a repo with 0 ERROR, 0 WARN and a dozen OK headlined as "skipped": five repos
|
|
// in org-ops census 05, `okr` among them with the most OK in the org. Coverage
|
|
// is carried on its own axis instead — see `notCheckedOf`.
|
|
const LEVELS = ['OK', 'WARN', 'ERROR'];
|
|
|
|
// Findings carry a level AND a bucket, and the two are independent axes.
|
|
// The level says how sure and how loud; the bucket says what KIND of problem it
|
|
// is, which is what a reader triages on:
|
|
//
|
|
// broken works wrongly right now — a stranger is blocked or misled
|
|
// missing an expected artefact is simply absent
|
|
// weakening present and functional, but it reads as amateur
|
|
//
|
|
// A weakening finding can still be an ERROR: a README opening line that
|
|
// contradicts the published description blocks nobody, and is still wrong.
|
|
export const BUCKETS = ['broken', 'missing', 'weakening'];
|
|
|
|
// ------------------------------------------------------------ pure helpers
|
|
|
|
// Codepoints. Not bytes (an em-dash costs 3) and not UTF-16 units (`👉` costs 2).
|
|
// The em-dash exposes only the outer layer, which is why "characters, not bytes"
|
|
// was not enough on its own.
|
|
export function countCodepoints(s) {
|
|
return [...String(s ?? '')].length;
|
|
}
|
|
|
|
// ~20 "dead" repo names collapsed to 3 real ones once this ran. A clone URL
|
|
// ending in .git is a legitimate reference, not a broken one.
|
|
export function normalizeRepoRef(raw) {
|
|
return String(raw ?? '')
|
|
.replace(/\/+$/, '')
|
|
.replace(/\.git$/, '');
|
|
}
|
|
|
|
// Only names in URL position are resolvable references. That single rule
|
|
// excludes all three of the measured "correct text that looks broken" cases at
|
|
// once: a path position (`~/.claude/coord/_broadcast/`), running prose (`coord`
|
|
// is still the transport protocol's name), and a bare directory name.
|
|
//
|
|
// The host segment excludes `/`: `open` must be the FIRST path segment after
|
|
// the host, matching how every real repo URL is shaped
|
|
// (`https://host/open/<name>`, `user@host:open/<name>.git`). Without that
|
|
// restriction, an API endpoint like `/api/v1/orgs/open/repos` also matches —
|
|
// `open` there is the org argument to the API, and `repos` is the literal
|
|
// resource segment, not a repo name (measured: catalog RUNBOOK.md:39, :114).
|
|
//
|
|
// The third alternative is the SCHEMELESS host: `git.fromaitochitta.com/open/x`
|
|
// written without `https://`, which a subtree instruction routinely is
|
|
// (measured false negative: llm-security/V3-UPGRADE.md:343 — the scheme was
|
|
// doing work it was never entitled to, and its absence hid a WARN). What makes
|
|
// a name resolvable is its position after a HOST, not the scheme in front of
|
|
// it. Two guards keep that from becoming the noise the scheme was masking: the
|
|
// host must end in a TLD-shaped label, and the lookbehind refuses a host
|
|
// preceded by `/`, `.` or a word character — because that is a PATH segment
|
|
// that merely contains a dot (`docs/v1.2/open/`, `test/nav.golden/open/`), not
|
|
// a host. The API-endpoint rule survives untouched: `orgs` carries no dot.
|
|
const URL_REF = /(?::\/\/[^\s)\]"'`/]+\/|@[^\s:]+:|(?<![A-Za-z0-9._/-])[A-Za-z0-9-]+(?:\.[A-Za-z0-9-]+)*\.[A-Za-z]{2,}\/)open\/([A-Za-z0-9._-]+)/g;
|
|
|
|
export function extractOpenRefs(text) {
|
|
const out = [];
|
|
const lines = String(text ?? '').split('\n');
|
|
lines.forEach((line, i) => {
|
|
for (const m of line.matchAll(URL_REF)) {
|
|
out.push({ name: normalizeRepoRef(m[1]), line: i + 1, raw: m[0] });
|
|
}
|
|
});
|
|
return out;
|
|
}
|
|
|
|
// Three outcomes, never two. "No match" and "match on something that is not a
|
|
// repo" must stay distinguishable — if they share an outcome, the loss goes
|
|
// silent, and silent loss is the defect class this standard exists to catch.
|
|
export function classifyRef(name, register) {
|
|
if (Object.prototype.hasOwnProperty.call(register.repos ?? {}, name)) return 'repo';
|
|
if (Object.prototype.hasOwnProperty.call(register.non_repos ?? {}, name)) return 'non-repo';
|
|
return 'unknown';
|
|
}
|
|
|
|
// The same "three outcomes, never two" rule this file applies to classifyRef,
|
|
// applied to the check's own result. Emitting nothing on success made "no dead
|
|
// references" and "the check never ran" identical in the output, so a sweep
|
|
// across the org could not tell 19 clean repos from 19 unread ones (measured:
|
|
// org-ops census 03b). Silence is not a pass here either.
|
|
export function checkLinks({ files }, register) {
|
|
const entries = Object.entries(files ?? {});
|
|
if (entries.length === 0) {
|
|
return [{
|
|
level: 'SKIP',
|
|
skip: 'notRun',
|
|
code: 'LINKS-OPEN-REFS-UNAVAILABLE',
|
|
msg: 'no files were enumerated — the `open/` reference check did not run',
|
|
}];
|
|
}
|
|
const findings = [];
|
|
// One name, one line, one reference. `[host/open/x](https://host/open/x)`
|
|
// puts the same reference on both halves of a markdown link and matched
|
|
// twice once the schemaless host became legible (measured:
|
|
// llm-security/V3-ANNOUNCEMENT.md:124). The key keeps name AND line, so two
|
|
// different names on one line — or the same name on two lines — stay two.
|
|
const seen = new Set();
|
|
let checked = 0;
|
|
for (const [path, text] of entries) {
|
|
for (const ref of extractOpenRefs(text)) {
|
|
const key = `${path}\n${ref.line}\n${ref.name}`;
|
|
if (seen.has(key)) continue;
|
|
seen.add(key);
|
|
checked += 1;
|
|
const kind = classifyRef(ref.name, register);
|
|
if (kind === 'repo') continue;
|
|
if (kind === 'non-repo') {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'LINK-NON-REPO',
|
|
bucket: 'weakening',
|
|
msg: `${path}:${ref.line} — \`open/${ref.name}\` resolves to a known non-repo: ${register.non_repos[ref.name]}`,
|
|
});
|
|
} else {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'LINK-DEAD',
|
|
bucket: 'broken',
|
|
msg: `${path}:${ref.line} — \`open/${ref.name}\` matches no repo in the register (dead reference)`,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
// The count IS the evidence. An OK that cannot say how many references it
|
|
// resolved is the same silence wearing a different level.
|
|
if (findings.length === 0) {
|
|
findings.push({
|
|
level: 'OK',
|
|
code: 'LINKS-OPEN-REFS',
|
|
msg: checked === 0
|
|
? `no \`open/\` references found in ${entries.length} scanned file(s)`
|
|
: `${checked} \`open/\` reference(s) checked — every one resolves to a registered repo`,
|
|
});
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
export function checkDescription(description, register) {
|
|
if (description === null || description === undefined) {
|
|
return [{ level: 'SKIP', skip: 'notRun', code: 'DESC-UNAVAILABLE', msg: 'forge description not available — check not run (offline, or the listing failed)' }];
|
|
}
|
|
const max = register.description_max_codepoints ?? 180;
|
|
const n = countCodepoints(description);
|
|
if (n === 0) return [{ level: 'ERROR', code: 'DESC-EMPTY', bucket: 'missing', msg: 'forge description is empty' }];
|
|
if (n > max) {
|
|
return [{ level: 'ERROR', code: 'DESC-TOO-LONG', bucket: 'weakening', msg: `forge description is ${n} codepoints, bound is ${max}` }];
|
|
}
|
|
return [{ level: 'OK', code: 'DESC', msg: `description ${n}/${max} codepoints` }];
|
|
}
|
|
|
|
// The opening line makes description == catalog == README: the same thread on a
|
|
// third surface, and the only one of the three a machine can check from inside
|
|
// the repo.
|
|
export function checkFirstScreen({ readme, name, description, klass }, register) {
|
|
const findings = [];
|
|
const lines = String(readme ?? '').split('\n');
|
|
const firstIdx = lines.findIndex((l) => l.trim() !== '');
|
|
|
|
const heading = firstIdx === -1 ? null : lines[firstIdx].trim();
|
|
|
|
// No heading at all is broken. A heading that merely differs from the repo
|
|
// name is not: the thread that has to hold is description == catalog ==
|
|
// opening line, and the H1 is none of those three. A human title like
|
|
// `# OKR for Public Sector` is a naming choice the operator owns, so it is
|
|
// surfaced and left to them — a gate that fails a correct repo is the
|
|
// mechanism that gets gates switched off.
|
|
if (heading === null || !heading.startsWith('# ')) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'README-H1',
|
|
bucket: 'missing',
|
|
msg: `README must open with an H1 (expected \`# ${name}\`, found: ${heading === null ? '<empty file>' : `\`${heading}\``})`,
|
|
});
|
|
return findings;
|
|
}
|
|
|
|
// A registered title is where that operator call gets WRITTEN DOWN. Without
|
|
// one, the same WARN reappears every census and "we decided this is correct"
|
|
// is indistinguishable from "nobody has looked". With one, the two separate —
|
|
// and a repo whose title nobody has ruled on is left standing alone, which is
|
|
// the wanted side effect, not a cost.
|
|
const title = register?.titles?.[name];
|
|
if (heading === `# ${name}`) {
|
|
findings.push({ level: 'OK', code: 'README-H1', msg: `H1 is \`# ${name}\`` });
|
|
} else if (title && heading === `# ${title}`) {
|
|
findings.push({ level: 'OK', code: 'README-H1', msg: `H1 is \`# ${title}\` — the registered title for \`${name}\`` });
|
|
} else if (title) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'README-H1',
|
|
bucket: 'weakening',
|
|
msg: `H1 is \`${heading}\`, which is neither \`# ${name}\` nor the registered title \`${title}\` — one of the two has drifted.`,
|
|
});
|
|
} else {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'README-H1',
|
|
bucket: 'weakening',
|
|
msg: `H1 is \`${heading}\`, not \`# ${name}\` — deliberate title, or drift? Operator's call. Record the decision as \`titles.${name}\` in the register.`,
|
|
});
|
|
}
|
|
|
|
// For an ordinary repo the README opening and the forge description describe
|
|
// the SAME subject, and equality is the right demand. `org-profile` is the one
|
|
// class where they do not: its README is the organisation's landing page and
|
|
// the forge text describes the repo. Both are correct about their own subject,
|
|
// so it is the EQUALITY that does not apply — and a landing page's opening
|
|
// link could only ever match by putting raw markdown on a plain-text surface.
|
|
// Class-level data, not a hardcoded name: class rules live in the register.
|
|
if (register?.classes?.[klass]?.readme_desc_match === false) {
|
|
findings.push({
|
|
level: 'OK',
|
|
code: 'README-DESC',
|
|
msg: `class \`${klass}\` is exempt: the README describes the org, the forge description describes the repo — different subjects, so equality is not required`,
|
|
});
|
|
return findings;
|
|
}
|
|
|
|
if (description === null || description === undefined) {
|
|
findings.push({ level: 'SKIP', skip: 'notRun', code: 'README-DESC', msg: 'forge description not available — opening-line match not checked' });
|
|
return findings;
|
|
}
|
|
|
|
const restIdx = lines.findIndex((l, i) => i > firstIdx && l.trim() !== '');
|
|
const opening = restIdx === -1 ? '' : lines[restIdx].trim();
|
|
if (opening !== String(description).trim()) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'README-DESC',
|
|
bucket: 'weakening',
|
|
msg: `README opening line does not match the forge description\n README: ${opening}\n forge: ${description}`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'OK', code: 'README-DESC', msg: 'opening line matches the forge description' });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// `claude plugin install x@mkt` or `/plugin install x@mkt` — the two CLI forms.
|
|
function hasCliInstall(readme, name, mkt) {
|
|
const esc = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
return new RegExp(`(?:claude\\s+plugin|/plugin)\\s+install\\s+${esc(name)}@${esc(mkt)}\\b`).test(readme);
|
|
}
|
|
|
|
function hasAnyPluginInstall(readme) {
|
|
return /(?:claude\s+plugin|\/plugin)\s+install\s+\S+@\S+/.test(readme);
|
|
}
|
|
|
|
export function checkInstallBlock({ readme, name, klass }, register) {
|
|
const form = register.classes?.[klass]?.install ?? 'none';
|
|
const text = String(readme ?? '');
|
|
const mkt = register.marketplace ?? {};
|
|
const findings = [];
|
|
|
|
if (form === 'none') return findings;
|
|
|
|
const addLines = text.split('\n').filter((l) => /plugin\s+marketplace\s+add/.test(l));
|
|
const hasAdd = addLines.length > 0;
|
|
|
|
// The forge UI's clone button hands out the ssh URL, and `marketplace add`
|
|
// answers it with "Invalid git URL" — a message that never mentions the
|
|
// protocol. Measured end-to-end 2026-07-25.
|
|
if (addLines.some((l) => /ssh:\/\//.test(l))) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-SSH',
|
|
bucket: 'broken',
|
|
msg: '`marketplace add` is shown with an ssh:// URL — it rejects those ("Invalid git URL"). Use the https form.',
|
|
});
|
|
}
|
|
|
|
// A well-formed command pointing at the wrong marketplace is still a command
|
|
// that does not work. Checked against the register, so it needs no network.
|
|
if (hasAdd && mkt.url) {
|
|
const urls = addLines
|
|
.map((l) => /marketplace\s+add\s+(\S+)/.exec(l)?.[1])
|
|
.filter(Boolean)
|
|
.map((u) => u.replace(/[`'"]+$/, ''));
|
|
if (urls.length && !urls.some((u) => normalizeRepoRef(u) === normalizeRepoRef(mkt.url))) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-URL-MISMATCH',
|
|
bucket: 'broken',
|
|
msg: `\`marketplace add\` points at ${urls[0]}, but this marketplace is ${mkt.url}`,
|
|
});
|
|
}
|
|
}
|
|
|
|
if (form === 'plugin' || form === 'catalog') {
|
|
if (!hasAdd) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-NO-MARKETPLACE',
|
|
bucket: 'broken',
|
|
msg: `no \`plugin marketplace add\` line — the reader is never told to add \`${mkt.name}\` (${mkt.url})`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'OK', code: 'INSTALL-MARKETPLACE', msg: '`marketplace add` present' });
|
|
}
|
|
}
|
|
|
|
if (form === 'plugin') {
|
|
// The corrected defect A. `enabledPlugins` in settings.json is a LEGITIMATE
|
|
// second form and it stands in 10 of 11 plugin READMEs — what is missing in
|
|
// 7 of them is a CLI command. So the contract requires the command and
|
|
// permits the JSON alongside it; it never accepts the JSON as a substitute.
|
|
// A reader who scrolls to the JSON block has a complete path; an agent told
|
|
// "install this" reaches for the CLI and finds `marketplace add` and nothing else.
|
|
if (!hasCliInstall(text, name, mkt.name)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-NO-CLI',
|
|
bucket: 'broken',
|
|
msg: `no CLI install command for this repo — expected \`claude plugin install ${name}@${mkt.name}\` (or the \`/plugin install\` form). An \`enabledPlugins\` block is a welcome addition, but it is not a CLI command.`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'OK', code: 'INSTALL-CLI', msg: `CLI install command names ${name}@${mkt.name}` });
|
|
}
|
|
}
|
|
|
|
if (form === 'vendor' && hasAnyPluginInstall(text)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-WRONG-FORM',
|
|
bucket: 'broken',
|
|
msg: 'shared asset shows a plugin install line — it is vendored into consumers, not installed. Document how to vendor it.',
|
|
});
|
|
}
|
|
|
|
if (form === 'package') {
|
|
if (hasAnyPluginInstall(text)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-WRONG-FORM',
|
|
bucket: 'broken',
|
|
msg: 'standalone project shows a plugin install line — use the pip/uv form.',
|
|
});
|
|
} else if (!/\b(pip\s+install|uv\s+(?:pip\s+)?(?:add|sync|install|run)|uvx)\b/.test(text)) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'INSTALL-NO-PACKAGE-FORM',
|
|
bucket: 'missing',
|
|
msg: 'no pip/uv install form found — expected for a standalone project',
|
|
});
|
|
}
|
|
}
|
|
|
|
return findings;
|
|
}
|
|
|
|
// Per class, never flat. A flat standard demands a CONTRIBUTING from a CSS
|
|
// library that takes no contributions and a ROADMAP from a five-line profile.
|
|
// Syntax is not truth. The most disqualifying failure a repo can have is an
|
|
// install command that does not work for a stranger, and a perfectly formed
|
|
// `claude plugin install x@mkt` fails silently if `x` was never pinned in the
|
|
// catalog. This is the one check that answers the brief's first question.
|
|
export function checkInstallTruth({ name, klass, catalogNames }) {
|
|
if (klass !== 'plugin') return [{ level: 'OK', code: 'INSTALL-TRUTH', msg: 'not a marketplace plugin — nothing to resolve' }];
|
|
if (!catalogNames) {
|
|
return [{ level: 'SKIP', skip: 'notRun', code: 'INSTALL-TRUTH', msg: 'catalog not reachable — cannot verify the install command actually resolves' }];
|
|
}
|
|
if (!catalogNames.includes(name)) {
|
|
return [{
|
|
level: 'ERROR',
|
|
code: 'INSTALL-NOT-IN-CATALOG',
|
|
bucket: 'broken',
|
|
msg: `\`${name}\` is not pinned in the marketplace catalog — the documented install command cannot succeed for anyone`,
|
|
}];
|
|
}
|
|
return [{ level: 'OK', code: 'INSTALL-TRUTH', msg: 'the install command resolves against the catalog' }];
|
|
}
|
|
|
|
// A pin is the one command a stranger actually runs. Reported by org-ops
|
|
// (census 08) and re-measured here against the FORGE: 3 pins in the org, 1
|
|
// dead — `llm-ingestion-pipeline-security` pins ITSELF to `@v0.7.0`, and that
|
|
// tag does not exist (newest is v0.6.1). ERROR, not WARN, because a dead
|
|
// documentation link costs a stranger a 404 while a dead pin costs them the
|
|
// install.
|
|
//
|
|
// Deliberately NOT reusing `LINK-DEAD`'s check, only the idea of enumerating:
|
|
// LINK-DEAD asks "does the repo exist", this asks "does the reference exist".
|
|
// It is also not `VERSION-TAG`, which reads the MANIFEST — the two coincide on
|
|
// guard today only because the same wrong number was written in both places.
|
|
//
|
|
// Resolved against the forge, never the clone: a local tag can exist without
|
|
// having been pushed, which is exactly what portfolio-optimiser demonstrates.
|
|
const escapeRe = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
|
|
export function extractInstallPins(readme, register) {
|
|
const forge = String(register?.forge ?? '').replace(/\/+$/, '');
|
|
const org = register?.org;
|
|
if (!forge || !org) return [];
|
|
const re = new RegExp(`(?:git\\+)?${escapeRe(forge)}/${escapeRe(org)}/([A-Za-z0-9._-]+)\\.git@([^\\s"'\`)\\]#]+)`, 'g');
|
|
const seen = new Set();
|
|
const pins = [];
|
|
for (const m of String(readme ?? '').matchAll(re)) {
|
|
const key = `${m[1]}@${m[2]}`;
|
|
if (seen.has(key)) continue;
|
|
seen.add(key);
|
|
pins.push({ repo: m[1], ref: m[2] });
|
|
}
|
|
return pins;
|
|
}
|
|
|
|
// A dotted numeric component is what makes a ref answerable by `ls-remote
|
|
// --tags`. `main` and a bare sha are neither dead nor alive to this check.
|
|
const TAG_SHAPED = /^v?\d+\.\d+/;
|
|
|
|
export function checkInstallPins({ readme, forgeTagsByRepo }, register) {
|
|
const pins = extractInstallPins(readme, register);
|
|
if (pins.length === 0) {
|
|
// The VERSION-NONE shape: the check ran, read the whole README, and found
|
|
// no subject. Nothing here can be wrong, which is a verdict.
|
|
return [{ level: 'OK', code: 'PINS-NONE', msg: 'README pins no install reference — no ref exists here that could be dead' }];
|
|
}
|
|
|
|
const findings = [];
|
|
let resolved = 0;
|
|
for (const { repo, ref } of pins) {
|
|
if (!TAG_SHAPED.test(ref)) {
|
|
// A branch or a sha is a different weakness — an unpinned install — and
|
|
// this check can never turn it into a verdict, so nobody has an action.
|
|
findings.push({
|
|
level: 'SKIP',
|
|
skip: 'byDesign',
|
|
code: 'PIN-NOT-A-TAG',
|
|
msg: `install pin \`${repo}@${ref}\` is not a version tag — \`ls-remote --tags\` cannot resolve it, and a branch pin is a looseness this check does not judge`,
|
|
});
|
|
continue;
|
|
}
|
|
const tags = forgeTagsByRepo?.[repo];
|
|
if (!tags) {
|
|
findings.push({
|
|
level: 'SKIP',
|
|
skip: 'notRun',
|
|
code: 'PIN-UNAVAILABLE',
|
|
msg: `could not read tags for \`${repo}\` from the forge — the pin \`@${ref}\` was not verified (offline, or the ref listing failed)`,
|
|
});
|
|
continue;
|
|
}
|
|
if (!tags.includes(ref)) {
|
|
const newest = [...tags].filter((t) => TAG_SHAPED.test(t)).sort(compareTags).slice(-1)[0];
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'PIN-DEAD',
|
|
bucket: 'broken',
|
|
msg: `README install pin \`${repo}@${ref}\` does not exist on the forge — the one command a stranger runs fails outright${newest ? ` (newest tag is \`${newest}\`)` : ''}`,
|
|
});
|
|
} else {
|
|
resolved += 1;
|
|
}
|
|
}
|
|
// Counts what was actually verified, never what was merely present. The
|
|
// first version said "N install pin(s) resolve" whenever no ERROR fired,
|
|
// which meant an offline run asserted a pass for a pin nothing had read.
|
|
if (resolved > 0) {
|
|
findings.push({
|
|
level: 'OK',
|
|
code: 'PINS',
|
|
msg: `${resolved} of ${pins.length} install pin(s) resolve against the forge`,
|
|
});
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// Requirements come from two axes. The CLASS is structural — it can be read off
|
|
// the catalog and the remotes. A TRAIT is about what the code does, which no
|
|
// remote can tell you: `security` attaches the obligations a tool acquires by
|
|
// handling untrusted input.
|
|
//
|
|
// Note what is NOT here: CONTRIBUTING, CODE_OF_CONDUCT, MAINTAINERS. The
|
|
// maintainer works alone and the published stance says so. Contributor-facing
|
|
// documentation for a project that accepts no contributors is theatre, and a
|
|
// code of conduct with an unattended placeholder address is worse than none.
|
|
// Consumer-facing documents are untouched by that — SECURITY.md exists for the
|
|
// outsider who finds a hole, and being solo does not remove them.
|
|
function requirementsFor(klass, traits, register) {
|
|
const cls = register.classes?.[klass] ?? {};
|
|
const files = [...(cls.required_files ?? [])];
|
|
const headings = [...(cls.required_headings ?? [])];
|
|
for (const t of traits ?? []) {
|
|
const tr = register.trait_requirements?.[t];
|
|
if (!tr) continue;
|
|
for (const f of tr.required_files ?? []) if (!files.includes(f)) files.push(f);
|
|
for (const h of tr.required_headings ?? []) if (!headings.includes(h)) headings.push(h);
|
|
}
|
|
return { files, headings };
|
|
}
|
|
|
|
export function checkRequiredFiles({ present, klass, traits }, register) {
|
|
const { files: required } = requirementsFor(klass, traits, register);
|
|
const have = new Set(present ?? []);
|
|
const findings = [];
|
|
for (const f of required) {
|
|
if (!have.has(f)) {
|
|
findings.push({ level: 'ERROR', code: 'FILE-MISSING', bucket: 'missing', msg: `missing required file for class \`${klass}\`: ${f}` });
|
|
}
|
|
}
|
|
if (findings.length === 0 && required.length > 0) {
|
|
findings.push({ level: 'OK', code: 'FILES', msg: `all ${required.length} required files present` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// Fixed headings, because experienced readers skip rather than read. `## Install`
|
|
// on a predictable heading is what agents pattern-match on, and `## Non-goals`
|
|
// is the cheapest trust-builder there is: it proves someone thought about the
|
|
// boundary, and it stops misuse before it starts.
|
|
export function checkHeadings({ readme, klass, traits }, register) {
|
|
const { headings: required } = requirementsFor(klass, traits, register);
|
|
const text = String(readme ?? '');
|
|
const present = new Set(
|
|
text.split('\n').map((l) => l.trim()).filter((l) => l.startsWith('#')),
|
|
);
|
|
const findings = [];
|
|
for (const h of required) {
|
|
if ([...present].some((p) => p.toLowerCase() === h.toLowerCase())) continue;
|
|
|
|
// Same title, wrong depth: say that, rather than "missing". The contract
|
|
// wants a predictable top-level heading because that is what an agent
|
|
// pattern-matches on — but the section does exist, and the fix is a
|
|
// different edit than writing one from scratch.
|
|
const title = h.replace(/^#+\s*/, '');
|
|
const atOtherLevel = [...present].find(
|
|
(p) => p.replace(/^#+\s*/, '').toLowerCase() === title.toLowerCase(),
|
|
);
|
|
if (atOtherLevel) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'HEADING-LEVEL',
|
|
bucket: 'weakening',
|
|
msg: `README has \`${atOtherLevel}\` but the contract wants \`${h}\` — a predictable top-level heading is what readers and agents scan for`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'ERROR', code: 'HEADING-MISSING', bucket: 'missing', msg: `README has no \`${h}\` section` });
|
|
}
|
|
}
|
|
if (findings.length === 0 && required.length > 0) {
|
|
findings.push({ level: 'OK', code: 'HEADINGS', msg: `all ${required.length} required headings present` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// One version, four places it can be written down. This is the check that
|
|
// removes a whole defect class — "README says v0.3.1, the tag does not exist" —
|
|
// and the one that would have caught this repo's own 32→34 test-count drift.
|
|
export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogTop, tags }) {
|
|
const findings = [];
|
|
const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null;
|
|
if (!v) {
|
|
// Not a SKIP. This check ran, saw all four places a version can be written
|
|
// down, and found no version claimed in any of them — no SUBJECT to judge,
|
|
// the shape checkReadmeLanguage answers with OK. It sat at `SKIP`/`notRun`
|
|
// until 0.9.0, deferred once on the ground that re-levelling would move a
|
|
// repo's status. That was measured false: an added OK cannot worsen the
|
|
// worst *judged* finding, and all three repos that reach this line already
|
|
// read OK. Nor can OK bless a real gap — no class requires a version file,
|
|
// and a `plugin` missing its manifest is an independent FILE-MISSING ERROR.
|
|
return [{ level: 'OK', code: 'VERSION-NONE', msg: 'no version claimed anywhere — nothing to check' }];
|
|
}
|
|
|
|
if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) {
|
|
findings.push({ level: 'ERROR', code: 'VERSION-BADGE', bucket: 'weakening', msg: `README version badge is ${readmeBadge}, manifest says ${v}` });
|
|
}
|
|
if (changelogTop !== null && changelogTop !== undefined && changelogTop !== v) {
|
|
findings.push({ level: 'ERROR', code: 'VERSION-CHANGELOG', bucket: 'weakening', msg: `newest CHANGELOG entry is ${changelogTop}, manifest says ${v}` });
|
|
}
|
|
|
|
// Nothing released yet is a state, not a defect — and it must say so rather
|
|
// than pass quietly, because "SKIP is never a pass" is the whole discipline.
|
|
if (!tags || tags.length === 0) {
|
|
findings.push({ level: 'SKIP', skip: 'notRun', code: 'VERSION-TAG', msg: `repo has no tags — cannot verify that v${v} was ever released` });
|
|
} else if (!tags.includes(`v${v}`)) {
|
|
findings.push({ level: 'ERROR', code: 'VERSION-TAG', bucket: 'broken', msg: `no tag \`v${v}\` — the documented version was never released (tags: ${tags.slice(-3).join(', ')})` });
|
|
}
|
|
|
|
if (findings.every((f) => f.level === 'OK' || f.level === 'SKIP')) {
|
|
findings.push({ level: 'OK', code: 'VERSION', msg: `version ${v} agrees across manifest, README and CHANGELOG` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// Version order, not the order git handed the tags over. `git tag --list` sorts
|
|
// lexically, where v10.0.0 lands BEFORE v9.0.0 — so reading "newest" off an
|
|
// unsorted list picks the wrong tag on precisely the repos with the longest
|
|
// release history (repo-mailbox has 27). Numeric triple first; a pre-release
|
|
// suffix sorts BELOW the bare release, as semver has it, which is what keeps
|
|
// `v0.5.0a2` from outranking `v0.5.0`.
|
|
function compareTags(a, b) {
|
|
const parts = (s) => {
|
|
const m = /^v?(\d+)\.(\d+)\.(\d+)(.*)$/.exec(String(s));
|
|
return m ? [Number(m[1]), Number(m[2]), Number(m[3]), m[4]] : [0, 0, 0, String(s)];
|
|
};
|
|
const [aM, aN, aP, aRest] = parts(a);
|
|
const [bM, bN, bP, bRest] = parts(b);
|
|
if (aM !== bM) return aM - bM;
|
|
if (aN !== bN) return aN - bN;
|
|
if (aP !== bP) return aP - bP;
|
|
if (aRest === bRest) return 0;
|
|
if (aRest === '') return 1;
|
|
if (bRest === '') return -1;
|
|
return aRest < bRest ? -1 : 1;
|
|
}
|
|
|
|
// A lightweight tag is a branch-like ref: it can be moved to another commit
|
|
// with nothing recorded that it ever pointed elsewhere. The catalog pins every
|
|
// plugin to `ref: vX.Y.Z`, so a movable tag is a movable pin — this is a supply
|
|
// chain property, not a tidiness one.
|
|
//
|
|
// The two levels come from a measurement, not from taste. Across all 19 clones:
|
|
// 155 tags, 14 of them lightweight, but only ONE repo whose NEWEST tag is
|
|
// lightweight. The newest is what a consumer resolves today and what an
|
|
// operator can re-cut at no cost, so it is an ERROR. The older ones can only be
|
|
// "fixed" by force-moving an already published ref — the exact act this check
|
|
// exists to warn about — so they are exposed once, as a count, and never as
|
|
// fourteen separate findings. A gate that demands an unsafe remedy is a gate
|
|
// that gets switched off.
|
|
//
|
|
// Read entirely from local git objects: `git for-each-ref` reports the object
|
|
// type with no network call, so this costs nothing against the two-call budget.
|
|
//
|
|
// `tags_lightweight_accepted` is where a decided YES about tag HISTORY lives.
|
|
// Without it the WARN below can never be cleared — the only remedy is force-
|
|
// moving a published ref, the act the check exists to warn about — so the gate
|
|
// would report the same thing forever and make "we decided this" and "nobody
|
|
// looked" the same output. That is the `titles` defect, one axis over.
|
|
// Keyed on tag NAME, never a count: a count stays satisfied the moment one tag
|
|
// is re-cut and a different, unaccepted one takes its place.
|
|
// Acceptance reaches history ONLY. The newest tag is the one lightweight tag
|
|
// with a safe remedy (`git tag -a -f`), so it cannot be accepted away.
|
|
export function checkTagIntegrity({ tagObjects, name }, register) {
|
|
const tags = [...(tagObjects ?? [])].sort((a, b) => compareTags(a.name, b.name));
|
|
if (tags.length === 0) {
|
|
// The VERSION-NONE shape: the check ran, saw every tag there is, and found
|
|
// no subject. A repo with no tags has no ref that could be moved — there is
|
|
// nothing here to be wrong, which is a verdict, not an absent one.
|
|
return [{ level: 'OK', code: 'TAGS-NONE', msg: 'repo has no version tags — no tag exists that could be moved' }];
|
|
}
|
|
|
|
const findings = [];
|
|
const newest = tags[tags.length - 1];
|
|
if (!newest.annotated) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'TAG-ANNOTATED',
|
|
bucket: 'broken',
|
|
msg: `newest tag \`${newest.name}\` is lightweight — it can be moved to another commit with no record that it ever pointed elsewhere, and the catalog pins releases by tag. Re-cut it annotated: \`git tag -a -f ${newest.name} ${newest.name}^{}\`.`,
|
|
});
|
|
}
|
|
const accepted = new Set(register?.tags_lightweight_accepted?.[name] ?? []);
|
|
const olderLightweight = tags.slice(0, -1).filter((t) => !t.annotated);
|
|
const older = olderLightweight.filter((t) => !accepted.has(t.name));
|
|
const excused = olderLightweight.filter((t) => accepted.has(t.name));
|
|
if (older.length > 0) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'TAG-ANNOTATED-HISTORY',
|
|
bucket: 'weakening',
|
|
msg: `${older.length} older lightweight tag(s) (${older.slice(0, 3).map((t) => t.name).join(', ')}${older.length > 3 ? ', …' : ''}) — each is movable without a trace. WARN, not ERROR: the only remedy is force-moving an already published ref, which is the risk itself. Cut every NEW tag annotated (\`git tag -a\`).`,
|
|
});
|
|
}
|
|
// An exemption is a finding, not a deletion — the same rule `readme_desc_match`
|
|
// follows. An exception nobody can see reads exactly like a check that
|
|
// silently stopped running.
|
|
if (excused.length > 0) {
|
|
findings.push({
|
|
level: 'OK',
|
|
code: 'TAG-ANNOTATED-ACCEPTED',
|
|
msg: `${excused.length} older lightweight tag(s) (${excused.map((t) => t.name).join(', ')}) are recorded in the register as accepted history — force-moving a published ref is the only remedy, so the operator accepted them rather than rewrite them. Any NEW lightweight tag, and the newest tag, are still judged.`,
|
|
});
|
|
}
|
|
if (findings.length === 0) {
|
|
findings.push({ level: 'OK', code: 'TAGS', msg: `all ${tags.length} version tag(s) are annotated — none can be moved without a record` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// A static image asserting "tests: 642 passing" is a claim dressed as evidence.
|
|
// Version, licence and platform badges assert no run, so they are fine static.
|
|
// Bare `status` used to be in this list and caught a self-declared maturity
|
|
// badge ("status: alpha") as if it were a run claim — reported by
|
|
// llm-ingestion-pipeline-security. `build`/`ci`/`passing` already catch the
|
|
// run-asserting compounds ("build status", "CI status"), so dropping the bare
|
|
// word loses no real detection.
|
|
// Matched WORD by word, never as a substring. `selftest_checks-402` — a count
|
|
// of checks that exist, asserting nothing about a run — fired for three
|
|
// censuses because `tests?` matched the letters inside "selfTESTs" (org-ops,
|
|
// 2026-08-12, on repo-mailbox's dispute). Splitting on every non-alphanumeric
|
|
// run, rather than leaning on `\b`, is what keeps the fix from creating the
|
|
// opposite defect: shields.io writes a space as `_`, so `\btests\b` would have
|
|
// gone quiet on the genuine claim `tests-402_passing`.
|
|
const CLAIM_WORD = /^(tests?|build|ci|coverage|passing)$/i;
|
|
const claimsARun = (s) => s.split(/[^a-z0-9]+/i).some((w) => CLAIM_WORD.test(w));
|
|
|
|
// The forge's release page is where a stranger lands when they want a version
|
|
// they can name, and it is the one surface refs cannot answer — a release is
|
|
// not a ref, so `git ls-remote` has nothing to report. That is why this is the
|
|
// THIRD API call and the only new one the acquisition model adds.
|
|
//
|
|
// Both sides come from the FORGE, never from the clone. Comparing a local tag
|
|
// against a published release would report portfolio-optimiser as having a
|
|
// stale release when the real finding is a tag that was never pushed (v1.0.0
|
|
// local, v0.1.0 published) — a different defect, owned by a different check.
|
|
//
|
|
// The levels come from a measurement across all 22 registered repos
|
|
// (2026-08-12): 4 have no tags at all, 2 tag without ever publishing a release,
|
|
// 11 are current, 5 lag their newest tag. Those 2 are why "no releases" is an
|
|
// OK and not a finding: nothing in a repo says which of the two legitimate
|
|
// conventions it follows, and a gate that fails a correct repository is the
|
|
// mechanism that gets gates switched off — the same measurement that rejected
|
|
// VERSION-DRIFT one check over. Lagging is a WARN rather than an ERROR because
|
|
// the remedy is safe: publishing a release for a tag that already exists moves
|
|
// no published ref, unlike the remedy TAG-ANNOTATED has to withhold.
|
|
export function checkReleaseCurrent({ forgeTagsSelf, releases }) {
|
|
if (releases === null || releases === undefined) {
|
|
return [{ level: 'SKIP', skip: 'notRun', code: 'RELEASE-UNAVAILABLE', msg: 'forge releases not available — check not run (offline, or the listing failed)' }];
|
|
}
|
|
// One side missing is not agreement. Refs and releases are acquired over two
|
|
// different channels, so either can fail alone.
|
|
if (forgeTagsSelf === null || forgeTagsSelf === undefined) {
|
|
return [{ level: 'SKIP', skip: 'notRun', code: 'RELEASE-UNAVAILABLE', msg: 'forge refs not readable — cannot tell whether the newest release is the newest tag' }];
|
|
}
|
|
|
|
const tags = [...forgeTagsSelf].sort(compareTags);
|
|
if (tags.length === 0) {
|
|
// The VERSION-NONE shape: the check ran, saw every tag the forge has, and
|
|
// found no subject. Nothing could have been released, so there is nothing
|
|
// here to be wrong — a verdict, not an absent one.
|
|
return [{ level: 'OK', code: 'RELEASE-NONE', msg: 'no tags on the forge — no release could exist' }];
|
|
}
|
|
const newestTag = tags[tags.length - 1];
|
|
|
|
if (releases.length === 0) {
|
|
return [{
|
|
level: 'OK',
|
|
code: 'RELEASE-TAGS-ONLY',
|
|
msg: `${tags.length} tag(s) and no release published — this repo tags without publishing releases, which is a convention this gate does not judge`,
|
|
}];
|
|
}
|
|
|
|
// Version order, not the order the API handed them over: the releases listing
|
|
// sorts by creation time, and a patch cut after a minor would read as newest.
|
|
const newestRelease = [...releases].sort(compareTags).pop();
|
|
if (compareTags(newestRelease, newestTag) < 0) {
|
|
return [{
|
|
level: 'WARN',
|
|
code: 'RELEASE-STALE',
|
|
bucket: 'weakening',
|
|
msg: `newest release is \`${newestRelease}\` but the newest tag is \`${newestTag}\` — the release page shows a version older than the code. Publish a release for \`${newestTag}\`.`,
|
|
}];
|
|
}
|
|
return [{
|
|
level: 'OK',
|
|
code: 'RELEASE-CURRENT',
|
|
msg: newestRelease === newestTag
|
|
? `newest release \`${newestRelease}\` is the newest tag`
|
|
: `newest release \`${newestRelease}\` is ahead of every tag the forge lists`,
|
|
}];
|
|
}
|
|
|
|
// Counting badges needs a NARROWER rule than detecting a dishonest one. The
|
|
// claim check reads any image, any host, on purpose. Here the opposite error
|
|
// matters: counting a screenshot or an architecture diagram as clutter would
|
|
// punish exactly the visual work this standard wants more of.
|
|
const BADGE_URL = /shields\.io|badgen\.net|\/badges?[/.]/i;
|
|
|
|
// Trockman et al., ICSE 2018 (doi 10.1145/3180155.3180209, n=294,941 npm
|
|
// packages): badge count relates to popularity non-linearly with a predicted
|
|
// inflection at five, and surveyed maintainers called over-badged READMEs
|
|
// cluttered and "trying too hard". WARN, never ERROR — the coefficient sits in
|
|
// an appendix with no CI or p-value, so it carries "more is not better" and
|
|
// cannot carry a hard limit.
|
|
const BADGE_INFLECTION = 5;
|
|
|
|
export function checkBadges({ readme, present }) {
|
|
const have = new Set(present ?? []);
|
|
const findings = [];
|
|
let badgeCount = 0;
|
|
for (const line of String(readme ?? '').split('\n')) {
|
|
// Any image, any host. Restricting this to img.shields.io would have missed
|
|
// a self-hosted SVG asserting exactly the same unverified thing.
|
|
// The trailing `(?:\]\(target\))?` is the LINK the badge is wrapped in —
|
|
// previously unmatched, so being linked at all silently ended scrutiny
|
|
// whether or not the link actually went anywhere.
|
|
for (const m of line.matchAll(/(\[)?!\[([^\]]*)\]\(([^)\s]+)\)(?:\]\(([^)\s]+)\))?/g)) {
|
|
const linkTarget = m[1] === '[' ? m[4] : undefined;
|
|
const linked = linkTarget !== undefined;
|
|
const label = `${m[2]} ${m[3]}`;
|
|
if (BADGE_URL.test(m[3])) badgeCount++;
|
|
if (!claimsARun(label)) continue;
|
|
if (!linked) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'BADGE-STATIC-CLAIM',
|
|
bucket: 'weakening',
|
|
msg: `static badge asserts a run that nothing verifies: \`${m[2]}\`. A badge like this is a claim dressed as evidence — link it to a real run, or drop it.`,
|
|
});
|
|
continue;
|
|
}
|
|
// A linked badge is only as honest as its target. An external target
|
|
// (the ordinary case — a CI provider's own page) needs the network to
|
|
// verify and is deliberately out of scope, same as checkInternalLinks.
|
|
// A relative target this gate CAN check without the network — and a
|
|
// relative target that resolves nowhere is worse than a static badge:
|
|
// it LOOKS verified.
|
|
if (/^[a-z][a-z0-9+.-]*:/i.test(linkTarget)) continue;
|
|
const resolved = resolveRelative('README.md', linkTarget.split('#')[0]);
|
|
if (resolved !== null && !have.has(resolved)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'BADGE-DEAD-LINK',
|
|
bucket: 'broken',
|
|
msg: `${m[2]} badge links to \`${linkTarget}\`, which does not resolve — a linked badge pointing nowhere is a claim dressed as evidence, worse than a static one because it looks verified.`,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
if (badgeCount > BADGE_INFLECTION) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'BADGE-COUNT',
|
|
bucket: 'weakening',
|
|
msg: `${badgeCount} badges — past the measured inflection of ${BADGE_INFLECTION}, where a badge row starts reading as clutter rather than as evidence (Trockman et al., ICSE 2018). Keep the ones a reader acts on.`,
|
|
});
|
|
}
|
|
if (findings.length === 0) findings.push({ level: 'OK', code: 'BADGES', msg: 'no static badge asserts an unverified run' });
|
|
return findings;
|
|
}
|
|
|
|
// Which language a README is written in is not a property of the code, so no
|
|
// remote can report it — it is a property of the READER, and the operator owns
|
|
// it. English is the default; a repo aimed only at a Norwegian readership is
|
|
// declared `nb` in the register and is then wrong in English, not right.
|
|
//
|
|
// Detection is a stopword-frequency comparison rather than a dependency: both
|
|
// word sets below are chosen to have NO member that is also a common word in
|
|
// the other language, which is why `at` and `for` (Norwegian and English both)
|
|
// are deliberately absent from each.
|
|
const STOPWORDS = {
|
|
nb: ['og', 'ikke', 'som', 'det', 'den', 'er', 'på', 'til', 'av', 'med', 'om',
|
|
'har', 'kan', 'skal', 'blir', 'være', 'etter', 'når', 'også', 'hvis',
|
|
'eller', 'men', 'fra', 'ved', 'mot', 'uten', 'hver', 'alle', 'andre',
|
|
'seg', 'dette', 'disse', 'mellom', 'gjennom', 'siden', 'fordi', 'derfor'],
|
|
en: ['the', 'and', 'of', 'to', 'in', 'is', 'that', 'with', 'this', 'are',
|
|
'be', 'from', 'by', 'as', 'an', 'or', 'not', 'you', 'your', 'we', 'our',
|
|
'it', 'on', 'which', 'when', 'what', 'how', 'its', 'they', 'their', 'has',
|
|
'can', 'will', 'should', 'each', 'between', 'because', 'therefore'],
|
|
};
|
|
|
|
// Below this there is not enough running prose for a frequency count to mean
|
|
// anything, and below the ratio the document is genuinely mixed. Both say so
|
|
// rather than guessing — a wrong verdict on language is worse than no verdict.
|
|
const LANG_MIN_HITS = 10;
|
|
const LANG_MIN_RATIO = 1.5;
|
|
|
|
function countStopwords(text, words) {
|
|
const lower = text.toLowerCase();
|
|
let n = 0;
|
|
for (const w of words) {
|
|
const m = lower.match(new RegExp(`(^|[^\\p{L}])${w}([^\\p{L}]|$)`, 'gu'));
|
|
if (m) n += m.length;
|
|
}
|
|
return n;
|
|
}
|
|
|
|
export function checkReadmeLanguage({ readme, name }, register) {
|
|
const declared = register?.locales?.[name] ?? 'en';
|
|
const other = declared === 'nb' ? 'en' : 'nb';
|
|
// Same discipline as the link and boilerplate checks: a Norwegian flag name
|
|
// in a shell example must not decide what language the DOCUMENT is in.
|
|
const prose = stripCode(String(readme ?? ''));
|
|
const hits = { nb: countStopwords(prose, STOPWORDS.nb), en: countStopwords(prose, STOPWORDS.en) };
|
|
|
|
// Not a SKIP. SKIP is for a check that could not RUN — the catalog was
|
|
// unreachable, the file unreadable. This one ran, saw everything, and found
|
|
// no prose to be in the wrong language, the same shape as "no licence claim
|
|
// to back". A thin README is a real problem, and it is checkFirstScreen's;
|
|
// routing it here would stop any terse repo from ever reaching OK.
|
|
if (hits[declared] + hits[other] < LANG_MIN_HITS) {
|
|
return [{
|
|
level: 'OK',
|
|
code: 'LANGUAGE',
|
|
msg: `no running prose to judge (${hits[declared] + hits[other]} marker words) — nothing claims a language`,
|
|
}];
|
|
}
|
|
if (hits[other] >= hits[declared] * LANG_MIN_RATIO) {
|
|
return [{
|
|
level: 'WARN',
|
|
code: 'README-LANGUAGE',
|
|
bucket: 'weakening',
|
|
msg: `README reads as \`${other}\` but this repo is declared \`${declared}\` (${hits[other]} vs ${hits[declared]} marker words). Who the reader is decides the language — fix the prose, or fix \`locales\` in the register.`,
|
|
}];
|
|
}
|
|
if (hits[declared] < hits[other] * LANG_MIN_RATIO) {
|
|
return [{
|
|
level: 'SKIP',
|
|
skip: 'notRun',
|
|
code: 'README-LANGUAGE-UNDECIDABLE',
|
|
msg: `README mixes languages too evenly to call (${hits.nb} nb vs ${hits.en} en) — declared \`${declared}\`, unverified`,
|
|
}];
|
|
}
|
|
return [{ level: 'OK', code: 'LANGUAGE', msg: `README reads as \`${declared}\`, as declared` }];
|
|
}
|
|
|
|
// Template text that was never filled in. A visible unfinished template costs
|
|
// more trust than the missing document would have.
|
|
const FIXME_RE = /FIXME/;
|
|
const BOILERPLATE = [
|
|
/your-project-name/i,
|
|
/\byour-org\b/i,
|
|
/\[INSERT[^\]]*\]/i,
|
|
/<your[- ][a-z]+>/i,
|
|
/TODO:\s*(fill|replace|update)/i,
|
|
/example@example\.(com|org)/i,
|
|
FIXME_RE,
|
|
];
|
|
|
|
// "TODO/FIXME" named together names the convention, not a live instance of
|
|
// one — reported by config-audit: a scanner whose job is finding these
|
|
// markers names its own detection target in its own docs, unquoted. A lone
|
|
// FIXME is still caught; only the paired reference is exempt.
|
|
const NAMES_THE_CONVENTION = /\bTODO\s*\/\s*FIXME\b|\bFIXME\s*\/\s*TODO\b/i;
|
|
|
|
export function checkBoilerplate({ files }) {
|
|
const findings = [];
|
|
for (const [path, text] of Object.entries(files ?? {})) {
|
|
// Same discipline as the link check: code spans and fenced blocks are where
|
|
// a document ABOUT placeholders keeps its examples.
|
|
stripCode(text).split('\n').forEach((line, i) => {
|
|
const namesTheConvention = NAMES_THE_CONVENTION.test(line);
|
|
for (const re of BOILERPLATE) {
|
|
if (re === FIXME_RE && namesTheConvention) continue;
|
|
if (re.test(line)) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'BOILERPLATE',
|
|
bucket: 'weakening',
|
|
msg: `${path}:${i + 1} — unfilled template text: \`${line.trim().slice(0, 70)}\``,
|
|
});
|
|
return;
|
|
}
|
|
}
|
|
});
|
|
}
|
|
if (findings.length === 0) findings.push({ level: 'OK', code: 'BOILERPLATE', msg: 'no unfilled template text found' });
|
|
return findings;
|
|
}
|
|
|
|
// "LICENSE mentioned in the README, no file in the repo" is its own anti-signal:
|
|
// the claim is load-bearing for anyone deciding whether they may use this.
|
|
export function checkLicenseClaim({ readme, present }) {
|
|
const text = String(readme ?? '');
|
|
const claims = /\bLICEN[SC]E\b/i.test(text) || /\b(MIT|Apache|BSD|GPL)\b.{0,20}licen[sc]e/i.test(text);
|
|
const have = (present ?? []).some((f) => /^LICEN[SC]E(\.\w+)?$/i.test(f));
|
|
if (claims && !have) {
|
|
return [{
|
|
level: 'ERROR',
|
|
code: 'LICENSE-CLAIMED-ABSENT',
|
|
bucket: 'broken',
|
|
msg: 'README cites a licence but the repo has no LICENSE file — the claim a reader relies on to use this is unbacked',
|
|
}];
|
|
}
|
|
return [{ level: 'OK', code: 'LICENSE-CLAIM', msg: have ? 'LICENSE present' : 'no licence claim to back' }];
|
|
}
|
|
|
|
// Blank out fenced blocks and inline code spans, keeping line numbers intact.
|
|
// Documentation about regexes is full of strings that ARE markdown links to a
|
|
// naive scanner: `["']([A-Za-z0-9\-._]{16,64})["']` is `[...](...)` exactly.
|
|
// Running the first version against a real repo produced ~30 findings and every
|
|
// one of them was noise.
|
|
export function stripCode(text) {
|
|
let fenced = false;
|
|
let prevBlank = true;
|
|
let inIndented = false;
|
|
return String(text ?? '')
|
|
.split('\n')
|
|
.map((line) => {
|
|
if (/^\s*(```|~~~)/.test(line)) {
|
|
fenced = !fenced;
|
|
return '';
|
|
}
|
|
if (fenced) return '';
|
|
|
|
const blank = line.trim() === '';
|
|
const indented = /^(\s{4,}|\t)\S/.test(line);
|
|
// An indented line OPENS a code block only after a blank line — otherwise
|
|
// a nested list item would count, which made links inside nested bullets
|
|
// invisible. But once open, the block CONTINUES while lines stay indented;
|
|
// requiring a blank line before every line let everything after line 1
|
|
// leak back into scanning.
|
|
if (indented && (prevBlank || inIndented)) {
|
|
inIndented = true;
|
|
prevBlank = false;
|
|
return '';
|
|
}
|
|
if (!blank && !indented) inIndented = false;
|
|
prevBlank = blank;
|
|
if (inIndented && blank) return '';
|
|
return line.replace(/`[^`]*`/g, '');
|
|
})
|
|
.join('\n');
|
|
}
|
|
|
|
// A relative link resolves against the file it sits in, not against the repo
|
|
// root. Getting this wrong called two files missing that were right there next
|
|
// to the README linking them — and it would have done so in every nested doc.
|
|
// Returns null when the path escapes the repo, which is unresolvable from
|
|
// inside one repo rather than broken.
|
|
export function resolveRelative(fromFile, target) {
|
|
if (target.startsWith('/')) return null;
|
|
const baseParts = String(fromFile).split('/').slice(0, -1);
|
|
const out = [...baseParts];
|
|
for (const part of target.split('/')) {
|
|
if (part === '' || part === '.') continue;
|
|
if (part === '..') {
|
|
if (out.length === 0) return null;
|
|
out.pop();
|
|
} else {
|
|
out.push(part);
|
|
}
|
|
}
|
|
return out.join('/');
|
|
}
|
|
|
|
// Who the reader is decides the level. A dead link in a root document — README,
|
|
// CHANGELOG, SECURITY — is in the shop window and blocks a stranger. The same
|
|
// link three directories down is in a session plan, an agent working file, or a
|
|
// test fixture whose target is invalid ON PURPOSE. Measured across the org: 30
|
|
// of 43 findings sat below the root, and every one of them was an ERROR. A gate
|
|
// that is wrong that often gets switched off, so the level moves — and only the
|
|
// level. The finding is still reported, with its file and line.
|
|
function linkLevelFor(path) {
|
|
return String(path).includes('/') ? 'WARN' : 'ERROR';
|
|
}
|
|
|
|
// A file living in a test/fixture path is presumed to break its own links on
|
|
// purpose — `nav-golden-escape/bundle/index.md` escapes with `../../../../etc/passwd`
|
|
// deliberately, and the deep `..` pops the whole base path rather than resolving
|
|
// to `null`, so it read as a genuine WARN. Third tool in the org to hit this
|
|
// exact pattern, which is the signal that the check was at fault, not the repos.
|
|
// Only `*golden*` is a substring glob; the other three are exact segment names,
|
|
// so `testing/` or `fixturesque/` — real directories — are not swept in.
|
|
function isFixturePath(path) {
|
|
return String(path)
|
|
.toLowerCase()
|
|
.split('/')
|
|
.some((seg) => seg === 'test' || seg === 'tests' || seg === 'fixtures' || seg.includes('golden'));
|
|
}
|
|
|
|
// A home directory is what makes a `file:` URL a leak rather than a scheme the
|
|
// gate declines to resolve. Anchored on the two roots a real machine path
|
|
// starts with; a bare `file:///abs/path.html` placeholder is not one.
|
|
const FILE_URL_LEAK = /^file:\/\/\/?(Users|home)\//i;
|
|
|
|
// Relative file links only. Anchor resolution depends on per-renderer heading
|
|
// slug rules and is a rabbit hole; external URLs need the network. Both are
|
|
// deliberately out — a check that is sometimes wrong teaches people to ignore it.
|
|
export function checkInternalLinks({ files, present }) {
|
|
const have = new Set(present ?? []);
|
|
// `present` holds tracked FILES only, so a link to a directory — `[x](dir/)`
|
|
// — never has a member to match even when every file under it is tracked.
|
|
// Derive the directories a tracked file actually lives in from the same set.
|
|
const haveDirs = new Set();
|
|
for (const p of have) {
|
|
const parts = String(p).split('/');
|
|
for (let i = 1; i < parts.length; i++) haveDirs.add(parts.slice(0, i).join('/'));
|
|
}
|
|
const findings = [];
|
|
for (const [path, text] of Object.entries(files ?? {})) {
|
|
stripCode(text).split('\n').forEach((line, i) => {
|
|
for (const m of line.matchAll(/\[[^\]]*\]\(([^)\s]+)\)/g)) {
|
|
const target = m[1];
|
|
// A `file:` URL naming a real home directory is the one scheme that is
|
|
// NOT somebody else's to resolve: it is dead for every reader but its
|
|
// author, and it publishes that author's directory layout.
|
|
//
|
|
// The scheme alone is not the rule. Measured over the corpus, 40 such
|
|
// links split 18/22 between a documented convention example — the same
|
|
// two lines copy-pasted into nine CLAUDE.md files — and real machine
|
|
// paths in three repos. `...` is not a path segment, so a target
|
|
// containing `/.../` cannot resolve on ANY machine and is by
|
|
// construction an illustration, not a leak.
|
|
if (FILE_URL_LEAK.test(target) && !/\/\.\.\.\//.test(target)) {
|
|
findings.push({
|
|
level: linkLevelFor(path),
|
|
code: 'LINK-FILE-URL',
|
|
bucket: 'broken',
|
|
msg: `${path}:${i + 1} — \`${target}\` is a link into a local filesystem: dead for every reader but its author, and it publishes the author's directory layout. Link the repository-relative path, or the published URL.`,
|
|
});
|
|
continue;
|
|
}
|
|
// Any other scheme, not just http — `vscode:`, `ftp:` are all somebody
|
|
// else's to resolve.
|
|
if (/^[a-z][a-z0-9+.-]*:/i.test(target) || /^[#<]/.test(target)) continue;
|
|
const clean = target.split('#')[0];
|
|
if (!clean) continue;
|
|
|
|
const resolved = resolveRelative(path, clean);
|
|
// A path that leaves the repo cannot be judged from inside it — a
|
|
// plugin README pointing up at its marketplace is the ordinary case.
|
|
if (resolved === null) {
|
|
findings.push({
|
|
level: 'SKIP',
|
|
skip: 'byDesign',
|
|
code: 'LINK-OUTSIDE-REPO',
|
|
msg: `${path}:${i + 1} — \`${clean}\` points outside this repo; the gate sees one repo and cannot resolve it`,
|
|
});
|
|
continue;
|
|
}
|
|
if (!have.has(resolved) && !haveDirs.has(resolved)) {
|
|
if (isFixturePath(path)) {
|
|
findings.push({
|
|
level: 'SKIP',
|
|
skip: 'byDesign',
|
|
code: 'LINK-INTERNAL-FIXTURE',
|
|
msg: `${path}:${i + 1} — link points at \`${clean}\` (${resolved}), which is not a tracked file; ${path} is a test/fixture path, so this is presumed intentional and not judged`,
|
|
});
|
|
} else {
|
|
findings.push({
|
|
level: linkLevelFor(path),
|
|
code: 'LINK-INTERNAL-MISSING',
|
|
bucket: 'broken',
|
|
msg: `${path}:${i + 1} — link points at \`${clean}\` (${resolved}), which is not a tracked file`,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
});
|
|
}
|
|
// The OK line asserts that every link resolved. Keying it on ERROR alone would
|
|
// have printed it beside a pile of WARN findings saying the opposite.
|
|
if (!findings.some((f) => f.code === 'LINK-INTERNAL-MISSING')) {
|
|
findings.push({ level: 'OK', code: 'LINKS-INTERNAL', msg: 'every resolvable relative link resolves' });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// Worst of the judged findings; `SKIP` only when there is nothing to be worst
|
|
// OF. That second half is what keeps "`SKIP` is never a pass" true: an
|
|
// unregistered repo, or an empty finding set, still says so plainly. What the
|
|
// rule no longer does is let one un-runnable check speak for twelve that ran.
|
|
export function levelOf(findings) {
|
|
let worst = null;
|
|
for (const f of findings ?? []) {
|
|
if (f.level === 'SKIP') continue;
|
|
if (worst === null || LEVELS.indexOf(f.level) > LEVELS.indexOf(worst)) worst = f.level;
|
|
}
|
|
return worst ?? 'SKIP';
|
|
}
|
|
|
|
// The coverage axis, counted rather than left for each consumer to re-derive
|
|
// from `findings`. Same reason `buckets` is precomputed beside `status`: a
|
|
// number nobody can see reads exactly like a check that silently stopped
|
|
// running.
|
|
export function notCheckedOf(findings) {
|
|
return (findings ?? []).filter((f) => f.level === 'SKIP').length;
|
|
}
|
|
|
|
// The coverage axis is really two facts, and merging them made a repo look
|
|
// unread when nothing was: `portfolio-optimiser — OK · 11 not checked`, all
|
|
// eleven of them links the gate declines to judge on purpose. org-ops named
|
|
// the split (20260809T124015Z, observation 2) without a name for it.
|
|
//
|
|
// byDesign the check saw the thing and declined — it can never become a
|
|
// verdict and nobody has an action. Out-of-repo links, fixture
|
|
// paths.
|
|
// notRun a re-run or an operator action turns it into a verdict. An
|
|
// unreachable catalog, an unregistered repo, a repo with no tags.
|
|
//
|
|
// The kind is read off the finding, never off its code: `VERSION-TAG` is
|
|
// emitted at SKIP with no tags and at ERROR with the wrong one, so a
|
|
// code→kind map would have to re-derive a reason the emission site already
|
|
// had. Untagged falls to `notRun` — the loud side, because a skip of unknown
|
|
// kind must not inherit "deliberate, nothing to see".
|
|
export function groupSkips(findings) {
|
|
const out = { byDesign: [], notRun: [] };
|
|
for (const f of findings ?? []) {
|
|
if (f.level !== 'SKIP') continue;
|
|
out[f.skip === 'byDesign' ? 'byDesign' : 'notRun'].push(f);
|
|
}
|
|
return out;
|
|
}
|
|
|
|
export function skipsOf(findings) {
|
|
const g = groupSkips(findings);
|
|
return { byDesign: g.byDesign.length, notRun: g.notRun.length };
|
|
}
|
|
|
|
export function bucketsOf(findings) {
|
|
const out = { broken: 0, missing: 0, weakening: 0 };
|
|
for (const f of findings ?? []) {
|
|
if (f.bucket && out[f.bucket] !== undefined) out[f.bucket] += 1;
|
|
}
|
|
return out;
|
|
}
|
|
|
|
export function classifyRepo(
|
|
{ name, files, present, description, pluginVersion, readmeBadge, changelogTop, tags, tagObjects, catalogNames, forgeTagsByRepo, forgeTagsSelf, releases },
|
|
register,
|
|
) {
|
|
const klass = register.repos?.[name];
|
|
if (!klass) {
|
|
return {
|
|
name,
|
|
klass: null,
|
|
traits: [],
|
|
status: 'SKIP',
|
|
notChecked: 1,
|
|
skips: { byDesign: 0, notRun: 1 },
|
|
buckets: { broken: 0, missing: 0, weakening: 0 },
|
|
findings: [{
|
|
level: 'SKIP',
|
|
skip: 'notRun',
|
|
code: 'REPO-UNREGISTERED',
|
|
msg: `\`${name}\` is not in the register — class unknown, so no class-specific rule can be applied. Add it to register/repos.json (or run --refresh).`,
|
|
}],
|
|
};
|
|
}
|
|
|
|
const traits = register.traits?.[name] ?? [];
|
|
const readme = (files ?? {})['README.md'] ?? '';
|
|
const findings = [
|
|
...checkFirstScreen({ readme, name, description, klass }, register),
|
|
...checkInstallBlock({ readme, name, klass }, register),
|
|
...checkInstallTruth({ name, klass, catalogNames }),
|
|
...checkInstallPins({ readme, forgeTagsByRepo }, register),
|
|
...checkHeadings({ readme, klass, traits }, register),
|
|
...checkRequiredFiles({ present, klass, traits }, register),
|
|
...checkLinks({ files }, register),
|
|
...checkInternalLinks({ files, present }),
|
|
...checkLicenseClaim({ readme, present }),
|
|
...checkBadges({ readme, present }),
|
|
...checkReadmeLanguage({ readme, name }, register),
|
|
...checkBoilerplate({ files }),
|
|
...checkVersionConsistency({ pluginVersion, readmeBadge, changelogTop, tags }),
|
|
...checkTagIntegrity({ tagObjects, name }, register),
|
|
...checkReleaseCurrent({ forgeTagsSelf, releases }),
|
|
...checkDescription(description, register),
|
|
];
|
|
|
|
return {
|
|
name,
|
|
klass,
|
|
traits,
|
|
status: levelOf(findings),
|
|
// Still a NUMBER, and still the total. A consumer doing `notChecked > 0`
|
|
// against an object gets a silent false — the same class of quiet wrong
|
|
// answer this whole axis exists to remove.
|
|
notChecked: notCheckedOf(findings),
|
|
skips: skipsOf(findings),
|
|
buckets: bucketsOf(findings),
|
|
findings,
|
|
};
|
|
}
|
|
|
|
// ---------------------------------------------------------------- I/O shell
|
|
|
|
export function loadRegister(path = REGISTER_PATH) {
|
|
return JSON.parse(readFileSync(path, 'utf8'));
|
|
}
|
|
|
|
// TWO calls per invocation (corrected 2026-08-04 — this used to say ONE, from
|
|
// before fetchCatalogNames existed; a 13-repo shell loop trusting that count
|
|
// looked safe at 13 requests and was actually 26). Both anonymous — verified
|
|
// — so this works for any reader, not only for someone holding a token. A
|
|
// sweep across every repo does NOT belong here: it needs the org listing
|
|
// exactly once, not once per invocation, and "see all repos at once" is
|
|
// org-ops's job by this file's own header. What DOES belong here is not
|
|
// silently giving up on a transient 429 — that turns a rate-limit blip into
|
|
// a false SKIP, which this repo's own rule says is never a pass.
|
|
const defaultSleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
|
|
// Measured 2026-08-04 against the live forge: nginx never sends a
|
|
// `Retry-After` header on its 429s, so the exponential fallback below is the
|
|
// ONLY path that ever actually runs — the branch above it is dead in
|
|
// practice, kept only because a future proxy config could add the header.
|
|
// The 429 itself is a leaky-bucket burst limit, not a fixed-duration ban: a
|
|
// 20-25 request burst took up to ~15s to fully drain, and a 20s pause always
|
|
// cleared it. `retries: 3` (7s worst case) was tuned for a hard ban that
|
|
// turned out not to exist; `retries: 5` with `maxDelayMs: 8000` (23s worst
|
|
// case) covers the measured drain time without one attempt blocking minutes.
|
|
export async function fetchWithRetry(
|
|
url,
|
|
options,
|
|
{ fetchImpl = fetch, retries = 5, baseDelayMs = 1000, maxDelayMs = 8000, sleep = defaultSleep } = {},
|
|
) {
|
|
for (let attempt = 0; ; attempt += 1) {
|
|
const res = await fetchImpl(url, options);
|
|
if (res.status !== 429 || attempt >= retries) return res;
|
|
const retryAfter = Number(res.headers?.get?.('retry-after'));
|
|
const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
|
|
? retryAfter * 1000
|
|
: Math.min(baseDelayMs * 2 ** attempt, maxDelayMs);
|
|
await sleep(delayMs);
|
|
}
|
|
}
|
|
|
|
// The catalog's plugin list, read straight from the forge. One more call, and
|
|
// it is what turns "the install line is well-formed" into "the install line
|
|
// works". Null on any failure, which reads as SKIP rather than a pass.
|
|
async function fetchCatalogNames(register) {
|
|
const mkt = register.marketplace ?? {};
|
|
if (!mkt.name) return null;
|
|
const url = `${register.forge}/api/v1/repos/${register.org}/${mkt.name}/raw/.claude-plugin/marketplace.json`;
|
|
try {
|
|
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
|
|
if (!res.ok) return null;
|
|
const json = JSON.parse(await res.text());
|
|
return (json.plugins ?? []).map((p) => p.name).filter(Boolean);
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
// The third API call. Releases have no git equivalent — `ls-remote` reports
|
|
// refs, and a release is not a ref — so this is the one subject the cheaper
|
|
// channel cannot cover. Drafts are excluded: a draft is not published, so it is
|
|
// not a claim anyone can read. Pre-releases are kept; llm-ingestion-okf's
|
|
// v0.5.0a2 is its real newest release. Null on any failure, which reads as SKIP
|
|
// rather than as a pass.
|
|
async function fetchReleases(register, repo) {
|
|
const url = `${register.forge}/api/v1/repos/${register.org}/${repo}/releases?limit=50`;
|
|
try {
|
|
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
|
|
if (!res.ok) return null;
|
|
const json = JSON.parse(await res.text());
|
|
if (!Array.isArray(json)) return null;
|
|
return json.filter((r) => !r.draft).map((r) => r.tag_name).filter(Boolean);
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
async function fetchOrgListing(register) {
|
|
const url = `${register.forge}/api/v1/orgs/${register.org}/repos?limit=50`;
|
|
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
|
|
if (!res.ok) throw new Error(`org listing returned HTTP ${res.status}`);
|
|
return res.json();
|
|
}
|
|
|
|
function gitFiles(dir) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'ls-files'], { encoding: 'utf8' })
|
|
.split('\n')
|
|
.map((s) => s.trim())
|
|
.filter(Boolean);
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
// The remote is ground truth for what a repo is CALLED; the directory is only
|
|
// where it happens to sit. `catalog/` is the working directory of the repo named
|
|
// `ktg-plugin-marketplace`, and deriving the name from the basename left it
|
|
// REPO-UNREGISTERED — zero checks run against the one repo the catalog rule
|
|
// depends on. Handles the scp form too: the forge's clone button hands it out.
|
|
export function parseRepoNameFromRemote(url) {
|
|
const raw = String(url ?? '').trim();
|
|
if (!raw) return null;
|
|
const path = raw.includes('://') ? raw.split('://')[1] : raw;
|
|
const segments = path.replace(/\/+$/, '').split(/[/:]/).filter(Boolean);
|
|
// A bare host is not a repository. Without this, `https://the-forge/` parsed
|
|
// as a repo named after the host and every class rule matched the wrong thing.
|
|
if (segments.length < 2) return null;
|
|
const name = segments.pop().replace(/\.git$/, '');
|
|
return name || null;
|
|
}
|
|
|
|
function repoNameFrom(dir) {
|
|
try {
|
|
const remote = execFileSync('git', ['-C', dir, 'remote', 'get-url', 'origin'], {
|
|
encoding: 'utf8',
|
|
stdio: ['ignore', 'pipe', 'ignore'],
|
|
});
|
|
const fromRemote = parseRepoNameFromRemote(remote);
|
|
if (fromRemote) return fromRemote;
|
|
} catch { /* no remote yet — a repo before its first push is the ordinary case */ }
|
|
try {
|
|
return basename(execFileSync('git', ['-C', dir, 'rev-parse', '--show-toplevel'], { encoding: 'utf8' }).trim());
|
|
} catch {
|
|
return basename(dir);
|
|
}
|
|
}
|
|
|
|
// The version the package itself claims, from whichever manifest this class uses.
|
|
function readPackageVersion(dir) {
|
|
for (const p of ['.claude-plugin/plugin.json', 'package.json', 'pyproject.toml']) {
|
|
const full = join(dir, p);
|
|
if (!existsSync(full)) continue;
|
|
try {
|
|
const raw = readFileSync(full, 'utf8');
|
|
if (p.endsWith('.json')) {
|
|
const v = JSON.parse(raw).version;
|
|
if (v) return String(v);
|
|
} else {
|
|
const m = /^\s*version\s*=\s*["']([^"']+)["']/m.exec(raw);
|
|
if (m) return m[1];
|
|
}
|
|
} catch { /* unparseable — try the next one */ }
|
|
}
|
|
return null;
|
|
}
|
|
|
|
export function extractBadgeVersion(readmeText) {
|
|
const m = /badge\/version-(\d+\.\d+\.\d+)/.exec(readmeText || '');
|
|
return m ? m[1] : null;
|
|
}
|
|
|
|
// Newest released version in the CHANGELOG. `## [Unreleased]` is skipped by
|
|
// design — it is not a claim that anything shipped.
|
|
export function extractChangelogTop(changelogText) {
|
|
for (const line of String(changelogText || '').split('\n')) {
|
|
// The suffix class stops at `]`, whitespace or end of string, so a
|
|
// pre-release token (PEP 440 `a2`, semver `-beta.1`) is kept without
|
|
// reaching into a trailing `] — DATE`. Reported by llm-ingestion-okf:
|
|
// truncating this to X.Y.Z made VERSION-CHANGELOG disagree with
|
|
// VERSION-TAG, which compares the untruncated tag and does not have
|
|
// this problem — a repo on a pre-release could never reach 0 ERROR.
|
|
const m = /^##\s*\[?v?(\d+\.\d+\.\d+[0-9A-Za-z.+-]*)\]?/.exec(line.trim());
|
|
if (m) return m[1];
|
|
}
|
|
return null;
|
|
}
|
|
|
|
function gitTags(dir) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'tag', '--list', 'v*'], { encoding: 'utf8' })
|
|
.split('\n').map((s) => s.trim()).filter(Boolean);
|
|
} catch {
|
|
return [];
|
|
}
|
|
}
|
|
|
|
// `%(objecttype)` is `tag` for an annotated tag and `commit` for a lightweight
|
|
// one — the distinction read straight off the local object database, with no
|
|
// network call, so tag integrity costs nothing against the two-call budget.
|
|
// Refs from the FORGE, over the git protocol — anonymous, and measured not to
|
|
// share the API's rate-limit bucket, so it costs nothing against the two-call
|
|
// budget. The URL is derived from the register, never from `origin`: at least
|
|
// one repo's origin is `ssh://git@…`, which would need the operator's key and
|
|
// so would work here and fail for every other reader.
|
|
function forgeTags(register, repo) {
|
|
const forge = String(register?.forge ?? '').replace(/\/+$/, '');
|
|
if (!forge || !register?.org) return null;
|
|
try {
|
|
const out = execFileSync('git', ['ls-remote', '--tags', `${forge}/${register.org}/${repo}.git`], {
|
|
encoding: 'utf8',
|
|
env: { ...process.env, GIT_TERMINAL_PROMPT: '0' },
|
|
});
|
|
return out.split('\n')
|
|
.map((l) => l.split('\t')[1])
|
|
.filter((r) => r && !r.endsWith('^{}'))
|
|
.map((r) => r.replace(/^refs\/tags\//, ''));
|
|
} catch {
|
|
// Unreachable forge leaves this null, which reads as SKIP/notRun — never
|
|
// as a pass.
|
|
return null;
|
|
}
|
|
}
|
|
|
|
function gitTagObjects(dir) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'for-each-ref', '--format=%(objecttype) %(refname:short)', 'refs/tags/v*'], { encoding: 'utf8' })
|
|
.split('\n').map((s) => s.trim()).filter(Boolean)
|
|
.map((line) => {
|
|
const [type, ...rest] = line.split(' ');
|
|
return { name: rest.join(' '), annotated: type === 'tag' };
|
|
});
|
|
} catch {
|
|
return [];
|
|
}
|
|
}
|
|
|
|
export function inspectRepo(dir, name, register, description, catalogNames = null, offline = false, releases = null) {
|
|
const tracked = gitFiles(dir);
|
|
const present = (tracked ?? []).filter((f) => existsSync(join(dir, f)));
|
|
|
|
// Link scanning covers every tracked Markdown file — a dead reference in a
|
|
// doc is as broken as one in the README.
|
|
const files = {};
|
|
for (const f of (tracked ?? []).filter((p) => p.endsWith('.md'))) {
|
|
try { files[f] = readFileSync(join(dir, f), 'utf8'); } catch { /* unreadable — skip */ }
|
|
}
|
|
if (!files['README.md'] && existsSync(join(dir, 'README.md'))) {
|
|
files['README.md'] = readFileSync(join(dir, 'README.md'), 'utf8');
|
|
}
|
|
|
|
const readme = files['README.md'] ?? '';
|
|
let changelog = null;
|
|
try { changelog = readFileSync(join(dir, 'CHANGELOG.md'), 'utf8'); } catch { /* absent */ }
|
|
|
|
// Only the repos this README actually pins are fetched — one ref listing
|
|
// each, and nothing at all for the common case of no pins.
|
|
// This repo's own refs, from the forge rather than the clone — the side
|
|
// RELEASE-CURRENT compares a published release against, and the side
|
|
// REMOTE-SYNC will need next. Over the git protocol, so it costs nothing
|
|
// against the API budget.
|
|
let forgeTagsSelf = null;
|
|
let forgeTagsByRepo = null;
|
|
if (!offline) {
|
|
forgeTagsSelf = forgeTags(register, name);
|
|
forgeTagsByRepo = {};
|
|
for (const repo of new Set(extractInstallPins(readme, register).map((p) => p.repo))) {
|
|
const t = forgeTags(register, repo);
|
|
if (t) forgeTagsByRepo[repo] = t;
|
|
}
|
|
}
|
|
|
|
return classifyRepo({
|
|
name,
|
|
files,
|
|
present,
|
|
description,
|
|
forgeTagsByRepo,
|
|
pluginVersion: readPackageVersion(dir),
|
|
readmeBadge: extractBadgeVersion(readme),
|
|
changelogTop: changelog === null ? null : extractChangelogTop(changelog),
|
|
tags: gitTags(dir),
|
|
tagObjects: gitTagObjects(dir),
|
|
catalogNames,
|
|
forgeTagsSelf,
|
|
releases,
|
|
}, register);
|
|
}
|
|
|
|
// Grouped by bucket, because that is the order the findings actually get acted
|
|
// on: what blocks a stranger today, then what is absent, then what merely reads
|
|
// badly. Severity within a bucket is secondary to that.
|
|
const BUCKET_TITLE = {
|
|
broken: 'BROKEN NOW — a stranger is blocked or misled',
|
|
missing: 'MISSING — an expected artefact is absent',
|
|
weakening: 'WEAKENING — present, but it reads as amateur',
|
|
};
|
|
|
|
// A stale plugin cache once served 0.1.1 while 0.2.0 was installed and
|
|
// pinned, silently — the output looked like a clean pass, because nothing
|
|
// said which engine had run. This is the fix: name the version so a wrong
|
|
// engine is visible, not just correctable in hindsight.
|
|
const MARK = { OK: '✓', WARN: '!', ERROR: '✗', SKIP: '·' };
|
|
|
|
// Both axes on the one line a sweep actually reads. Letting `status` mean
|
|
// judgement fixed "clean repos look skipped"; printing a bare OK next to a
|
|
// check that never ran would trade it for "skipped checks look clean", which is
|
|
// the worse direction.
|
|
//
|
|
// Since 0.8.0 the line names only what someone has an ACTION on (operator
|
|
// decision 2026-08-09). A deliberate skip is a recorded decision, not an
|
|
// unread check, and eleven of them behind an otherwise clean repo said the
|
|
// opposite on every row of the sweep. They are not silenced: they keep their
|
|
// own sub-heading in the body, which is where "exposure, not silence" lives.
|
|
//
|
|
// Two generations of older result objects still print correctly, and neither
|
|
// absence reads as zero: no `skips` falls back to the 0.7.0 total, no
|
|
// `notChecked` to the line from before coverage existed at all.
|
|
export function headerLine(result, engineVersion, engineCommit = null) {
|
|
const klass = result.klass ? ` [${result.klass}]` : '';
|
|
const traits = result.traits?.length ? ` {${result.traits.join(', ')}}` : '';
|
|
const sha = engineCommit ? ` @${String(engineCommit).slice(0, 7)}` : '';
|
|
const coverage = result.skips
|
|
? (result.skips.notRun > 0 ? ` · ${result.skips.notRun} not run` : '')
|
|
: (result.notChecked > 0 ? ` · ${result.notChecked} not checked` : '');
|
|
return `${MARK[result.status]} ${result.name}${klass}${traits} — ${result.status}${coverage} (repo-standard v${engineVersion}${sha})`;
|
|
}
|
|
|
|
// `engineCommit` is always present, null when underivable: an ABSENT key means
|
|
// an older engine, an explicit null means this engine ran and had no HEAD to
|
|
// read. A consumer sorting raw files by stamp needs those to be different.
|
|
export function withEngineVersion(result, engineVersion, engineCommit = null) {
|
|
return { ...result, engineVersion, engineCommit };
|
|
}
|
|
|
|
function render(result, engineVersion, engineCommit) {
|
|
const mark = MARK;
|
|
console.log(`\n${headerLine(result, engineVersion, engineCommit)}`);
|
|
|
|
for (const bucket of BUCKETS) {
|
|
const inBucket = result.findings.filter((f) => f.bucket === bucket);
|
|
if (!inBucket.length) continue;
|
|
console.log(`\n ${BUCKET_TITLE[bucket]}`);
|
|
for (const f of inBucket) console.log(` ${mark[f.level]} ${f.level} ${f.code}: ${f.msg}`);
|
|
}
|
|
|
|
// Two sub-headings, because the header line no longer carries the deliberate
|
|
// ones. This is the only place they are visible, and a decision nobody can
|
|
// see reads exactly like a check that silently stopped running.
|
|
const skipped = groupSkips(result.findings);
|
|
if (skipped.notRun.length) {
|
|
console.log('\n NOT CHECKED — these are not passes');
|
|
for (const f of skipped.notRun) console.log(` ${mark.SKIP} ${f.code}: ${f.msg}`);
|
|
}
|
|
if (skipped.byDesign.length) {
|
|
console.log('\n NOT JUDGED — deliberately outside what this gate decides');
|
|
for (const f of skipped.byDesign) console.log(` ${mark.SKIP} ${f.code}: ${f.msg}`);
|
|
}
|
|
|
|
const okCount = result.findings.filter((f) => f.level === 'OK').length;
|
|
console.log(`\n ${mark.OK} ${okCount} check(s) passed`);
|
|
}
|
|
|
|
async function refresh(register) {
|
|
const live = await fetchOrgListing(register);
|
|
const liveNames = new Set(live.map((r) => r.name));
|
|
const known = new Set(Object.keys(register.repos ?? {}));
|
|
|
|
const added = [...liveNames].filter((n) => !known.has(n)).sort();
|
|
const gone = [...known].filter((n) => !liveNames.has(n)).sort();
|
|
|
|
console.log(`register: ${known.size} repos · forge: ${liveNames.size} repos`);
|
|
if (added.length) console.log(`\n on the forge, not in the register (add with a class):\n ${added.join('\n ')}`);
|
|
if (gone.length) console.log(`\n in the register, not on the forge:\n ${gone.join('\n ')}`);
|
|
if (!added.length && !gone.length) console.log('\n ✓ register matches the forge');
|
|
return added.length + gone.length === 0 ? 0 : 1;
|
|
}
|
|
|
|
async function main(argv) {
|
|
const arg = (flag, fallback = null) => {
|
|
const i = argv.indexOf(flag);
|
|
return i === -1 ? fallback : argv[i + 1];
|
|
};
|
|
const register = loadRegister();
|
|
|
|
if (argv.includes('--refresh')) {
|
|
process.exit(await refresh(register));
|
|
}
|
|
|
|
const dir = arg('--dir', process.cwd());
|
|
const name = arg('--name', repoNameFrom(dir));
|
|
|
|
let description = null;
|
|
let catalogNames = null;
|
|
let releases = null;
|
|
if (!argv.includes('--offline')) {
|
|
catalogNames = await fetchCatalogNames(register);
|
|
releases = await fetchReleases(register, name);
|
|
try {
|
|
const listing = await fetchOrgListing(register);
|
|
const row = listing.find((r) => r.name === name);
|
|
description = row ? (row.description ?? '') : null;
|
|
} catch {
|
|
// Unreachable forge leaves description null, which reads as SKIP — never
|
|
// as a pass. A check that could not run says so.
|
|
}
|
|
}
|
|
|
|
const result = inspectRepo(dir, name, register, description, catalogNames, argv.includes('--offline'), releases);
|
|
const engineVersion = readEngineVersion();
|
|
const engineCommit = readEngineCommit();
|
|
|
|
if (argv.includes('--json')) {
|
|
console.log(JSON.stringify(withEngineVersion(result, engineVersion, engineCommit), null, 2));
|
|
} else {
|
|
render(result, engineVersion, engineCommit);
|
|
}
|
|
process.exit(result.status === 'ERROR' ? 1 : 0);
|
|
}
|
|
|
|
if (process.argv[1] && process.argv[1].endsWith('repo-standard-check.mjs')) {
|
|
main(process.argv.slice(2));
|
|
}
|