Kjell Tore Guttormsen
5884a64e54
Three changes at the register/engine boundary, all agreed with org-ops after census 05 and all about a check missing a place to record a legitimate exception. `titles`: an optional per-repo README title. Set, the H1 matches it and is OK; unset, the WARN stands as before. A human title was already this engine's stated position and rds-v1's prescription, but a decided YES had nowhere to live, so the same 6 WARNs were reported three censuses running and would have been reported forever. Five registered, each H1 read from the repo rather than copied from the census; `ai-psychosis` deliberately left out so the one repo where a reader cannot connect title to name stands alone. Measured across 21 local clones: 6 WARN before, 1 after, nothing else moved. `readme_desc_match: false` on the org-profile class: for an ordinary repo the README opening and the forge description describe the same subject and equality is right; for this class they do not — the README is the org's landing page, the forge text describes the repo. The equality is what does not apply, not either text. Class data, not a hardcoded name, and the exemption is RECORDED as an OK naming its reason, not dropped. `.profile` went ERROR to 0 ERROR / 0 WARN; the same README under a plugin class is still an ERROR. `engineCommit`: the version names a file, only the sha names the code. A sweep stamped 18 files 0.4.0 while four carried a 0.5.0-only finding — feature and version bump are two commits, so the stamp lied without being broken. Present-and-null when underivable, never absent: an absent key means an older engine, null means this one ran without a HEAD to read. 135 to 147 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NELsvPY5gnJjN3esdhWYWC
1250 lines
54 KiB
JavaScript
1250 lines
54 KiB
JavaScript
#!/usr/bin/env node
|
|
// repo-standard — the per-repo gate.
|
|
//
|
|
// Checks ONE repository against the standard for its class:
|
|
// - README first screen: H1 is the repo name, next line IS the forge description
|
|
// - Install block complete, in the form its class actually uses
|
|
// - Required files present for its class
|
|
// - Every `open/<name>` reference in URL position resolves
|
|
// - Description within the length bound, measured in codepoints
|
|
//
|
|
// What it deliberately does NOT do: anything that needs to see all repos at once.
|
|
// Divergence across the org (0/18 topics, three competing install forms, README
|
|
// release notes duplicating a CHANGELOG that 16 of 18 repos have) is invisible
|
|
// from inside one repo. Those checks live in org-ops, not here.
|
|
//
|
|
// Structure mirrors the marketplace's check-versions.mjs on purpose: pure
|
|
// classifiers with all I/O resolved into their input, findings tagged
|
|
// ERROR/WARN/SKIP/OK, exit 1 on ERROR. This is a gate, not a checklist —
|
|
// the catalog's eleven descriptions are good because a gate runs on them; the
|
|
// forge's nine were empty. Same care, different outcome.
|
|
//
|
|
// Usage:
|
|
// node scripts/repo-standard-check.mjs [--dir <path>] [--name <repo>] [--offline] [--json]
|
|
// node scripts/repo-standard-check.mjs --refresh # register vs. live org listing
|
|
import { readFileSync, existsSync } from 'node:fs';
|
|
import { execFileSync } from 'node:child_process';
|
|
import { join, dirname, basename } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
const REGISTER_PATH = join(HERE, '..', 'register', 'repos.json');
|
|
const PACKAGE_PATH = join(HERE, '..', 'package.json');
|
|
|
|
// This engine's own version, not the target repo's — distinct from
|
|
// readPackageVersion(dir) below, which reads the REPO BEING CHECKED.
|
|
export function readEngineVersion(path = PACKAGE_PATH) {
|
|
return JSON.parse(readFileSync(path, 'utf8')).version;
|
|
}
|
|
|
|
// The version names a FILE; only the sha names the CODE. A sweep once stamped
|
|
// 18 raw files `0.4.0` while four of them carried findings from a check that
|
|
// only exists in 0.5.0 — the feature and the version bump are two commits, so
|
|
// the worktree held new code under an old number for a window. Derived from
|
|
// this checkout, no network call; null outside a git checkout (a vendored copy
|
|
// or a tarball has no HEAD, and that is not a crash).
|
|
export function readEngineCommit(dir = join(HERE, '..')) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'rev-parse', 'HEAD'], {
|
|
encoding: 'utf8',
|
|
stdio: ['ignore', 'pipe', 'ignore'],
|
|
}).trim() || null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
const LEVELS = ['OK', 'SKIP', 'WARN', 'ERROR'];
|
|
|
|
// Findings carry a level AND a bucket, and the two are independent axes.
|
|
// The level says how sure and how loud; the bucket says what KIND of problem it
|
|
// is, which is what a reader triages on:
|
|
//
|
|
// broken works wrongly right now — a stranger is blocked or misled
|
|
// missing an expected artefact is simply absent
|
|
// weakening present and functional, but it reads as amateur
|
|
//
|
|
// A weakening finding can still be an ERROR: a README opening line that
|
|
// contradicts the published description blocks nobody, and is still wrong.
|
|
export const BUCKETS = ['broken', 'missing', 'weakening'];
|
|
|
|
// ------------------------------------------------------------ pure helpers
|
|
|
|
// Codepoints. Not bytes (an em-dash costs 3) and not UTF-16 units (`👉` costs 2).
|
|
// The em-dash exposes only the outer layer, which is why "characters, not bytes"
|
|
// was not enough on its own.
|
|
export function countCodepoints(s) {
|
|
return [...String(s ?? '')].length;
|
|
}
|
|
|
|
// ~20 "dead" repo names collapsed to 3 real ones once this ran. A clone URL
|
|
// ending in .git is a legitimate reference, not a broken one.
|
|
export function normalizeRepoRef(raw) {
|
|
return String(raw ?? '')
|
|
.replace(/\/+$/, '')
|
|
.replace(/\.git$/, '');
|
|
}
|
|
|
|
// Only names in URL position are resolvable references. That single rule
|
|
// excludes all three of the measured "correct text that looks broken" cases at
|
|
// once: a path position (`~/.claude/coord/_broadcast/`), running prose (`coord`
|
|
// is still the transport protocol's name), and a bare directory name.
|
|
//
|
|
// The host segment excludes `/`: `open` must be the FIRST path segment after
|
|
// the host, matching how every real repo URL is shaped
|
|
// (`https://host/open/<name>`, `user@host:open/<name>.git`). Without that
|
|
// restriction, an API endpoint like `/api/v1/orgs/open/repos` also matches —
|
|
// `open` there is the org argument to the API, and `repos` is the literal
|
|
// resource segment, not a repo name (measured: catalog RUNBOOK.md:39, :114).
|
|
//
|
|
// The third alternative is the SCHEMELESS host: `git.fromaitochitta.com/open/x`
|
|
// written without `https://`, which a subtree instruction routinely is
|
|
// (measured false negative: llm-security/V3-UPGRADE.md:343 — the scheme was
|
|
// doing work it was never entitled to, and its absence hid a WARN). What makes
|
|
// a name resolvable is its position after a HOST, not the scheme in front of
|
|
// it. Two guards keep that from becoming the noise the scheme was masking: the
|
|
// host must end in a TLD-shaped label, and the lookbehind refuses a host
|
|
// preceded by `/`, `.` or a word character — because that is a PATH segment
|
|
// that merely contains a dot (`docs/v1.2/open/`, `test/nav.golden/open/`), not
|
|
// a host. The API-endpoint rule survives untouched: `orgs` carries no dot.
|
|
const URL_REF = /(?::\/\/[^\s)\]"'`/]+\/|@[^\s:]+:|(?<![A-Za-z0-9._/-])[A-Za-z0-9-]+(?:\.[A-Za-z0-9-]+)*\.[A-Za-z]{2,}\/)open\/([A-Za-z0-9._-]+)/g;
|
|
|
|
export function extractOpenRefs(text) {
|
|
const out = [];
|
|
const lines = String(text ?? '').split('\n');
|
|
lines.forEach((line, i) => {
|
|
for (const m of line.matchAll(URL_REF)) {
|
|
out.push({ name: normalizeRepoRef(m[1]), line: i + 1, raw: m[0] });
|
|
}
|
|
});
|
|
return out;
|
|
}
|
|
|
|
// Three outcomes, never two. "No match" and "match on something that is not a
|
|
// repo" must stay distinguishable — if they share an outcome, the loss goes
|
|
// silent, and silent loss is the defect class this standard exists to catch.
|
|
export function classifyRef(name, register) {
|
|
if (Object.prototype.hasOwnProperty.call(register.repos ?? {}, name)) return 'repo';
|
|
if (Object.prototype.hasOwnProperty.call(register.non_repos ?? {}, name)) return 'non-repo';
|
|
return 'unknown';
|
|
}
|
|
|
|
// The same "three outcomes, never two" rule this file applies to classifyRef,
|
|
// applied to the check's own result. Emitting nothing on success made "no dead
|
|
// references" and "the check never ran" identical in the output, so a sweep
|
|
// across the org could not tell 19 clean repos from 19 unread ones (measured:
|
|
// org-ops census 03b). Silence is not a pass here either.
|
|
export function checkLinks({ files }, register) {
|
|
const entries = Object.entries(files ?? {});
|
|
if (entries.length === 0) {
|
|
return [{
|
|
level: 'SKIP',
|
|
code: 'LINKS-OPEN-REFS-UNAVAILABLE',
|
|
msg: 'no files were enumerated — the `open/` reference check did not run',
|
|
}];
|
|
}
|
|
const findings = [];
|
|
// One name, one line, one reference. `[host/open/x](https://host/open/x)`
|
|
// puts the same reference on both halves of a markdown link and matched
|
|
// twice once the schemaless host became legible (measured:
|
|
// llm-security/V3-ANNOUNCEMENT.md:124). The key keeps name AND line, so two
|
|
// different names on one line — or the same name on two lines — stay two.
|
|
const seen = new Set();
|
|
let checked = 0;
|
|
for (const [path, text] of entries) {
|
|
for (const ref of extractOpenRefs(text)) {
|
|
const key = `${path}\n${ref.line}\n${ref.name}`;
|
|
if (seen.has(key)) continue;
|
|
seen.add(key);
|
|
checked += 1;
|
|
const kind = classifyRef(ref.name, register);
|
|
if (kind === 'repo') continue;
|
|
if (kind === 'non-repo') {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'LINK-NON-REPO',
|
|
bucket: 'weakening',
|
|
msg: `${path}:${ref.line} — \`open/${ref.name}\` resolves to a known non-repo: ${register.non_repos[ref.name]}`,
|
|
});
|
|
} else {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'LINK-DEAD',
|
|
bucket: 'broken',
|
|
msg: `${path}:${ref.line} — \`open/${ref.name}\` matches no repo in the register (dead reference)`,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
// The count IS the evidence. An OK that cannot say how many references it
|
|
// resolved is the same silence wearing a different level.
|
|
if (findings.length === 0) {
|
|
findings.push({
|
|
level: 'OK',
|
|
code: 'LINKS-OPEN-REFS',
|
|
msg: checked === 0
|
|
? `no \`open/\` references found in ${entries.length} scanned file(s)`
|
|
: `${checked} \`open/\` reference(s) checked — every one resolves to a registered repo`,
|
|
});
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
export function checkDescription(description, register) {
|
|
if (description === null || description === undefined) {
|
|
return [{ level: 'SKIP', code: 'DESC-UNAVAILABLE', msg: 'forge description not available — check not run (offline, or the listing failed)' }];
|
|
}
|
|
const max = register.description_max_codepoints ?? 180;
|
|
const n = countCodepoints(description);
|
|
if (n === 0) return [{ level: 'ERROR', code: 'DESC-EMPTY', bucket: 'missing', msg: 'forge description is empty' }];
|
|
if (n > max) {
|
|
return [{ level: 'ERROR', code: 'DESC-TOO-LONG', bucket: 'weakening', msg: `forge description is ${n} codepoints, bound is ${max}` }];
|
|
}
|
|
return [{ level: 'OK', code: 'DESC', msg: `description ${n}/${max} codepoints` }];
|
|
}
|
|
|
|
// The opening line makes description == catalog == README: the same thread on a
|
|
// third surface, and the only one of the three a machine can check from inside
|
|
// the repo.
|
|
export function checkFirstScreen({ readme, name, description, klass }, register) {
|
|
const findings = [];
|
|
const lines = String(readme ?? '').split('\n');
|
|
const firstIdx = lines.findIndex((l) => l.trim() !== '');
|
|
|
|
const heading = firstIdx === -1 ? null : lines[firstIdx].trim();
|
|
|
|
// No heading at all is broken. A heading that merely differs from the repo
|
|
// name is not: the thread that has to hold is description == catalog ==
|
|
// opening line, and the H1 is none of those three. A human title like
|
|
// `# OKR for Public Sector` is a naming choice the operator owns, so it is
|
|
// surfaced and left to them — a gate that fails a correct repo is the
|
|
// mechanism that gets gates switched off.
|
|
if (heading === null || !heading.startsWith('# ')) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'README-H1',
|
|
bucket: 'missing',
|
|
msg: `README must open with an H1 (expected \`# ${name}\`, found: ${heading === null ? '<empty file>' : `\`${heading}\``})`,
|
|
});
|
|
return findings;
|
|
}
|
|
|
|
// A registered title is where that operator call gets WRITTEN DOWN. Without
|
|
// one, the same WARN reappears every census and "we decided this is correct"
|
|
// is indistinguishable from "nobody has looked". With one, the two separate —
|
|
// and a repo whose title nobody has ruled on is left standing alone, which is
|
|
// the wanted side effect, not a cost.
|
|
const title = register?.titles?.[name];
|
|
if (heading === `# ${name}`) {
|
|
findings.push({ level: 'OK', code: 'README-H1', msg: `H1 is \`# ${name}\`` });
|
|
} else if (title && heading === `# ${title}`) {
|
|
findings.push({ level: 'OK', code: 'README-H1', msg: `H1 is \`# ${title}\` — the registered title for \`${name}\`` });
|
|
} else if (title) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'README-H1',
|
|
bucket: 'weakening',
|
|
msg: `H1 is \`${heading}\`, which is neither \`# ${name}\` nor the registered title \`${title}\` — one of the two has drifted.`,
|
|
});
|
|
} else {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'README-H1',
|
|
bucket: 'weakening',
|
|
msg: `H1 is \`${heading}\`, not \`# ${name}\` — deliberate title, or drift? Operator's call. Record the decision as \`titles.${name}\` in the register.`,
|
|
});
|
|
}
|
|
|
|
// For an ordinary repo the README opening and the forge description describe
|
|
// the SAME subject, and equality is the right demand. `org-profile` is the one
|
|
// class where they do not: its README is the organisation's landing page and
|
|
// the forge text describes the repo. Both are correct about their own subject,
|
|
// so it is the EQUALITY that does not apply — and a landing page's opening
|
|
// link could only ever match by putting raw markdown on a plain-text surface.
|
|
// Class-level data, not a hardcoded name: class rules live in the register.
|
|
if (register?.classes?.[klass]?.readme_desc_match === false) {
|
|
findings.push({
|
|
level: 'OK',
|
|
code: 'README-DESC',
|
|
msg: `class \`${klass}\` is exempt: the README describes the org, the forge description describes the repo — different subjects, so equality is not required`,
|
|
});
|
|
return findings;
|
|
}
|
|
|
|
if (description === null || description === undefined) {
|
|
findings.push({ level: 'SKIP', code: 'README-DESC', msg: 'forge description not available — opening-line match not checked' });
|
|
return findings;
|
|
}
|
|
|
|
const restIdx = lines.findIndex((l, i) => i > firstIdx && l.trim() !== '');
|
|
const opening = restIdx === -1 ? '' : lines[restIdx].trim();
|
|
if (opening !== String(description).trim()) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'README-DESC',
|
|
bucket: 'weakening',
|
|
msg: `README opening line does not match the forge description\n README: ${opening}\n forge: ${description}`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'OK', code: 'README-DESC', msg: 'opening line matches the forge description' });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// `claude plugin install x@mkt` or `/plugin install x@mkt` — the two CLI forms.
|
|
function hasCliInstall(readme, name, mkt) {
|
|
const esc = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
return new RegExp(`(?:claude\\s+plugin|/plugin)\\s+install\\s+${esc(name)}@${esc(mkt)}\\b`).test(readme);
|
|
}
|
|
|
|
function hasAnyPluginInstall(readme) {
|
|
return /(?:claude\s+plugin|\/plugin)\s+install\s+\S+@\S+/.test(readme);
|
|
}
|
|
|
|
export function checkInstallBlock({ readme, name, klass }, register) {
|
|
const form = register.classes?.[klass]?.install ?? 'none';
|
|
const text = String(readme ?? '');
|
|
const mkt = register.marketplace ?? {};
|
|
const findings = [];
|
|
|
|
if (form === 'none') return findings;
|
|
|
|
const addLines = text.split('\n').filter((l) => /plugin\s+marketplace\s+add/.test(l));
|
|
const hasAdd = addLines.length > 0;
|
|
|
|
// The forge UI's clone button hands out the ssh URL, and `marketplace add`
|
|
// answers it with "Invalid git URL" — a message that never mentions the
|
|
// protocol. Measured end-to-end 2026-07-25.
|
|
if (addLines.some((l) => /ssh:\/\//.test(l))) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-SSH',
|
|
bucket: 'broken',
|
|
msg: '`marketplace add` is shown with an ssh:// URL — it rejects those ("Invalid git URL"). Use the https form.',
|
|
});
|
|
}
|
|
|
|
// A well-formed command pointing at the wrong marketplace is still a command
|
|
// that does not work. Checked against the register, so it needs no network.
|
|
if (hasAdd && mkt.url) {
|
|
const urls = addLines
|
|
.map((l) => /marketplace\s+add\s+(\S+)/.exec(l)?.[1])
|
|
.filter(Boolean)
|
|
.map((u) => u.replace(/[`'"]+$/, ''));
|
|
if (urls.length && !urls.some((u) => normalizeRepoRef(u) === normalizeRepoRef(mkt.url))) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-URL-MISMATCH',
|
|
bucket: 'broken',
|
|
msg: `\`marketplace add\` points at ${urls[0]}, but this marketplace is ${mkt.url}`,
|
|
});
|
|
}
|
|
}
|
|
|
|
if (form === 'plugin' || form === 'catalog') {
|
|
if (!hasAdd) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-NO-MARKETPLACE',
|
|
bucket: 'broken',
|
|
msg: `no \`plugin marketplace add\` line — the reader is never told to add \`${mkt.name}\` (${mkt.url})`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'OK', code: 'INSTALL-MARKETPLACE', msg: '`marketplace add` present' });
|
|
}
|
|
}
|
|
|
|
if (form === 'plugin') {
|
|
// The corrected defect A. `enabledPlugins` in settings.json is a LEGITIMATE
|
|
// second form and it stands in 10 of 11 plugin READMEs — what is missing in
|
|
// 7 of them is a CLI command. So the contract requires the command and
|
|
// permits the JSON alongside it; it never accepts the JSON as a substitute.
|
|
// A reader who scrolls to the JSON block has a complete path; an agent told
|
|
// "install this" reaches for the CLI and finds `marketplace add` and nothing else.
|
|
if (!hasCliInstall(text, name, mkt.name)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-NO-CLI',
|
|
bucket: 'broken',
|
|
msg: `no CLI install command for this repo — expected \`claude plugin install ${name}@${mkt.name}\` (or the \`/plugin install\` form). An \`enabledPlugins\` block is a welcome addition, but it is not a CLI command.`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'OK', code: 'INSTALL-CLI', msg: `CLI install command names ${name}@${mkt.name}` });
|
|
}
|
|
}
|
|
|
|
if (form === 'vendor' && hasAnyPluginInstall(text)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-WRONG-FORM',
|
|
bucket: 'broken',
|
|
msg: 'shared asset shows a plugin install line — it is vendored into consumers, not installed. Document how to vendor it.',
|
|
});
|
|
}
|
|
|
|
if (form === 'package') {
|
|
if (hasAnyPluginInstall(text)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'INSTALL-WRONG-FORM',
|
|
bucket: 'broken',
|
|
msg: 'standalone project shows a plugin install line — use the pip/uv form.',
|
|
});
|
|
} else if (!/\b(pip\s+install|uv\s+(?:pip\s+)?(?:add|sync|install|run)|uvx)\b/.test(text)) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'INSTALL-NO-PACKAGE-FORM',
|
|
bucket: 'missing',
|
|
msg: 'no pip/uv install form found — expected for a standalone project',
|
|
});
|
|
}
|
|
}
|
|
|
|
return findings;
|
|
}
|
|
|
|
// Per class, never flat. A flat standard demands a CONTRIBUTING from a CSS
|
|
// library that takes no contributions and a ROADMAP from a five-line profile.
|
|
// Syntax is not truth. The most disqualifying failure a repo can have is an
|
|
// install command that does not work for a stranger, and a perfectly formed
|
|
// `claude plugin install x@mkt` fails silently if `x` was never pinned in the
|
|
// catalog. This is the one check that answers the brief's first question.
|
|
export function checkInstallTruth({ name, klass, catalogNames }) {
|
|
if (klass !== 'plugin') return [{ level: 'OK', code: 'INSTALL-TRUTH', msg: 'not a marketplace plugin — nothing to resolve' }];
|
|
if (!catalogNames) {
|
|
return [{ level: 'SKIP', code: 'INSTALL-TRUTH', msg: 'catalog not reachable — cannot verify the install command actually resolves' }];
|
|
}
|
|
if (!catalogNames.includes(name)) {
|
|
return [{
|
|
level: 'ERROR',
|
|
code: 'INSTALL-NOT-IN-CATALOG',
|
|
bucket: 'broken',
|
|
msg: `\`${name}\` is not pinned in the marketplace catalog — the documented install command cannot succeed for anyone`,
|
|
}];
|
|
}
|
|
return [{ level: 'OK', code: 'INSTALL-TRUTH', msg: 'the install command resolves against the catalog' }];
|
|
}
|
|
|
|
// Requirements come from two axes. The CLASS is structural — it can be read off
|
|
// the catalog and the remotes. A TRAIT is about what the code does, which no
|
|
// remote can tell you: `security` attaches the obligations a tool acquires by
|
|
// handling untrusted input.
|
|
//
|
|
// Note what is NOT here: CONTRIBUTING, CODE_OF_CONDUCT, MAINTAINERS. The
|
|
// maintainer works alone and the published stance says so. Contributor-facing
|
|
// documentation for a project that accepts no contributors is theatre, and a
|
|
// code of conduct with an unattended placeholder address is worse than none.
|
|
// Consumer-facing documents are untouched by that — SECURITY.md exists for the
|
|
// outsider who finds a hole, and being solo does not remove them.
|
|
function requirementsFor(klass, traits, register) {
|
|
const cls = register.classes?.[klass] ?? {};
|
|
const files = [...(cls.required_files ?? [])];
|
|
const headings = [...(cls.required_headings ?? [])];
|
|
for (const t of traits ?? []) {
|
|
const tr = register.trait_requirements?.[t];
|
|
if (!tr) continue;
|
|
for (const f of tr.required_files ?? []) if (!files.includes(f)) files.push(f);
|
|
for (const h of tr.required_headings ?? []) if (!headings.includes(h)) headings.push(h);
|
|
}
|
|
return { files, headings };
|
|
}
|
|
|
|
export function checkRequiredFiles({ present, klass, traits }, register) {
|
|
const { files: required } = requirementsFor(klass, traits, register);
|
|
const have = new Set(present ?? []);
|
|
const findings = [];
|
|
for (const f of required) {
|
|
if (!have.has(f)) {
|
|
findings.push({ level: 'ERROR', code: 'FILE-MISSING', bucket: 'missing', msg: `missing required file for class \`${klass}\`: ${f}` });
|
|
}
|
|
}
|
|
if (findings.length === 0 && required.length > 0) {
|
|
findings.push({ level: 'OK', code: 'FILES', msg: `all ${required.length} required files present` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// Fixed headings, because experienced readers skip rather than read. `## Install`
|
|
// on a predictable heading is what agents pattern-match on, and `## Non-goals`
|
|
// is the cheapest trust-builder there is: it proves someone thought about the
|
|
// boundary, and it stops misuse before it starts.
|
|
export function checkHeadings({ readme, klass, traits }, register) {
|
|
const { headings: required } = requirementsFor(klass, traits, register);
|
|
const text = String(readme ?? '');
|
|
const present = new Set(
|
|
text.split('\n').map((l) => l.trim()).filter((l) => l.startsWith('#')),
|
|
);
|
|
const findings = [];
|
|
for (const h of required) {
|
|
if ([...present].some((p) => p.toLowerCase() === h.toLowerCase())) continue;
|
|
|
|
// Same title, wrong depth: say that, rather than "missing". The contract
|
|
// wants a predictable top-level heading because that is what an agent
|
|
// pattern-matches on — but the section does exist, and the fix is a
|
|
// different edit than writing one from scratch.
|
|
const title = h.replace(/^#+\s*/, '');
|
|
const atOtherLevel = [...present].find(
|
|
(p) => p.replace(/^#+\s*/, '').toLowerCase() === title.toLowerCase(),
|
|
);
|
|
if (atOtherLevel) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'HEADING-LEVEL',
|
|
bucket: 'weakening',
|
|
msg: `README has \`${atOtherLevel}\` but the contract wants \`${h}\` — a predictable top-level heading is what readers and agents scan for`,
|
|
});
|
|
} else {
|
|
findings.push({ level: 'ERROR', code: 'HEADING-MISSING', bucket: 'missing', msg: `README has no \`${h}\` section` });
|
|
}
|
|
}
|
|
if (findings.length === 0 && required.length > 0) {
|
|
findings.push({ level: 'OK', code: 'HEADINGS', msg: `all ${required.length} required headings present` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// One version, four places it can be written down. This is the check that
|
|
// removes a whole defect class — "README says v0.3.1, the tag does not exist" —
|
|
// and the one that would have caught this repo's own 32→34 test-count drift.
|
|
export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogTop, tags }) {
|
|
const findings = [];
|
|
const v = pluginVersion ? String(pluginVersion).replace(/^v/, '') : null;
|
|
if (!v) {
|
|
return [{ level: 'SKIP', code: 'VERSION-UNAVAILABLE', msg: 'no package version found — nothing to compare against' }];
|
|
}
|
|
|
|
if (readmeBadge !== null && readmeBadge !== undefined && readmeBadge !== v) {
|
|
findings.push({ level: 'ERROR', code: 'VERSION-BADGE', bucket: 'weakening', msg: `README version badge is ${readmeBadge}, manifest says ${v}` });
|
|
}
|
|
if (changelogTop !== null && changelogTop !== undefined && changelogTop !== v) {
|
|
findings.push({ level: 'ERROR', code: 'VERSION-CHANGELOG', bucket: 'weakening', msg: `newest CHANGELOG entry is ${changelogTop}, manifest says ${v}` });
|
|
}
|
|
|
|
// Nothing released yet is a state, not a defect — and it must say so rather
|
|
// than pass quietly, because "SKIP is never a pass" is the whole discipline.
|
|
if (!tags || tags.length === 0) {
|
|
findings.push({ level: 'SKIP', code: 'VERSION-TAG', msg: `repo has no tags — cannot verify that v${v} was ever released` });
|
|
} else if (!tags.includes(`v${v}`)) {
|
|
findings.push({ level: 'ERROR', code: 'VERSION-TAG', bucket: 'broken', msg: `no tag \`v${v}\` — the documented version was never released (tags: ${tags.slice(-3).join(', ')})` });
|
|
}
|
|
|
|
if (findings.every((f) => f.level === 'OK' || f.level === 'SKIP')) {
|
|
findings.push({ level: 'OK', code: 'VERSION', msg: `version ${v} agrees across manifest, README and CHANGELOG` });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
// A static image asserting "tests: 642 passing" is a claim dressed as evidence.
|
|
// Version, licence and platform badges assert no run, so they are fine static.
|
|
// Bare `status` used to be in this list and caught a self-declared maturity
|
|
// badge ("status: alpha") as if it were a run claim — reported by
|
|
// llm-ingestion-pipeline-security. `build`/`ci`/`passing` already catch the
|
|
// run-asserting compounds ("build status", "CI status"), so dropping the bare
|
|
// word loses no real detection.
|
|
const CLAIM_BADGE = /(tests?|build|ci|coverage|passing)/i;
|
|
|
|
// Counting badges needs a NARROWER rule than detecting a dishonest one. The
|
|
// claim check reads any image, any host, on purpose. Here the opposite error
|
|
// matters: counting a screenshot or an architecture diagram as clutter would
|
|
// punish exactly the visual work this standard wants more of.
|
|
const BADGE_URL = /shields\.io|badgen\.net|\/badges?[/.]/i;
|
|
|
|
// Trockman et al., ICSE 2018 (doi 10.1145/3180155.3180209, n=294,941 npm
|
|
// packages): badge count relates to popularity non-linearly with a predicted
|
|
// inflection at five, and surveyed maintainers called over-badged READMEs
|
|
// cluttered and "trying too hard". WARN, never ERROR — the coefficient sits in
|
|
// an appendix with no CI or p-value, so it carries "more is not better" and
|
|
// cannot carry a hard limit.
|
|
const BADGE_INFLECTION = 5;
|
|
|
|
export function checkBadges({ readme, present }) {
|
|
const have = new Set(present ?? []);
|
|
const findings = [];
|
|
let badgeCount = 0;
|
|
for (const line of String(readme ?? '').split('\n')) {
|
|
// Any image, any host. Restricting this to img.shields.io would have missed
|
|
// a self-hosted SVG asserting exactly the same unverified thing.
|
|
// The trailing `(?:\]\(target\))?` is the LINK the badge is wrapped in —
|
|
// previously unmatched, so being linked at all silently ended scrutiny
|
|
// whether or not the link actually went anywhere.
|
|
for (const m of line.matchAll(/(\[)?!\[([^\]]*)\]\(([^)\s]+)\)(?:\]\(([^)\s]+)\))?/g)) {
|
|
const linkTarget = m[1] === '[' ? m[4] : undefined;
|
|
const linked = linkTarget !== undefined;
|
|
const label = `${m[2]} ${m[3]}`;
|
|
if (BADGE_URL.test(m[3])) badgeCount++;
|
|
if (!CLAIM_BADGE.test(label)) continue;
|
|
if (!linked) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'BADGE-STATIC-CLAIM',
|
|
bucket: 'weakening',
|
|
msg: `static badge asserts a run that nothing verifies: \`${m[2]}\`. A badge like this is a claim dressed as evidence — link it to a real run, or drop it.`,
|
|
});
|
|
continue;
|
|
}
|
|
// A linked badge is only as honest as its target. An external target
|
|
// (the ordinary case — a CI provider's own page) needs the network to
|
|
// verify and is deliberately out of scope, same as checkInternalLinks.
|
|
// A relative target this gate CAN check without the network — and a
|
|
// relative target that resolves nowhere is worse than a static badge:
|
|
// it LOOKS verified.
|
|
if (/^[a-z][a-z0-9+.-]*:/i.test(linkTarget)) continue;
|
|
const resolved = resolveRelative('README.md', linkTarget.split('#')[0]);
|
|
if (resolved !== null && !have.has(resolved)) {
|
|
findings.push({
|
|
level: 'ERROR',
|
|
code: 'BADGE-DEAD-LINK',
|
|
bucket: 'broken',
|
|
msg: `${m[2]} badge links to \`${linkTarget}\`, which does not resolve — a linked badge pointing nowhere is a claim dressed as evidence, worse than a static one because it looks verified.`,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
if (badgeCount > BADGE_INFLECTION) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'BADGE-COUNT',
|
|
bucket: 'weakening',
|
|
msg: `${badgeCount} badges — past the measured inflection of ${BADGE_INFLECTION}, where a badge row starts reading as clutter rather than as evidence (Trockman et al., ICSE 2018). Keep the ones a reader acts on.`,
|
|
});
|
|
}
|
|
if (findings.length === 0) findings.push({ level: 'OK', code: 'BADGES', msg: 'no static badge asserts an unverified run' });
|
|
return findings;
|
|
}
|
|
|
|
// Which language a README is written in is not a property of the code, so no
|
|
// remote can report it — it is a property of the READER, and the operator owns
|
|
// it. English is the default; a repo aimed only at a Norwegian readership is
|
|
// declared `nb` in the register and is then wrong in English, not right.
|
|
//
|
|
// Detection is a stopword-frequency comparison rather than a dependency: both
|
|
// word sets below are chosen to have NO member that is also a common word in
|
|
// the other language, which is why `at` and `for` (Norwegian and English both)
|
|
// are deliberately absent from each.
|
|
const STOPWORDS = {
|
|
nb: ['og', 'ikke', 'som', 'det', 'den', 'er', 'på', 'til', 'av', 'med', 'om',
|
|
'har', 'kan', 'skal', 'blir', 'være', 'etter', 'når', 'også', 'hvis',
|
|
'eller', 'men', 'fra', 'ved', 'mot', 'uten', 'hver', 'alle', 'andre',
|
|
'seg', 'dette', 'disse', 'mellom', 'gjennom', 'siden', 'fordi', 'derfor'],
|
|
en: ['the', 'and', 'of', 'to', 'in', 'is', 'that', 'with', 'this', 'are',
|
|
'be', 'from', 'by', 'as', 'an', 'or', 'not', 'you', 'your', 'we', 'our',
|
|
'it', 'on', 'which', 'when', 'what', 'how', 'its', 'they', 'their', 'has',
|
|
'can', 'will', 'should', 'each', 'between', 'because', 'therefore'],
|
|
};
|
|
|
|
// Below this there is not enough running prose for a frequency count to mean
|
|
// anything, and below the ratio the document is genuinely mixed. Both say so
|
|
// rather than guessing — a wrong verdict on language is worse than no verdict.
|
|
const LANG_MIN_HITS = 10;
|
|
const LANG_MIN_RATIO = 1.5;
|
|
|
|
function countStopwords(text, words) {
|
|
const lower = text.toLowerCase();
|
|
let n = 0;
|
|
for (const w of words) {
|
|
const m = lower.match(new RegExp(`(^|[^\\p{L}])${w}([^\\p{L}]|$)`, 'gu'));
|
|
if (m) n += m.length;
|
|
}
|
|
return n;
|
|
}
|
|
|
|
export function checkReadmeLanguage({ readme, name }, register) {
|
|
const declared = register?.locales?.[name] ?? 'en';
|
|
const other = declared === 'nb' ? 'en' : 'nb';
|
|
// Same discipline as the link and boilerplate checks: a Norwegian flag name
|
|
// in a shell example must not decide what language the DOCUMENT is in.
|
|
const prose = stripCode(String(readme ?? ''));
|
|
const hits = { nb: countStopwords(prose, STOPWORDS.nb), en: countStopwords(prose, STOPWORDS.en) };
|
|
|
|
// Not a SKIP. SKIP is for a check that could not RUN — the catalog was
|
|
// unreachable, the file unreadable. This one ran, saw everything, and found
|
|
// no prose to be in the wrong language, the same shape as "no licence claim
|
|
// to back". A thin README is a real problem, and it is checkFirstScreen's;
|
|
// routing it here would stop any terse repo from ever reaching OK.
|
|
if (hits[declared] + hits[other] < LANG_MIN_HITS) {
|
|
return [{
|
|
level: 'OK',
|
|
code: 'LANGUAGE',
|
|
msg: `no running prose to judge (${hits[declared] + hits[other]} marker words) — nothing claims a language`,
|
|
}];
|
|
}
|
|
if (hits[other] >= hits[declared] * LANG_MIN_RATIO) {
|
|
return [{
|
|
level: 'WARN',
|
|
code: 'README-LANGUAGE',
|
|
bucket: 'weakening',
|
|
msg: `README reads as \`${other}\` but this repo is declared \`${declared}\` (${hits[other]} vs ${hits[declared]} marker words). Who the reader is decides the language — fix the prose, or fix \`locales\` in the register.`,
|
|
}];
|
|
}
|
|
if (hits[declared] < hits[other] * LANG_MIN_RATIO) {
|
|
return [{
|
|
level: 'SKIP',
|
|
code: 'README-LANGUAGE-UNDECIDABLE',
|
|
msg: `README mixes languages too evenly to call (${hits.nb} nb vs ${hits.en} en) — declared \`${declared}\`, unverified`,
|
|
}];
|
|
}
|
|
return [{ level: 'OK', code: 'LANGUAGE', msg: `README reads as \`${declared}\`, as declared` }];
|
|
}
|
|
|
|
// Template text that was never filled in. A visible unfinished template costs
|
|
// more trust than the missing document would have.
|
|
const FIXME_RE = /FIXME/;
|
|
const BOILERPLATE = [
|
|
/your-project-name/i,
|
|
/\byour-org\b/i,
|
|
/\[INSERT[^\]]*\]/i,
|
|
/<your[- ][a-z]+>/i,
|
|
/TODO:\s*(fill|replace|update)/i,
|
|
/example@example\.(com|org)/i,
|
|
FIXME_RE,
|
|
];
|
|
|
|
// "TODO/FIXME" named together names the convention, not a live instance of
|
|
// one — reported by config-audit: a scanner whose job is finding these
|
|
// markers names its own detection target in its own docs, unquoted. A lone
|
|
// FIXME is still caught; only the paired reference is exempt.
|
|
const NAMES_THE_CONVENTION = /\bTODO\s*\/\s*FIXME\b|\bFIXME\s*\/\s*TODO\b/i;
|
|
|
|
export function checkBoilerplate({ files }) {
|
|
const findings = [];
|
|
for (const [path, text] of Object.entries(files ?? {})) {
|
|
// Same discipline as the link check: code spans and fenced blocks are where
|
|
// a document ABOUT placeholders keeps its examples.
|
|
stripCode(text).split('\n').forEach((line, i) => {
|
|
const namesTheConvention = NAMES_THE_CONVENTION.test(line);
|
|
for (const re of BOILERPLATE) {
|
|
if (re === FIXME_RE && namesTheConvention) continue;
|
|
if (re.test(line)) {
|
|
findings.push({
|
|
level: 'WARN',
|
|
code: 'BOILERPLATE',
|
|
bucket: 'weakening',
|
|
msg: `${path}:${i + 1} — unfilled template text: \`${line.trim().slice(0, 70)}\``,
|
|
});
|
|
return;
|
|
}
|
|
}
|
|
});
|
|
}
|
|
if (findings.length === 0) findings.push({ level: 'OK', code: 'BOILERPLATE', msg: 'no unfilled template text found' });
|
|
return findings;
|
|
}
|
|
|
|
// "LICENSE mentioned in the README, no file in the repo" is its own anti-signal:
|
|
// the claim is load-bearing for anyone deciding whether they may use this.
|
|
export function checkLicenseClaim({ readme, present }) {
|
|
const text = String(readme ?? '');
|
|
const claims = /\bLICEN[SC]E\b/i.test(text) || /\b(MIT|Apache|BSD|GPL)\b.{0,20}licen[sc]e/i.test(text);
|
|
const have = (present ?? []).some((f) => /^LICEN[SC]E(\.\w+)?$/i.test(f));
|
|
if (claims && !have) {
|
|
return [{
|
|
level: 'ERROR',
|
|
code: 'LICENSE-CLAIMED-ABSENT',
|
|
bucket: 'broken',
|
|
msg: 'README cites a licence but the repo has no LICENSE file — the claim a reader relies on to use this is unbacked',
|
|
}];
|
|
}
|
|
return [{ level: 'OK', code: 'LICENSE-CLAIM', msg: have ? 'LICENSE present' : 'no licence claim to back' }];
|
|
}
|
|
|
|
// Blank out fenced blocks and inline code spans, keeping line numbers intact.
|
|
// Documentation about regexes is full of strings that ARE markdown links to a
|
|
// naive scanner: `["']([A-Za-z0-9\-._]{16,64})["']` is `[...](...)` exactly.
|
|
// Running the first version against a real repo produced ~30 findings and every
|
|
// one of them was noise.
|
|
export function stripCode(text) {
|
|
let fenced = false;
|
|
let prevBlank = true;
|
|
let inIndented = false;
|
|
return String(text ?? '')
|
|
.split('\n')
|
|
.map((line) => {
|
|
if (/^\s*(```|~~~)/.test(line)) {
|
|
fenced = !fenced;
|
|
return '';
|
|
}
|
|
if (fenced) return '';
|
|
|
|
const blank = line.trim() === '';
|
|
const indented = /^(\s{4,}|\t)\S/.test(line);
|
|
// An indented line OPENS a code block only after a blank line — otherwise
|
|
// a nested list item would count, which made links inside nested bullets
|
|
// invisible. But once open, the block CONTINUES while lines stay indented;
|
|
// requiring a blank line before every line let everything after line 1
|
|
// leak back into scanning.
|
|
if (indented && (prevBlank || inIndented)) {
|
|
inIndented = true;
|
|
prevBlank = false;
|
|
return '';
|
|
}
|
|
if (!blank && !indented) inIndented = false;
|
|
prevBlank = blank;
|
|
if (inIndented && blank) return '';
|
|
return line.replace(/`[^`]*`/g, '');
|
|
})
|
|
.join('\n');
|
|
}
|
|
|
|
// A relative link resolves against the file it sits in, not against the repo
|
|
// root. Getting this wrong called two files missing that were right there next
|
|
// to the README linking them — and it would have done so in every nested doc.
|
|
// Returns null when the path escapes the repo, which is unresolvable from
|
|
// inside one repo rather than broken.
|
|
export function resolveRelative(fromFile, target) {
|
|
if (target.startsWith('/')) return null;
|
|
const baseParts = String(fromFile).split('/').slice(0, -1);
|
|
const out = [...baseParts];
|
|
for (const part of target.split('/')) {
|
|
if (part === '' || part === '.') continue;
|
|
if (part === '..') {
|
|
if (out.length === 0) return null;
|
|
out.pop();
|
|
} else {
|
|
out.push(part);
|
|
}
|
|
}
|
|
return out.join('/');
|
|
}
|
|
|
|
// Who the reader is decides the level. A dead link in a root document — README,
|
|
// CHANGELOG, SECURITY — is in the shop window and blocks a stranger. The same
|
|
// link three directories down is in a session plan, an agent working file, or a
|
|
// test fixture whose target is invalid ON PURPOSE. Measured across the org: 30
|
|
// of 43 findings sat below the root, and every one of them was an ERROR. A gate
|
|
// that is wrong that often gets switched off, so the level moves — and only the
|
|
// level. The finding is still reported, with its file and line.
|
|
function linkLevelFor(path) {
|
|
return String(path).includes('/') ? 'WARN' : 'ERROR';
|
|
}
|
|
|
|
// A file living in a test/fixture path is presumed to break its own links on
|
|
// purpose — `nav-golden-escape/bundle/index.md` escapes with `../../../../etc/passwd`
|
|
// deliberately, and the deep `..` pops the whole base path rather than resolving
|
|
// to `null`, so it read as a genuine WARN. Third tool in the org to hit this
|
|
// exact pattern, which is the signal that the check was at fault, not the repos.
|
|
// Only `*golden*` is a substring glob; the other three are exact segment names,
|
|
// so `testing/` or `fixturesque/` — real directories — are not swept in.
|
|
function isFixturePath(path) {
|
|
return String(path)
|
|
.toLowerCase()
|
|
.split('/')
|
|
.some((seg) => seg === 'test' || seg === 'tests' || seg === 'fixtures' || seg.includes('golden'));
|
|
}
|
|
|
|
// Relative file links only. Anchor resolution depends on per-renderer heading
|
|
// slug rules and is a rabbit hole; external URLs need the network. Both are
|
|
// deliberately out — a check that is sometimes wrong teaches people to ignore it.
|
|
export function checkInternalLinks({ files, present }) {
|
|
const have = new Set(present ?? []);
|
|
// `present` holds tracked FILES only, so a link to a directory — `[x](dir/)`
|
|
// — never has a member to match even when every file under it is tracked.
|
|
// Derive the directories a tracked file actually lives in from the same set.
|
|
const haveDirs = new Set();
|
|
for (const p of have) {
|
|
const parts = String(p).split('/');
|
|
for (let i = 1; i < parts.length; i++) haveDirs.add(parts.slice(0, i).join('/'));
|
|
}
|
|
const findings = [];
|
|
for (const [path, text] of Object.entries(files ?? {})) {
|
|
stripCode(text).split('\n').forEach((line, i) => {
|
|
for (const m of line.matchAll(/\[[^\]]*\]\(([^)\s]+)\)/g)) {
|
|
const target = m[1];
|
|
// Any scheme at all, not just http — `file:`, `vscode:`, `ftp:` are all
|
|
// somebody else's to resolve.
|
|
if (/^[a-z][a-z0-9+.-]*:/i.test(target) || /^[#<]/.test(target)) continue;
|
|
const clean = target.split('#')[0];
|
|
if (!clean) continue;
|
|
|
|
const resolved = resolveRelative(path, clean);
|
|
// A path that leaves the repo cannot be judged from inside it — a
|
|
// plugin README pointing up at its marketplace is the ordinary case.
|
|
if (resolved === null) {
|
|
findings.push({
|
|
level: 'SKIP',
|
|
code: 'LINK-OUTSIDE-REPO',
|
|
msg: `${path}:${i + 1} — \`${clean}\` points outside this repo; the gate sees one repo and cannot resolve it`,
|
|
});
|
|
continue;
|
|
}
|
|
if (!have.has(resolved) && !haveDirs.has(resolved)) {
|
|
if (isFixturePath(path)) {
|
|
findings.push({
|
|
level: 'SKIP',
|
|
code: 'LINK-INTERNAL-FIXTURE',
|
|
msg: `${path}:${i + 1} — link points at \`${clean}\` (${resolved}), which is not a tracked file; ${path} is a test/fixture path, so this is presumed intentional and not judged`,
|
|
});
|
|
} else {
|
|
findings.push({
|
|
level: linkLevelFor(path),
|
|
code: 'LINK-INTERNAL-MISSING',
|
|
bucket: 'broken',
|
|
msg: `${path}:${i + 1} — link points at \`${clean}\` (${resolved}), which is not a tracked file`,
|
|
});
|
|
}
|
|
}
|
|
}
|
|
});
|
|
}
|
|
// The OK line asserts that every link resolved. Keying it on ERROR alone would
|
|
// have printed it beside a pile of WARN findings saying the opposite.
|
|
if (!findings.some((f) => f.code === 'LINK-INTERNAL-MISSING')) {
|
|
findings.push({ level: 'OK', code: 'LINKS-INTERNAL', msg: 'every resolvable relative link resolves' });
|
|
}
|
|
return findings;
|
|
}
|
|
|
|
export function levelOf(findings) {
|
|
let worst = 'OK';
|
|
for (const f of findings ?? []) {
|
|
if (LEVELS.indexOf(f.level) > LEVELS.indexOf(worst)) worst = f.level;
|
|
}
|
|
return worst;
|
|
}
|
|
|
|
export function bucketsOf(findings) {
|
|
const out = { broken: 0, missing: 0, weakening: 0 };
|
|
for (const f of findings ?? []) {
|
|
if (f.bucket && out[f.bucket] !== undefined) out[f.bucket] += 1;
|
|
}
|
|
return out;
|
|
}
|
|
|
|
export function classifyRepo(
|
|
{ name, files, present, description, pluginVersion, readmeBadge, changelogTop, tags, catalogNames },
|
|
register,
|
|
) {
|
|
const klass = register.repos?.[name];
|
|
if (!klass) {
|
|
return {
|
|
name,
|
|
klass: null,
|
|
traits: [],
|
|
status: 'SKIP',
|
|
buckets: { broken: 0, missing: 0, weakening: 0 },
|
|
findings: [{
|
|
level: 'SKIP',
|
|
code: 'REPO-UNREGISTERED',
|
|
msg: `\`${name}\` is not in the register — class unknown, so no class-specific rule can be applied. Add it to register/repos.json (or run --refresh).`,
|
|
}],
|
|
};
|
|
}
|
|
|
|
const traits = register.traits?.[name] ?? [];
|
|
const readme = (files ?? {})['README.md'] ?? '';
|
|
const findings = [
|
|
...checkFirstScreen({ readme, name, description, klass }, register),
|
|
...checkInstallBlock({ readme, name, klass }, register),
|
|
...checkInstallTruth({ name, klass, catalogNames }),
|
|
...checkHeadings({ readme, klass, traits }, register),
|
|
...checkRequiredFiles({ present, klass, traits }, register),
|
|
...checkLinks({ files }, register),
|
|
...checkInternalLinks({ files, present }),
|
|
...checkLicenseClaim({ readme, present }),
|
|
...checkBadges({ readme, present }),
|
|
...checkReadmeLanguage({ readme, name }, register),
|
|
...checkBoilerplate({ files }),
|
|
...checkVersionConsistency({ pluginVersion, readmeBadge, changelogTop, tags }),
|
|
...checkDescription(description, register),
|
|
];
|
|
|
|
return { name, klass, traits, status: levelOf(findings), buckets: bucketsOf(findings), findings };
|
|
}
|
|
|
|
// ---------------------------------------------------------------- I/O shell
|
|
|
|
export function loadRegister(path = REGISTER_PATH) {
|
|
return JSON.parse(readFileSync(path, 'utf8'));
|
|
}
|
|
|
|
// TWO calls per invocation (corrected 2026-08-04 — this used to say ONE, from
|
|
// before fetchCatalogNames existed; a 13-repo shell loop trusting that count
|
|
// looked safe at 13 requests and was actually 26). Both anonymous — verified
|
|
// — so this works for any reader, not only for someone holding a token. A
|
|
// sweep across every repo does NOT belong here: it needs the org listing
|
|
// exactly once, not once per invocation, and "see all repos at once" is
|
|
// org-ops's job by this file's own header. What DOES belong here is not
|
|
// silently giving up on a transient 429 — that turns a rate-limit blip into
|
|
// a false SKIP, which this repo's own rule says is never a pass.
|
|
const defaultSleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
|
|
// Measured 2026-08-04 against the live forge: nginx never sends a
|
|
// `Retry-After` header on its 429s, so the exponential fallback below is the
|
|
// ONLY path that ever actually runs — the branch above it is dead in
|
|
// practice, kept only because a future proxy config could add the header.
|
|
// The 429 itself is a leaky-bucket burst limit, not a fixed-duration ban: a
|
|
// 20-25 request burst took up to ~15s to fully drain, and a 20s pause always
|
|
// cleared it. `retries: 3` (7s worst case) was tuned for a hard ban that
|
|
// turned out not to exist; `retries: 5` with `maxDelayMs: 8000` (23s worst
|
|
// case) covers the measured drain time without one attempt blocking minutes.
|
|
export async function fetchWithRetry(
|
|
url,
|
|
options,
|
|
{ fetchImpl = fetch, retries = 5, baseDelayMs = 1000, maxDelayMs = 8000, sleep = defaultSleep } = {},
|
|
) {
|
|
for (let attempt = 0; ; attempt += 1) {
|
|
const res = await fetchImpl(url, options);
|
|
if (res.status !== 429 || attempt >= retries) return res;
|
|
const retryAfter = Number(res.headers?.get?.('retry-after'));
|
|
const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
|
|
? retryAfter * 1000
|
|
: Math.min(baseDelayMs * 2 ** attempt, maxDelayMs);
|
|
await sleep(delayMs);
|
|
}
|
|
}
|
|
|
|
// The catalog's plugin list, read straight from the forge. One more call, and
|
|
// it is what turns "the install line is well-formed" into "the install line
|
|
// works". Null on any failure, which reads as SKIP rather than a pass.
|
|
async function fetchCatalogNames(register) {
|
|
const mkt = register.marketplace ?? {};
|
|
if (!mkt.name) return null;
|
|
const url = `${register.forge}/api/v1/repos/${register.org}/${mkt.name}/raw/.claude-plugin/marketplace.json`;
|
|
try {
|
|
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
|
|
if (!res.ok) return null;
|
|
const json = JSON.parse(await res.text());
|
|
return (json.plugins ?? []).map((p) => p.name).filter(Boolean);
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
async function fetchOrgListing(register) {
|
|
const url = `${register.forge}/api/v1/orgs/${register.org}/repos?limit=50`;
|
|
const res = await fetchWithRetry(url, { headers: { accept: 'application/json' } });
|
|
if (!res.ok) throw new Error(`org listing returned HTTP ${res.status}`);
|
|
return res.json();
|
|
}
|
|
|
|
function gitFiles(dir) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'ls-files'], { encoding: 'utf8' })
|
|
.split('\n')
|
|
.map((s) => s.trim())
|
|
.filter(Boolean);
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
// The remote is ground truth for what a repo is CALLED; the directory is only
|
|
// where it happens to sit. `catalog/` is the working directory of the repo named
|
|
// `ktg-plugin-marketplace`, and deriving the name from the basename left it
|
|
// REPO-UNREGISTERED — zero checks run against the one repo the catalog rule
|
|
// depends on. Handles the scp form too: the forge's clone button hands it out.
|
|
export function parseRepoNameFromRemote(url) {
|
|
const raw = String(url ?? '').trim();
|
|
if (!raw) return null;
|
|
const path = raw.includes('://') ? raw.split('://')[1] : raw;
|
|
const segments = path.replace(/\/+$/, '').split(/[/:]/).filter(Boolean);
|
|
// A bare host is not a repository. Without this, `https://the-forge/` parsed
|
|
// as a repo named after the host and every class rule matched the wrong thing.
|
|
if (segments.length < 2) return null;
|
|
const name = segments.pop().replace(/\.git$/, '');
|
|
return name || null;
|
|
}
|
|
|
|
function repoNameFrom(dir) {
|
|
try {
|
|
const remote = execFileSync('git', ['-C', dir, 'remote', 'get-url', 'origin'], {
|
|
encoding: 'utf8',
|
|
stdio: ['ignore', 'pipe', 'ignore'],
|
|
});
|
|
const fromRemote = parseRepoNameFromRemote(remote);
|
|
if (fromRemote) return fromRemote;
|
|
} catch { /* no remote yet — a repo before its first push is the ordinary case */ }
|
|
try {
|
|
return basename(execFileSync('git', ['-C', dir, 'rev-parse', '--show-toplevel'], { encoding: 'utf8' }).trim());
|
|
} catch {
|
|
return basename(dir);
|
|
}
|
|
}
|
|
|
|
// The version the package itself claims, from whichever manifest this class uses.
|
|
function readPackageVersion(dir) {
|
|
for (const p of ['.claude-plugin/plugin.json', 'package.json', 'pyproject.toml']) {
|
|
const full = join(dir, p);
|
|
if (!existsSync(full)) continue;
|
|
try {
|
|
const raw = readFileSync(full, 'utf8');
|
|
if (p.endsWith('.json')) {
|
|
const v = JSON.parse(raw).version;
|
|
if (v) return String(v);
|
|
} else {
|
|
const m = /^\s*version\s*=\s*["']([^"']+)["']/m.exec(raw);
|
|
if (m) return m[1];
|
|
}
|
|
} catch { /* unparseable — try the next one */ }
|
|
}
|
|
return null;
|
|
}
|
|
|
|
export function extractBadgeVersion(readmeText) {
|
|
const m = /badge\/version-(\d+\.\d+\.\d+)/.exec(readmeText || '');
|
|
return m ? m[1] : null;
|
|
}
|
|
|
|
// Newest released version in the CHANGELOG. `## [Unreleased]` is skipped by
|
|
// design — it is not a claim that anything shipped.
|
|
export function extractChangelogTop(changelogText) {
|
|
for (const line of String(changelogText || '').split('\n')) {
|
|
// The suffix class stops at `]`, whitespace or end of string, so a
|
|
// pre-release token (PEP 440 `a2`, semver `-beta.1`) is kept without
|
|
// reaching into a trailing `] — DATE`. Reported by llm-ingestion-okf:
|
|
// truncating this to X.Y.Z made VERSION-CHANGELOG disagree with
|
|
// VERSION-TAG, which compares the untruncated tag and does not have
|
|
// this problem — a repo on a pre-release could never reach 0 ERROR.
|
|
const m = /^##\s*\[?v?(\d+\.\d+\.\d+[0-9A-Za-z.+-]*)\]?/.exec(line.trim());
|
|
if (m) return m[1];
|
|
}
|
|
return null;
|
|
}
|
|
|
|
function gitTags(dir) {
|
|
try {
|
|
return execFileSync('git', ['-C', dir, 'tag', '--list', 'v*'], { encoding: 'utf8' })
|
|
.split('\n').map((s) => s.trim()).filter(Boolean);
|
|
} catch {
|
|
return [];
|
|
}
|
|
}
|
|
|
|
export function inspectRepo(dir, name, register, description, catalogNames = null) {
|
|
const tracked = gitFiles(dir);
|
|
const present = (tracked ?? []).filter((f) => existsSync(join(dir, f)));
|
|
|
|
// Link scanning covers every tracked Markdown file — a dead reference in a
|
|
// doc is as broken as one in the README.
|
|
const files = {};
|
|
for (const f of (tracked ?? []).filter((p) => p.endsWith('.md'))) {
|
|
try { files[f] = readFileSync(join(dir, f), 'utf8'); } catch { /* unreadable — skip */ }
|
|
}
|
|
if (!files['README.md'] && existsSync(join(dir, 'README.md'))) {
|
|
files['README.md'] = readFileSync(join(dir, 'README.md'), 'utf8');
|
|
}
|
|
|
|
const readme = files['README.md'] ?? '';
|
|
let changelog = null;
|
|
try { changelog = readFileSync(join(dir, 'CHANGELOG.md'), 'utf8'); } catch { /* absent */ }
|
|
|
|
return classifyRepo({
|
|
name,
|
|
files,
|
|
present,
|
|
description,
|
|
pluginVersion: readPackageVersion(dir),
|
|
readmeBadge: extractBadgeVersion(readme),
|
|
changelogTop: changelog === null ? null : extractChangelogTop(changelog),
|
|
tags: gitTags(dir),
|
|
catalogNames,
|
|
}, register);
|
|
}
|
|
|
|
// Grouped by bucket, because that is the order the findings actually get acted
|
|
// on: what blocks a stranger today, then what is absent, then what merely reads
|
|
// badly. Severity within a bucket is secondary to that.
|
|
const BUCKET_TITLE = {
|
|
broken: 'BROKEN NOW — a stranger is blocked or misled',
|
|
missing: 'MISSING — an expected artefact is absent',
|
|
weakening: 'WEAKENING — present, but it reads as amateur',
|
|
};
|
|
|
|
// A stale plugin cache once served 0.1.1 while 0.2.0 was installed and
|
|
// pinned, silently — the output looked like a clean pass, because nothing
|
|
// said which engine had run. This is the fix: name the version so a wrong
|
|
// engine is visible, not just correctable in hindsight.
|
|
const MARK = { OK: '✓', WARN: '!', ERROR: '✗', SKIP: '·' };
|
|
|
|
export function headerLine(result, engineVersion, engineCommit = null) {
|
|
const klass = result.klass ? ` [${result.klass}]` : '';
|
|
const traits = result.traits?.length ? ` {${result.traits.join(', ')}}` : '';
|
|
const sha = engineCommit ? ` @${String(engineCommit).slice(0, 7)}` : '';
|
|
return `${MARK[result.status]} ${result.name}${klass}${traits} — ${result.status} (repo-standard v${engineVersion}${sha})`;
|
|
}
|
|
|
|
// `engineCommit` is always present, null when underivable: an ABSENT key means
|
|
// an older engine, an explicit null means this engine ran and had no HEAD to
|
|
// read. A consumer sorting raw files by stamp needs those to be different.
|
|
export function withEngineVersion(result, engineVersion, engineCommit = null) {
|
|
return { ...result, engineVersion, engineCommit };
|
|
}
|
|
|
|
function render(result, engineVersion, engineCommit) {
|
|
const mark = MARK;
|
|
console.log(`\n${headerLine(result, engineVersion, engineCommit)}`);
|
|
|
|
for (const bucket of BUCKETS) {
|
|
const inBucket = result.findings.filter((f) => f.bucket === bucket);
|
|
if (!inBucket.length) continue;
|
|
console.log(`\n ${BUCKET_TITLE[bucket]}`);
|
|
for (const f of inBucket) console.log(` ${mark[f.level]} ${f.level} ${f.code}: ${f.msg}`);
|
|
}
|
|
|
|
const skipped = result.findings.filter((f) => f.level === 'SKIP');
|
|
if (skipped.length) {
|
|
console.log('\n NOT CHECKED — these are not passes');
|
|
for (const f of skipped) console.log(` ${mark.SKIP} ${f.code}: ${f.msg}`);
|
|
}
|
|
|
|
const okCount = result.findings.filter((f) => f.level === 'OK').length;
|
|
console.log(`\n ${mark.OK} ${okCount} check(s) passed`);
|
|
}
|
|
|
|
async function refresh(register) {
|
|
const live = await fetchOrgListing(register);
|
|
const liveNames = new Set(live.map((r) => r.name));
|
|
const known = new Set(Object.keys(register.repos ?? {}));
|
|
|
|
const added = [...liveNames].filter((n) => !known.has(n)).sort();
|
|
const gone = [...known].filter((n) => !liveNames.has(n)).sort();
|
|
|
|
console.log(`register: ${known.size} repos · forge: ${liveNames.size} repos`);
|
|
if (added.length) console.log(`\n on the forge, not in the register (add with a class):\n ${added.join('\n ')}`);
|
|
if (gone.length) console.log(`\n in the register, not on the forge:\n ${gone.join('\n ')}`);
|
|
if (!added.length && !gone.length) console.log('\n ✓ register matches the forge');
|
|
return added.length + gone.length === 0 ? 0 : 1;
|
|
}
|
|
|
|
async function main(argv) {
|
|
const arg = (flag, fallback = null) => {
|
|
const i = argv.indexOf(flag);
|
|
return i === -1 ? fallback : argv[i + 1];
|
|
};
|
|
const register = loadRegister();
|
|
|
|
if (argv.includes('--refresh')) {
|
|
process.exit(await refresh(register));
|
|
}
|
|
|
|
const dir = arg('--dir', process.cwd());
|
|
const name = arg('--name', repoNameFrom(dir));
|
|
|
|
let description = null;
|
|
let catalogNames = null;
|
|
if (!argv.includes('--offline')) {
|
|
catalogNames = await fetchCatalogNames(register);
|
|
try {
|
|
const listing = await fetchOrgListing(register);
|
|
const row = listing.find((r) => r.name === name);
|
|
description = row ? (row.description ?? '') : null;
|
|
} catch {
|
|
// Unreachable forge leaves description null, which reads as SKIP — never
|
|
// as a pass. A check that could not run says so.
|
|
}
|
|
}
|
|
|
|
const result = inspectRepo(dir, name, register, description, catalogNames);
|
|
const engineVersion = readEngineVersion();
|
|
const engineCommit = readEngineCommit();
|
|
|
|
if (argv.includes('--json')) {
|
|
console.log(JSON.stringify(withEngineVersion(result, engineVersion, engineCommit), null, 2));
|
|
} else {
|
|
render(result, engineVersion, engineCommit);
|
|
}
|
|
process.exit(result.status === 'ERROR' ? 1 : 0);
|
|
}
|
|
|
|
if (process.argv[1] && process.argv[1].endsWith('repo-standard-check.mjs')) {
|
|
main(process.argv.slice(2));
|
|
}
|