feat(gate): buckets, traits, and the checks the brief calls load-bearing
Measured this build against a documentation brief for public repos. The five original checks covered roughly one of its ten sections, so this adds what a single repo can answer on its own. New: required README headings per class (Non-goals is the cheapest trust-builder there is), in-repo version consistency across manifest / badge / CHANGELOG / tag, badge honesty, boilerplate, licence-claim, and relative links. Findings now carry a BUCKET beside the level - broken / missing / weakening - and output is grouped by it, because that is the order the work gets done in. Traits are a second axis beside class: class is structural and readable off the catalog, a trait says what the code does. `security` attaches SECURITY.md and a Known limitations section. The two names carrying it are proposed, not measured - that list is the operator's. Solo-maintained settles a category: CONTRIBUTING, CODE_OF_CONDUCT and MAINTAINERS are required by no class. Consumer-facing documents are untouched by that; SECURITY.md exists for the stranger who finds a hole. Three bugs found by running against llm-security, not by reading: - ~30 link findings, all noise. Regexes inside code spans are `[...](...)` to a naive scanner. Strip code first. - `file:` and other schemes were treated as repo-relative paths. - Relative links were resolved against the repo root instead of the file they sit in, calling two files missing that sat next to the README linking them. Same fix applied to the boilerplate check: a document ABOUT placeholder detection was tripping the placeholder detector. Also removed this repo's own static tests badge. There is no CI - the forge has zero Actions runners registered - so it could never become real, and it is the exact anti-pattern the gate now flags. 67 tests. Against llm-security every remaining finding is real and matches the census's independent hand-measurement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WYJ3FHLtVgzFXMZ6UF598h
This commit is contained in:
parent
9a72cefefa
commit
720850a9ad
7 changed files with 915 additions and 39 deletions
21
CLAUDE.md
21
CLAUDE.md
|
|
@ -29,6 +29,25 @@ would recreate, in data, exactly the drift this plugin exists to remove.
|
|||
- **It records, it does not fix.** Findings first, remediation afterwards.
|
||||
Patching while measuring is how the inconsistency it detects was produced.
|
||||
- **`SKIP` is never a pass.** A check that could not run says so and names why.
|
||||
- **When a check fires many times in one repo, suspect the CHECK.** The first
|
||||
link pass produced ~30 findings against `llm-security` and all were noise —
|
||||
regexes in code spans, `file:` URLs, relative paths resolved against the wrong
|
||||
directory. Strip code before scanning text; resolve links against the file
|
||||
they sit in. A gate that is wrong this often gets switched off.
|
||||
- **Two axes on every finding.** Level (`ERROR`/`WARN`/`SKIP`/`OK`) and bucket
|
||||
(`broken`/`missing`/`weakening`). They are independent — a `weakening` finding
|
||||
can be an `ERROR`.
|
||||
- **Class is structural, traits are judgement.** Class is read off the catalog
|
||||
and the remotes. A trait (`security`) says what the code does, which no remote
|
||||
knows, and the operator owns that list.
|
||||
- **Who the reader is decides what is required.** Contributor-facing documents
|
||||
are required by no class — solo-maintained, and the published stance says so.
|
||||
Consumer-facing ones (`SECURITY.md`, `LICENSE`, non-goals, limitations) are
|
||||
untouched by that. This is not a rule against having the others.
|
||||
- **No CI badge, because there is no CI.** The forge has zero Actions runners
|
||||
registered (measured). The substitute is one command from a clean clone, said
|
||||
plainly. A static badge asserting a run is the anti-pattern this gate flags —
|
||||
and an early draft of this README carried one.
|
||||
- **Three outcomes on references.** "No match" and "match on a known non-repo"
|
||||
must stay distinct findings. Collapsing them hides real loss inside correct
|
||||
text — the exact defect class this gate exists to catch.
|
||||
|
|
@ -44,7 +63,7 @@ would recreate, in data, exactly the drift this plugin exists to remove.
|
|||
## Commands
|
||||
|
||||
```bash
|
||||
npm test # 32 tests
|
||||
npm test # 67 tests
|
||||
node scripts/repo-standard-check.mjs --dir "$PWD" # gate one repo
|
||||
node scripts/repo-standard-check.mjs --offline # no network call
|
||||
node scripts/repo-standard-check.mjs --json # machine output
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue