fix: three more measured false positives, all reported the same day v0.1.2 shipped
- extractChangelogTop truncated PEP 440 pre-release versions (0.5.0a2 -> 0.5.0), so VERSION-CHANGELOG disagreed with VERSION-TAG over a version everything already agreed on. Reported by llm-ingestion-okf with a repro. - BADGE-STATIC-CLAIM treated a bare `status` badge as a run claim, same as tests/build/CI. A self-declared maturity label asserts no run, same class as version/licence/platform. Reported by llm-ingestion-pipeline-security. - BOILERPLATE flagged FIXME when a scanner's own docs named the TODO/FIXME convention rather than an instance of it. Reported by config-audit. 92 tests green, up from 86. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uwwfdmrfnp7FuGQ4z25RKH
This commit is contained in:
parent
3c2a535297
commit
5eba10acdd
7 changed files with 110 additions and 7 deletions
|
|
@ -1,6 +1,6 @@
|
||||||
{
|
{
|
||||||
"name": "repo-standard",
|
"name": "repo-standard",
|
||||||
"version": "0.1.2",
|
"version": "0.1.3",
|
||||||
"description": "Per-repo gate for the open/ presentation standard: README first screen, install block, files required by the repo's class, and dead repo references.",
|
"description": "Per-repo gate for the open/ presentation standard: README first screen, install block, files required by the repo's class, and dead repo references.",
|
||||||
"author": {
|
"author": {
|
||||||
"name": "Kjell Tore Guttormsen"
|
"name": "Kjell Tore Guttormsen"
|
||||||
|
|
|
||||||
29
CHANGELOG.md
29
CHANGELOG.md
|
|
@ -4,6 +4,34 @@ All notable changes to this project are documented here.
|
||||||
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
|
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
|
||||||
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
versioning is [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [0.1.3] — 2026-08-03
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
Three more measured false positives, all reported via coord the same day
|
||||||
|
0.1.2 shipped, from repos running the gate for the first time.
|
||||||
|
|
||||||
|
- `extractChangelogTop` truncated a PEP 440 pre-release (`0.5.0a2`) to `0.5.0`,
|
||||||
|
so `VERSION-CHANGELOG` disagreed with `VERSION-TAG` — which compares the
|
||||||
|
untruncated tag and does not have this problem — over a version the manifest
|
||||||
|
and the tag already agreed on. Reported by `llm-ingestion-okf` with a
|
||||||
|
repro. An affected repo could reach 0 ERROR only by lying about its version;
|
||||||
|
now the pre-release suffix is kept.
|
||||||
|
- `BADGE-STATIC-CLAIM` treated a bare `status` badge (`status-alpha`,
|
||||||
|
`status-experimental`) as a run claim, the same as `tests-34-passing`.
|
||||||
|
A self-declared maturity label asserts no run — the same class as version,
|
||||||
|
licence and platform, already exempt. Reported by
|
||||||
|
`llm-ingestion-pipeline-security`. `build`/`ci`/`passing` still catch the
|
||||||
|
run-asserting compounds ("build status", "CI status"); only the bare word
|
||||||
|
is dropped.
|
||||||
|
- `BOILERPLATE` flagged a bare `FIXME` even when it named the *convention*
|
||||||
|
rather than an instance of it — a scanner's own docs describing what it
|
||||||
|
detects ("Flag TODO/FIXME markers…", a table row listing "TODO/FIXME
|
||||||
|
comments" as a finding type) read the same as a forgotten marker. Reported
|
||||||
|
by `config-audit`, whose product literally is a marker scanner. "TODO/FIXME"
|
||||||
|
named together is now read as the convention; a lone `FIXME` is still
|
||||||
|
caught.
|
||||||
|
|
||||||
## [0.1.2] — 2026-08-03
|
## [0.1.2] — 2026-08-03
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
@ -133,6 +161,7 @@ First release. Covers the checks that a single repository can answer on its own.
|
||||||
- No hook ships in this release. A blocking gate has to be precise enough not to
|
- No hook ships in this release. A blocking gate has to be precise enough not to
|
||||||
fail a correct repository first.
|
fail a correct repository first.
|
||||||
|
|
||||||
|
[0.1.3]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.3
|
||||||
[0.1.2]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.2
|
[0.1.2]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.2
|
||||||
[0.1.1]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.1
|
[0.1.1]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.1
|
||||||
[0.1.0]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.0
|
[0.1.0]: https://git.fromaitochitta.com/open/repo-standard/src/tag/v0.1.0
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,7 @@ checks that surface in one repository and reports what it finds.
|
||||||
|
|
||||||
*AI-generated: all code produced by Claude Code through dialog-driven development.*
|
*AI-generated: all code produced by Claude Code through dialog-driven development.*
|
||||||
|
|
||||||

|

|
||||||

|

|
||||||

|

|
||||||

|

|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
{
|
{
|
||||||
"name": "repo-standard",
|
"name": "repo-standard",
|
||||||
"version": "0.1.2",
|
"version": "0.1.3",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"engines": {
|
"engines": {
|
||||||
|
|
|
||||||
|
|
@ -427,7 +427,12 @@ export function checkVersionConsistency({ pluginVersion, readmeBadge, changelogT
|
||||||
|
|
||||||
// A static image asserting "tests: 642 passing" is a claim dressed as evidence.
|
// A static image asserting "tests: 642 passing" is a claim dressed as evidence.
|
||||||
// Version, licence and platform badges assert no run, so they are fine static.
|
// Version, licence and platform badges assert no run, so they are fine static.
|
||||||
const CLAIM_BADGE = /(tests?|build|ci|coverage|passing|status)/i;
|
// Bare `status` used to be in this list and caught a self-declared maturity
|
||||||
|
// badge ("status: alpha") as if it were a run claim — reported by
|
||||||
|
// llm-ingestion-pipeline-security. `build`/`ci`/`passing` already catch the
|
||||||
|
// run-asserting compounds ("build status", "CI status"), so dropping the bare
|
||||||
|
// word loses no real detection.
|
||||||
|
const CLAIM_BADGE = /(tests?|build|ci|coverage|passing)/i;
|
||||||
|
|
||||||
export function checkBadges({ readme }) {
|
export function checkBadges({ readme }) {
|
||||||
const findings = [];
|
const findings = [];
|
||||||
|
|
@ -453,6 +458,7 @@ export function checkBadges({ readme }) {
|
||||||
|
|
||||||
// Template text that was never filled in. A visible unfinished template costs
|
// Template text that was never filled in. A visible unfinished template costs
|
||||||
// more trust than the missing document would have.
|
// more trust than the missing document would have.
|
||||||
|
const FIXME_RE = /FIXME/;
|
||||||
const BOILERPLATE = [
|
const BOILERPLATE = [
|
||||||
/your-project-name/i,
|
/your-project-name/i,
|
||||||
/\byour-org\b/i,
|
/\byour-org\b/i,
|
||||||
|
|
@ -460,16 +466,24 @@ const BOILERPLATE = [
|
||||||
/<your[- ][a-z]+>/i,
|
/<your[- ][a-z]+>/i,
|
||||||
/TODO:\s*(fill|replace|update)/i,
|
/TODO:\s*(fill|replace|update)/i,
|
||||||
/example@example\.(com|org)/i,
|
/example@example\.(com|org)/i,
|
||||||
/FIXME/,
|
FIXME_RE,
|
||||||
];
|
];
|
||||||
|
|
||||||
|
// "TODO/FIXME" named together names the convention, not a live instance of
|
||||||
|
// one — reported by config-audit: a scanner whose job is finding these
|
||||||
|
// markers names its own detection target in its own docs, unquoted. A lone
|
||||||
|
// FIXME is still caught; only the paired reference is exempt.
|
||||||
|
const NAMES_THE_CONVENTION = /\bTODO\s*\/\s*FIXME\b|\bFIXME\s*\/\s*TODO\b/i;
|
||||||
|
|
||||||
export function checkBoilerplate({ files }) {
|
export function checkBoilerplate({ files }) {
|
||||||
const findings = [];
|
const findings = [];
|
||||||
for (const [path, text] of Object.entries(files ?? {})) {
|
for (const [path, text] of Object.entries(files ?? {})) {
|
||||||
// Same discipline as the link check: code spans and fenced blocks are where
|
// Same discipline as the link check: code spans and fenced blocks are where
|
||||||
// a document ABOUT placeholders keeps its examples.
|
// a document ABOUT placeholders keeps its examples.
|
||||||
stripCode(text).split('\n').forEach((line, i) => {
|
stripCode(text).split('\n').forEach((line, i) => {
|
||||||
|
const namesTheConvention = NAMES_THE_CONVENTION.test(line);
|
||||||
for (const re of BOILERPLATE) {
|
for (const re of BOILERPLATE) {
|
||||||
|
if (re === FIXME_RE && namesTheConvention) continue;
|
||||||
if (re.test(line)) {
|
if (re.test(line)) {
|
||||||
findings.push({
|
findings.push({
|
||||||
level: 'WARN',
|
level: 'WARN',
|
||||||
|
|
@ -788,7 +802,13 @@ export function extractBadgeVersion(readmeText) {
|
||||||
// design — it is not a claim that anything shipped.
|
// design — it is not a claim that anything shipped.
|
||||||
export function extractChangelogTop(changelogText) {
|
export function extractChangelogTop(changelogText) {
|
||||||
for (const line of String(changelogText || '').split('\n')) {
|
for (const line of String(changelogText || '').split('\n')) {
|
||||||
const m = /^##\s*\[?v?(\d+\.\d+\.\d+)\]?/.exec(line.trim());
|
// The suffix class stops at `]`, whitespace or end of string, so a
|
||||||
|
// pre-release token (PEP 440 `a2`, semver `-beta.1`) is kept without
|
||||||
|
// reaching into a trailing `] — DATE`. Reported by llm-ingestion-okf:
|
||||||
|
// truncating this to X.Y.Z made VERSION-CHANGELOG disagree with
|
||||||
|
// VERSION-TAG, which compares the untruncated tag and does not have
|
||||||
|
// this problem — a repo on a pre-release could never reach 0 ERROR.
|
||||||
|
const m = /^##\s*\[?v?(\d+\.\d+\.\d+[0-9A-Za-z.+-]*)\]?/.exec(line.trim());
|
||||||
if (m) return m[1];
|
if (m) return m[1];
|
||||||
}
|
}
|
||||||
return null;
|
return null;
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,7 @@ import {
|
||||||
classifyRepo,
|
classifyRepo,
|
||||||
levelOf,
|
levelOf,
|
||||||
parseRepoNameFromRemote,
|
parseRepoNameFromRemote,
|
||||||
|
extractChangelogTop,
|
||||||
} from './repo-standard-check.mjs';
|
} from './repo-standard-check.mjs';
|
||||||
|
|
||||||
const REGISTER = {
|
const REGISTER = {
|
||||||
|
|
@ -490,6 +491,22 @@ test('a released version with no matching tag is an ERROR', () => {
|
||||||
assert.equal(f.some((x) => x.level === 'ERROR' && x.code === 'VERSION-TAG'), true);
|
assert.equal(f.some((x) => x.level === 'ERROR' && x.code === 'VERSION-TAG'), true);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Reported by llm-ingestion-okf (coord, 2026-08-03): a PEP 440 pre-release
|
||||||
|
// (`0.5.0a2`) matches the manifest and the tag exactly, but the CHANGELOG
|
||||||
|
// extractor truncated it to `0.5.0` — the only way to reach 0 ERROR would have
|
||||||
|
// been to announce a release that never happened.
|
||||||
|
test('extractChangelogTop keeps a PEP 440 pre-release suffix, not just X.Y.Z', () => {
|
||||||
|
assert.equal(extractChangelogTop('## [0.5.0a2] - 2026-07-31'), '0.5.0a2');
|
||||||
|
assert.equal(extractChangelogTop('## [0.1.1] — 2026-08-03'), '0.1.1');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a PEP 440 pre-release version agrees with its own CHANGELOG heading', () => {
|
||||||
|
const f = checkVersionConsistency({
|
||||||
|
pluginVersion: '0.5.0a2', readmeBadge: '0.5.0a2', changelogTop: extractChangelogTop('## [0.5.0a2] - 2026-07-31'), tags: ['v0.5.0a2'],
|
||||||
|
});
|
||||||
|
assert.equal(f.some((x) => x.level === 'ERROR'), false);
|
||||||
|
});
|
||||||
|
|
||||||
// -------------------------------------------------------- required headings
|
// -------------------------------------------------------- required headings
|
||||||
|
|
||||||
test('required headings are per class — Non-goals is required, not optional', () => {
|
test('required headings are per class — Non-goals is required, not optional', () => {
|
||||||
|
|
@ -530,6 +547,22 @@ test('a build badge that links to a real run is fine', () => {
|
||||||
assert.equal(checkBadges({ readme }).filter((f) => f.level !== 'OK').length, 0);
|
assert.equal(checkBadges({ readme }).filter((f) => f.level !== 'OK').length, 0);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Reported by llm-ingestion-pipeline-security (coord, 2026-08-03): a bare
|
||||||
|
// `status` badge is a self-declared maturity label ("alpha", "experimental"),
|
||||||
|
// the same class as version/licence/platform, which already assert no run —
|
||||||
|
// not a run claim like "build status" or "CI status".
|
||||||
|
test('a static maturity-status badge asserts no run, unlike build/CI status', () => {
|
||||||
|
const f = checkBadges({ readme: '' });
|
||||||
|
assert.equal(f.some((x) => x.code === 'BADGE-STATIC-CLAIM'), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a build- or CI-status badge is still caught — only bare "status" was too wide', () => {
|
||||||
|
const build = checkBadges({ readme: '' });
|
||||||
|
assert.equal(build.some((x) => x.code === 'BADGE-STATIC-CLAIM'), true);
|
||||||
|
const ci = checkBadges({ readme: '' });
|
||||||
|
assert.equal(ci.some((x) => x.code === 'BADGE-STATIC-CLAIM'), true);
|
||||||
|
});
|
||||||
|
|
||||||
// -------------------------------------------------------------- boilerplate
|
// -------------------------------------------------------------- boilerplate
|
||||||
|
|
||||||
test('unfinished template text is a finding', () => {
|
test('unfinished template text is a finding', () => {
|
||||||
|
|
@ -540,6 +573,27 @@ test('unfinished template text is a finding', () => {
|
||||||
assert.equal(g.some((x) => x.code === 'BOILERPLATE'), true);
|
assert.equal(g.some((x) => x.code === 'BOILERPLATE'), true);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('a lone FIXME with no TODO alongside is still caught', () => {
|
||||||
|
const f = checkBoilerplate({ files: { 'NOTES.md': 'FIXME: handle the null case here.' } });
|
||||||
|
assert.equal(f.some((x) => x.level === 'WARN'), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Reported by config-audit (coord, 2026-08-03): a scanner whose JOB is to find
|
||||||
|
// TODO/FIXME markers in OTHER repos names its own detection target in its own
|
||||||
|
// docs — prose and a table row, neither wrapped in backticks. "TODO/FIXME"
|
||||||
|
// named together is the convention itself, not a forgotten instance of one.
|
||||||
|
test('"TODO/FIXME" named together as the convention is not a live marker', () => {
|
||||||
|
const listItem = checkBoilerplate({
|
||||||
|
files: { 'agents/scanner-agent.md': "- Flag TODO/FIXME markers that haven't been addressed" },
|
||||||
|
});
|
||||||
|
assert.equal(listItem.some((x) => x.level === 'WARN'), false);
|
||||||
|
|
||||||
|
const tableRow = checkBoilerplate({
|
||||||
|
files: { 'knowledge/anti-patterns.md': '| 5 | TODO/FIXME comments in CLAUDE.md | CA-CML-005 | low |' },
|
||||||
|
});
|
||||||
|
assert.equal(tableRow.some((x) => x.level === 'WARN'), false);
|
||||||
|
});
|
||||||
|
|
||||||
test('ordinary prose is not boilerplate', () => {
|
test('ordinary prose is not boilerplate', () => {
|
||||||
const f = checkBoilerplate({ files: { 'README.md': 'This project solves a real problem.' } });
|
const f = checkBoilerplate({ files: { 'README.md': 'This project solves a real problem.' } });
|
||||||
assert.equal(f.filter((x) => x.level !== 'OK').length, 0);
|
assert.equal(f.filter((x) => x.level !== 'OK').length, 0);
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ description: >-
|
||||||
"fiks install-blokka", "finn døde repo-referanser", "gjør repoet presentabelt".
|
"fiks install-blokka", "finn døde repo-referanser", "gjør repoet presentabelt".
|
||||||
Trigger when someone is about to release, publish, or hand over a repository
|
Trigger when someone is about to release, publish, or hand over a repository
|
||||||
and wants its public surface to hold up.
|
and wants its public surface to hold up.
|
||||||
version: "0.1.2"
|
version: "0.1.3"
|
||||||
---
|
---
|
||||||
|
|
||||||
# repo-standard — the per-repo gate
|
# repo-standard — the per-repo gate
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue