Two defects that only org-wide measurement exposes. Both were found by running
the gate against seventeen repositories, not by reading the code.
Link level follows the reader. 30 of 43 LINK-INTERNAL-MISSING findings sat in
`shared/`, `docs/plan/` and `.claude/` -- session plans, agent working files,
and path-traversal fixtures whose targets are invalid on purpose. All 30 were
ERRORs. portfolio-optimiser scored 19 ERRORs and 15 of them were noise. A gate
that wrong gets switched off, so root documents stay ERROR and everything below
becomes WARN. Only the level moves; the finding keeps its file and line.
The OK line had to move with it: it asserted "every resolvable relative link
resolves" whenever no ERROR was present, which would have printed it beside a
pile of WARNs saying the opposite.
The repo name comes from `git remote get-url origin`. `catalog/` is the working
directory of `ktg-plugin-marketplace`; the basename left it REPO-UNREGISTERED
with zero checks run -- against the one repo every catalog rule depends on. The
scp form is handled because the forge's clone button hands it out, and a bare
host is not a repo name (that test caught a real bug: `https://host/` parsed as
a repo named after the host).
Re-measured: portfolio-optimiser 19 ERROR -> 4, all four genuine. catalog now
reports as `ktg-plugin-marketplace [catalog]` and runs its full rule set.
Still v0.1.0: nothing is pushed, no tag exists, no consumer has seen it.
77 -> 84 tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EeNsGsWukggfmLQ926WZPx
Install truth is the brief's first control and the gate only checked
syntax. Now: the marketplace URL must be the real one (offline, from the
register), and the plugin must actually be pinned in the catalog (one
call, SKIP if unreachable). A well-formed `claude plugin install x@mkt`
fails silently when x was never pinned.
This makes the gate block ITSELF until publication finishes - the run
against this repo now has exactly one ERROR, and it is true: repo-standard
is not in the catalog yet. That is the post-publish acceptance test,
enforced mechanically instead of remembered.
Badge honesty no longer keys on img.shields.io. A self-hosted SVG asserts
the same unverified thing, and the README claimed the general rule while
the code checked one host.
Two stripCode bugs, both silent false passes:
- 4-space indent treated as code unconditionally made links inside nested
list items invisible. Fixed by requiring a blank line to OPEN a block.
- That fix alone ended the block after line 1, so multi-line indented
templates leaked back into scanning. Caught by the gate on this repo's
own SKILL.md, which shows a README template containing a CHANGELOG link.
A block now opens on a blank line and continues while the indent holds.
Also corrected two claims in this README: it said "one network call"
when there are two, and it still argued against a CONTRIBUTING using
reasoning the solo-maintainer section had already replaced.
77 tests. llm-security regression: still zero link and boilerplate noise.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WYJ3FHLtVgzFXMZ6UF598h
Measured this build against a documentation brief for public repos. The
five original checks covered roughly one of its ten sections, so this
adds what a single repo can answer on its own.
New: required README headings per class (Non-goals is the cheapest
trust-builder there is), in-repo version consistency across manifest /
badge / CHANGELOG / tag, badge honesty, boilerplate, licence-claim,
and relative links. Findings now carry a BUCKET beside the level -
broken / missing / weakening - and output is grouped by it, because
that is the order the work gets done in.
Traits are a second axis beside class: class is structural and readable
off the catalog, a trait says what the code does. `security` attaches
SECURITY.md and a Known limitations section. The two names carrying it
are proposed, not measured - that list is the operator's.
Solo-maintained settles a category: CONTRIBUTING, CODE_OF_CONDUCT and
MAINTAINERS are required by no class. Consumer-facing documents are
untouched by that; SECURITY.md exists for the stranger who finds a hole.
Three bugs found by running against llm-security, not by reading:
- ~30 link findings, all noise. Regexes inside code spans are
`[...](...)` to a naive scanner. Strip code first.
- `file:` and other schemes were treated as repo-relative paths.
- Relative links were resolved against the repo root instead of the
file they sit in, calling two files missing that sat next to the
README linking them.
Same fix applied to the boilerplate check: a document ABOUT placeholder
detection was tripping the placeholder detector.
Also removed this repo's own static tests badge. There is no CI - the
forge has zero Actions runners registered - so it could never become
real, and it is the exact anti-pattern the gate now flags.
67 tests. Against llm-security every remaining finding is real and
matches the census's independent hand-measurement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WYJ3FHLtVgzFXMZ6UF598h
The H1 change added two tests; the badge, the README line and the
CHANGELOG still said 32. Exactly the drift this repo exists to catch,
in this repo - and check-versions.mjs would not have caught it, since
it only reads the version badge.
Also states in the README what actually fails the first-screen check
now that a differing H1 is a WARN.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WYJ3FHLtVgzFXMZ6UF598h
Five checks a single repository can answer on its own: README first
screen, install block, files required by its class, open/<name>
references, description length. Pure classifiers with I/O resolved into
their input, mirroring check-versions.mjs; ERROR/WARN/SKIP/OK, exit 1
on ERROR. 32 tests.
The reference check has THREE outcomes: "matches no repo" (ERROR) is
separate from "matches a known non-repo" (WARN). Sharing an outcome
would let real dead links hide inside correct text. Only names in URL
position count, and .git is normalised first - without that a raw scan
turns 3 dead names into ~20.
enabledPlugins is treated as a legitimate second install form; what the
gate requires in addition is a CLI command. The JSON form is never
reported as the defect.
STATE.md is gitignored from this first commit - public remote.
No hook yet: a blocking gate must first be precise enough not to fail a
correct repository.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WYJ3FHLtVgzFXMZ6UF598h
