fix(plugin-health): make the command able to read what the scanner found
Dogfooding `/config-audit plugin-health` against a fasit registered before the
run: 11 of 12 predictions confirmed, 1 refuted with evidence, 0 deviations.
The command's default path could not produce the report it documents.
M-BUG-21 (third arm): the argument loop ended in
`else if (!args[i].startsWith('-')) targetPath = args[i]` with no unknown-flag
branch, so `--output-file /tmp/x.json` was dropped and its value became the scan
target. Worse than in drift-cli: a non-existent path discovers no plugins, so the
scanner answered "No plugins found" (info) with exit 0 — a reassuring answer, not
an error. Unknown options and a value-less `--output-file` now exit 3.
M-BUG-33: the scanner had no `--output-file` and its default-mode report goes to
stderr, which `commands/plugin-health.md` discards with `2>/dev/null` before
telling the agent to read stdout. Zero bytes captured.
M-BUG-34: per-plugin rows and the grade formula never left `scan()` — the only
grade code, `formatPluginHealthReport`, had no caller — and cross-plugin findings
were flattened behind a `category` they share with per-plugin findings. The
mandated table and Cross-Plugin section were unbuildable, so the command had to
fabricate them. `scanDetailed()` now returns them; `scan()`'s frozen v5.0.0
envelope is unchanged by construction.
M-BUG-35: `.claude-plugin/marketplace.json` was flagged as an unknown file. It is
the documented catalog location, and `"source": "./"` makes the repo root its own
plugin, so one `.claude-plugin/` legitimately holds both.
Also: `commands/posture.md` ran both optional scanners in default mode under
`2>/dev/null` and read stdout — the same class as feature-gap.md:133 in the fix
chunk. A CLI-side flag fix does not close its callers.
Tests 1420 -> 1432, red first. Frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhhZ8zg1amR7YCAPqiZWdt
This commit is contained in:
parent
05f1e954d0
commit
001090261e
8 changed files with 398 additions and 33 deletions
25
CHANGELOG.md
25
CHANGELOG.md
|
|
@ -8,6 +8,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **`M-BUG-21` (third arm) — `plugin-health-scanner.mjs` swallowed unknown flags, and the wrong
|
||||
target looked *green*.** The same `else if (!args[i].startsWith('-')) targetPath = args[i]` loop:
|
||||
`--output-file /tmp/x.json` was dropped and `/tmp/x.json` became the scan target. Where `drift`
|
||||
produced phantom drift, this produced a **reassuring** answer — a non-existent path discovers no
|
||||
plugins, so the scanner reported `No plugins found` (info) and exit `0`. Unknown options and a
|
||||
value-less `--output-file` now exit `3`.
|
||||
- **`M-BUG-33` — `/config-audit plugin-health` read zero bytes.** The scanner had no `--output-file`
|
||||
(ux-rules rule 2) and its default-mode report goes to **stderr**, which `commands/plugin-health.md`
|
||||
discards with `2>/dev/null` before telling the agent to "read stdout output (JSON)". The command's
|
||||
default path could not produce the report it documents. `--output-file` now writes a humanized
|
||||
payload; `--raw`/`--json` stdout is unchanged and byte-stable.
|
||||
- **`M-BUG-34` — the report's per-plugin table and Cross-Plugin section were unbuildable.** Per-plugin
|
||||
data (`commandCount`, `agentCount`) and the grade formula never left `scan()` — the only grade code,
|
||||
`formatPluginHealthReport`, had no caller — and cross-plugin findings were flattened into `findings`
|
||||
behind a `category: 'plugin-hygiene'` they share with per-plugin findings. The command mandated both,
|
||||
so it had to fabricate them. The payload now carries `plugins[]` (name, declaredName, counts, score,
|
||||
grade via the shared `pluginGrade`) and `cross_plugin_findings[]` (also marked `crossPlugin: true`),
|
||||
via a new `scanDetailed()`; `scan()`'s frozen v5.0.0 envelope is untouched.
|
||||
- **`M-BUG-35` — `.claude-plugin/marketplace.json` was reported as an unknown file.** It is the
|
||||
documented, required location for a marketplace catalog, and a marketplace entry with
|
||||
`"source": "./"` makes the repo root its own plugin — such a repo legitimately carries both files.
|
||||
Genuinely unexpected files in `.claude-plugin/` are still flagged.
|
||||
- **`commands/posture.md` discarded both optional scanners' output.** Its `--drift` and
|
||||
`--plugin-health` sections ran `drift-cli.mjs` / `plugin-health-scanner.mjs` in default mode under
|
||||
`2>/dev/null` and read stdout, which is empty in that mode. Both calls now use `--output-file`.
|
||||
- **`M-BUG-21` — `drift-cli.mjs` had no `--output-file`, and its argument loop turned the missing
|
||||
flag into a wrong scan target.** The loop ended in `else if (!arg.startsWith('-')) targetPath = arg`
|
||||
with no unknown-flag branch, so an unrecognised flag was dropped silently and its *value* fell
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 16 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, cross-plugin collision, output-style, and always-loaded agent-listing-budget detection. Zero external dependencies.
|
||||
|
|
|
|||
|
|
@ -32,15 +32,22 @@ Auditing {N} plugin(s) for structure, frontmatter quality, and cross-plugin conf
|
|||
|
||||
### Step 2: Run scanner
|
||||
|
||||
Run silently for each plugin. Default mode emits a humanized JSON envelope where each PLH finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` alongside the v5.0.0 fields. `--raw` is passed through verbatim when present.
|
||||
Run silently for each plugin. Default mode writes a humanized JSON payload to `--output-file` where each PLH finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` alongside the v5.0.0 fields. `--raw` is passed through verbatim when present, and prints the byte-stable v5.0.0 envelope on stdout instead.
|
||||
|
||||
```bash
|
||||
TMPFILE="/tmp/config-audit-plugin-health-$$.json"
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <path> $RAW_FLAG 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <path> --output-file "$TMPFILE" $RAW_FLAG 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout output (JSON) using the Read tool. Parse findings.
|
||||
Read `$TMPFILE` with the Read tool. Exit codes 0, 1 and 2 are normal; only 3 is a real error.
|
||||
|
||||
The payload carries three things the report needs:
|
||||
|
||||
- `plugins[]` — one row per plugin: `name`, `declaredName`, `commandCount`, `agentCount`, `findingCount`, `score`, `grade`. Use these for the table; never estimate a grade yourself.
|
||||
- `cross_plugin_findings[]` — the namespace-collision and shared-command-name findings, already separated from the per-plugin ones (they also carry `crossPlugin: true` in `findings`).
|
||||
- `findings[]` — every finding, humanized.
|
||||
|
||||
### Step 3: Present results
|
||||
|
||||
|
|
@ -49,7 +56,7 @@ Read stdout output (JSON) using the Read tool. Parse findings.
|
|||
|
||||
| Plugin | Grade | Commands | Agents | Status |
|
||||
|--------|-------|----------|--------|--------|
|
||||
| {name} | {grade} ({score}) | {cmd_count} | {agent_count} | {Good/Issues found} |
|
||||
| {plugins[].name} | {plugins[].grade} ({plugins[].score}) | {plugins[].commandCount} | {plugins[].agentCount} | {Good/Issues found} |
|
||||
| ... | ... | ... | ... | ... |
|
||||
|
||||
{If cross-plugin issues:}
|
||||
|
|
|
|||
|
|
@ -93,19 +93,19 @@ Avoid hardcoded grade-to-prose ladders here — the humanized scorecard headline
|
|||
|
||||
Run drift comparison silently:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <target-path> 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <target-path> --output-file /tmp/config-audit-posture-drift-$$.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout output and append a "Configuration Drift" section showing what changed since the last baseline.
|
||||
Read that file and append a "Configuration Drift" section showing what changed since the last baseline. Both scanners report to stderr in default mode, which `2>/dev/null` discards — the payload is the only readable output.
|
||||
|
||||
**If `--plugin-health` flag is present:**
|
||||
|
||||
Run plugin health scanner silently:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <target-path> 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <target-path> --output-file /tmp/config-audit-posture-plh-$$.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout output and append a "Plugin Health" section.
|
||||
Read that file and append a "Plugin Health" section, using its `plugins[]` rows for per-plugin grades.
|
||||
|
||||
**If both flags:** Use `scanners/lib/report-generator.mjs` to produce a unified markdown report.
|
||||
|
||||
|
|
|
|||
|
|
@ -397,6 +397,43 @@ claimed CC "suggests the parent directory when an entry points at a file"; that
|
|||
primary-source-verified conditions. `skills` is deliberately *not* in `SHADOWING_PATH_FIELDS`
|
||||
(it adds to the default scan, never shadows).
|
||||
|
||||
### PLH scanner — `scanDetailed`, `--output-file`, and the marketplace.json exemption (økt #46)
|
||||
|
||||
`scan()` returns the **frozen v5.0.0 envelope** (`scanner, status, files_scanned, duration_ms,
|
||||
findings, counts`) and nothing else — `--raw`/`--json` print it verbatim and are snapshot-gated. Two
|
||||
things the `/config-audit plugin-health` report requires therefore cannot live there: one row per
|
||||
plugin (the `| Plugin | Grade | Commands | Agents |` table) and the cross-plugin/per-plugin split.
|
||||
Both were computed inside `scan()` and discarded at the return: `pluginResults` never escaped, and
|
||||
the only grade code — `formatPluginHealthReport` — had no caller anywhere in the repo.
|
||||
|
||||
`scanDetailed(targetPath)` is the seam. It returns `{ result, plugins, crossPluginFindings }`;
|
||||
`scan()` is now `(await scanDetailed(p)).result`, so the byte-stable envelope is unchanged by
|
||||
construction. `plugins[]` carries `name, declaredName, path, commandCount, agentCount, findingCount`
|
||||
plus `score`/`grade` from the shared `pluginGrade(issueCount)` helper (which
|
||||
`formatPluginHealthReport` now also calls, so the formula has exactly one home).
|
||||
|
||||
Cross-plugin findings are identified **positionally**, not by predicate: `crossPluginStart =
|
||||
allFindings.length` is taken immediately before the namespace/command-name sections, and the tail is
|
||||
sliced off at the end. A predicate would have to key on `category: 'plugin-hygiene'`, which the
|
||||
per-plugin shadow and skills findings share. The marker (`crossPlugin: true`) is stamped only on the
|
||||
**humanized copies** in the `--output-file` payload — never inside `finding()`, which would add a key
|
||||
to the frozen envelope.
|
||||
|
||||
`--output-file` follows the `drift-cli` contract: humanized payload in default mode, stdout
|
||||
untouched. This matters because default mode writes its report to **stderr**, and ux-rules rule 2
|
||||
requires the command to run under `2>/dev/null` — before this, `commands/plugin-health.md` (and both
|
||||
optional scanner calls in `commands/posture.md`) captured zero bytes. Argument parsing uses the same
|
||||
`BOOL_FLAGS`/`VALUE_FLAGS` + unknown-flag-throws shape as `drift-cli`/`fix-cli` (M-BUG-21, third
|
||||
arm); here the swallowed-flag failure mode was *worse than an error* — scanning the dropped flag's
|
||||
value found no plugins, so the scanner answered `No plugins found` (info) with exit `0`.
|
||||
|
||||
**`marketplace.json` exemption:** `.claude-plugin/`'s known-file set is `plugin.json` **and**
|
||||
`marketplace.json`. The catalog's location is documented and required (*"Create
|
||||
`.claude-plugin/marketplace.json` in your repository root"*), and a marketplace entry with
|
||||
`"source": "./"` makes the repo root its own plugin — so one `.claude-plugin/` legitimately holds
|
||||
both. Verified against the primary docs before the change; the check was a false positive, latent in
|
||||
this marketplace only because `catalog/` ships no `plugin.json` and is thus not scanned as a plugin.
|
||||
|
||||
### SET scanner — autoMode validation (`CA-SET`)
|
||||
|
||||
Per-file check in `settings-validator.mjs` (`autoMode` was in `KNOWN_KEYS` but had no nested
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { readdir, stat, readFile } from 'node:fs/promises';
|
||||
import { readdir, stat, readFile, writeFile } from 'node:fs/promises';
|
||||
import { join, basename, resolve, sep } from 'node:path';
|
||||
import { finding, scannerResult, resetCounter } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
|
|
@ -490,7 +490,13 @@ async function scanSinglePlugin(pluginDir) {
|
|||
const pluginMetaDir = join(pluginDir, '.claude-plugin');
|
||||
try {
|
||||
const entries = await readdir(pluginMetaDir);
|
||||
const known = new Set(['plugin.json']);
|
||||
// `marketplace.json` belongs here: it is the documented, required location
|
||||
// for a marketplace catalog (code.claude.com/docs plugin-marketplaces —
|
||||
// "Create `.claude-plugin/marketplace.json` in your repository root"), and a
|
||||
// marketplace entry with `"source": "./"` makes the repo root its own
|
||||
// plugin. Such a repo legitimately carries both files, so flagging the
|
||||
// catalog as an unknown file was a false positive.
|
||||
const known = new Set(['plugin.json', 'marketplace.json']);
|
||||
for (const entry of entries) {
|
||||
if (!known.has(entry)) {
|
||||
findings.push(finding({
|
||||
|
|
@ -508,27 +514,62 @@ async function scanSinglePlugin(pluginDir) {
|
|||
return { name: pluginName, declaredName, findings, commandCount, agentCount };
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-plugin score and grade. Single source for both the terminal report and
|
||||
* the --output-file payload — the grade formula used to live only inside
|
||||
* `formatPluginHealthReport`, which nothing called.
|
||||
* @param {number} issueCount
|
||||
* @returns {{ score: number, grade: string }}
|
||||
*/
|
||||
export function pluginGrade(issueCount) {
|
||||
const score = Math.max(0, 100 - issueCount * 10);
|
||||
const grade = score >= 90 ? 'A' : score >= 75 ? 'B' : score >= 60 ? 'C' : score >= 40 ? 'D' : 'F';
|
||||
return { score, grade };
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan one or more plugins and return aggregated results.
|
||||
*
|
||||
* The envelope is frozen at the v5.0.0 shape (byte-stable `--raw`/`--json`), so
|
||||
* per-plugin rows and the cross-plugin/per-plugin split are NOT in it. Callers
|
||||
* that need those — the `--output-file` payload, and therefore
|
||||
* `/config-audit plugin-health` — use `scanDetailed`.
|
||||
*
|
||||
* @param {string} targetPath - Plugin dir or marketplace root
|
||||
* @returns {Promise<object>} Scanner result
|
||||
*/
|
||||
export async function scan(targetPath) {
|
||||
return (await scanDetailed(targetPath)).result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan, and also return what `scan()`'s frozen envelope cannot carry: one row
|
||||
* per plugin (name, declared namespace, component counts, grade) and the
|
||||
* cross-plugin findings as a distinct set.
|
||||
*
|
||||
* @param {string} targetPath - Plugin dir or marketplace root
|
||||
* @returns {Promise<{ result: object, plugins: object[], crossPluginFindings: object[] }>}
|
||||
*/
|
||||
export async function scanDetailed(targetPath) {
|
||||
const start = Date.now();
|
||||
resetCounter();
|
||||
|
||||
const pluginDirs = await discoverPlugins(resolve(targetPath));
|
||||
|
||||
if (pluginDirs.length === 0) {
|
||||
return scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
title: 'No plugins found',
|
||||
description: `No Claude Code plugins found under ${targetPath}`,
|
||||
recommendation: 'Ensure plugins have .claude-plugin/plugin.json',
|
||||
}),
|
||||
], 0, Date.now() - start);
|
||||
return {
|
||||
result: scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
title: 'No plugins found',
|
||||
description: `No Claude Code plugins found under ${targetPath}`,
|
||||
recommendation: 'Ensure plugins have .claude-plugin/plugin.json',
|
||||
}),
|
||||
], 0, Date.now() - start),
|
||||
plugins: [],
|
||||
crossPluginFindings: [],
|
||||
};
|
||||
}
|
||||
|
||||
const allFindings = [];
|
||||
|
|
@ -540,6 +581,12 @@ export async function scan(targetPath) {
|
|||
allFindings.push(...result.findings);
|
||||
}
|
||||
|
||||
// Everything pushed from here on is a cross-plugin finding — the boundary the
|
||||
// payload uses to split them out (they are flattened into `findings` in the
|
||||
// frozen envelope, where `category: 'plugin-hygiene'` cannot tell them apart
|
||||
// from the per-plugin shadow/skills findings that share it).
|
||||
const crossPluginStart = allFindings.length;
|
||||
|
||||
// Cross-plugin checks: command-name ambiguity across DIFFERENT plugin namespaces.
|
||||
// Commands are namespaced by the plugin's declared name (/name:command), so a
|
||||
// shared command name across DIFFERENT plugins is ambiguity — not a hard
|
||||
|
|
@ -632,7 +679,19 @@ export async function scan(targetPath) {
|
|||
}));
|
||||
}
|
||||
|
||||
return scannerResult(SCANNER, 'ok', allFindings, pluginDirs.length, Date.now() - start);
|
||||
return {
|
||||
result: scannerResult(SCANNER, 'ok', allFindings, pluginDirs.length, Date.now() - start),
|
||||
plugins: pluginResults.map((p, idx) => ({
|
||||
name: p.name,
|
||||
declaredName: p.declaredName,
|
||||
path: pluginDirs[idx],
|
||||
commandCount: p.commandCount,
|
||||
agentCount: p.agentCount,
|
||||
findingCount: p.findings.length,
|
||||
...pluginGrade(p.findings.length),
|
||||
})),
|
||||
crossPluginFindings: allFindings.slice(crossPluginStart),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -649,9 +708,7 @@ export function formatPluginHealthReport(pluginResults, crossPluginFindings) {
|
|||
lines.push('');
|
||||
|
||||
for (const p of pluginResults) {
|
||||
const issueCount = p.findings.length;
|
||||
const score = Math.max(0, 100 - issueCount * 10);
|
||||
const grade = score >= 90 ? 'A' : score >= 75 ? 'B' : score >= 60 ? 'C' : score >= 40 ? 'D' : 'F';
|
||||
const { score, grade } = pluginGrade(p.findings.length);
|
||||
const padding = '.'.repeat(Math.max(1, 25 - p.name.length));
|
||||
lines.push(` ${p.name} ${padding} ${grade} (${score}) ${p.commandCount} commands, ${p.agentCount} agents`);
|
||||
}
|
||||
|
|
@ -675,19 +732,42 @@ export function formatPluginHealthReport(pluginResults, crossPluginFindings) {
|
|||
}
|
||||
|
||||
// --- CLI entry point ---
|
||||
const BOOL_FLAGS = ['--json', '--raw'];
|
||||
const VALUE_FLAGS = ['--output-file'];
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
let targetPath = '.';
|
||||
let jsonMode = false;
|
||||
let rawMode = false;
|
||||
let outputFile = null;
|
||||
|
||||
// M-BUG-21, third arm: this loop used to end in
|
||||
// `else if (!args[i].startsWith('-')) targetPath = args[i]` with no
|
||||
// unknown-flag branch. An unrecognised flag was dropped silently and its
|
||||
// VALUE became the scan target, so `--output-file /tmp/x.json` scanned
|
||||
// /tmp/x.json. Unlike drift-cli, the result LOOKS fine: a non-existent path
|
||||
// discovers no plugins, so the scanner reported "No plugins found" (info) and
|
||||
// exit 0 — a green answer to a question nobody asked. Now it fails loudly.
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--json') {
|
||||
jsonMode = true;
|
||||
} else if (args[i] === '--raw') {
|
||||
rawMode = true;
|
||||
} else if (!args[i].startsWith('-')) {
|
||||
targetPath = args[i];
|
||||
const arg = args[i];
|
||||
if (BOOL_FLAGS.includes(arg)) {
|
||||
if (arg === '--json') jsonMode = true;
|
||||
else if (arg === '--raw') rawMode = true;
|
||||
} else if (VALUE_FLAGS.includes(arg)) {
|
||||
const value = args[i + 1];
|
||||
if (value === undefined || value.startsWith('-')) {
|
||||
throw new Error(`Option ${arg} requires a value.`);
|
||||
}
|
||||
outputFile = value;
|
||||
i++;
|
||||
} else if (arg.startsWith('-')) {
|
||||
throw new Error(
|
||||
`Unknown option: ${arg}\n` +
|
||||
`Valid options: ${[...BOOL_FLAGS, ...VALUE_FLAGS].join(' ')}`
|
||||
);
|
||||
} else {
|
||||
targetPath = arg;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -695,7 +775,7 @@ async function main() {
|
|||
process.stderr.write(humanizedProgress ? `Plugin Health v2.1.0\n` : `Plugin Health Scanner v2.1.0\n`);
|
||||
process.stderr.write(`Target: ${resolve(targetPath)}\n\n`);
|
||||
|
||||
const result = await scan(targetPath);
|
||||
const { result, plugins, crossPluginFindings } = await scanDetailed(targetPath);
|
||||
|
||||
if (jsonMode || rawMode) {
|
||||
// --json and --raw both write the v5.0.0-shape result (byte-identical).
|
||||
|
|
@ -708,6 +788,24 @@ async function main() {
|
|||
for (const f of findings) {
|
||||
process.stderr.write(` [${f.severity}] ${f.title}\n`);
|
||||
}
|
||||
|
||||
// ux-rules rule 2: the command runs with `2>/dev/null`, so anything it must
|
||||
// ACT on has to ride in the --output-file payload. Everything above this
|
||||
// point is stderr, i.e. invisible to `/config-audit plugin-health`.
|
||||
if (outputFile) {
|
||||
const crossIds = new Set(crossPluginFindings.map(f => f.id));
|
||||
for (const f of findings) {
|
||||
if (crossIds.has(f.id)) f.crossPlugin = true;
|
||||
}
|
||||
const payload = {
|
||||
...result,
|
||||
findings,
|
||||
plugins,
|
||||
cross_plugin_findings: findings.filter(f => crossIds.has(f.id)),
|
||||
};
|
||||
await writeFile(outputFile, JSON.stringify(payload, null, 2), 'utf-8');
|
||||
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -132,3 +132,29 @@ test('status.md: preserves current_phase machine field and adds humanized phase
|
|||
`status.md must include at least 3 humanized phase labels; found ${present.length}: ${present.join(', ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Økt #46 — ux-rules rule 2 for the plugin-health scanner.
|
||||
//
|
||||
// plugin-health.md passed the humanized-field assertion above while the data it
|
||||
// names was unreachable: the scanner had no --output-file, and its default-mode
|
||||
// report went to stderr, which the command discards with `2>/dev/null`. A .md
|
||||
// contract test that only greps for prose cannot catch that — these assert the
|
||||
// plumbing that makes the prose true.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('plugin-health.md invokes the scanner with --output-file (ux-rules rule 2)', async () => {
|
||||
const content = await readCommand('plugin-health.md');
|
||||
const call = content.split('\n').find(l => l.includes('plugin-health-scanner.mjs'));
|
||||
assert.ok(call, 'plugin-health.md must invoke plugin-health-scanner.mjs');
|
||||
assert.match(call, /--output-file/, 'scanner call must write to a file, not stdout/stderr');
|
||||
});
|
||||
|
||||
test('posture.md invokes the plugin-health and drift scanners with --output-file', async () => {
|
||||
const content = await readCommand('posture.md');
|
||||
for (const scanner of ['plugin-health-scanner.mjs', 'drift-cli.mjs']) {
|
||||
const call = content.split('\n').find(l => l.includes(`scanners/${scanner}`));
|
||||
assert.ok(call, `posture.md must invoke ${scanner}`);
|
||||
assert.match(call, /--output-file/, `${scanner} call in posture.md discards its output`);
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,8 +2,9 @@ import { describe, it, beforeEach, afterEach } from 'node:test';
|
|||
import assert from 'node:assert/strict';
|
||||
import { resolve, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { mkdtemp, mkdir, writeFile, rm, readFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan, discoverPlugins } from '../../scanners/plugin-health-scanner.mjs';
|
||||
|
||||
|
|
@ -423,3 +424,174 @@ describe('PLH — plugin agent declares fields Claude Code ignores (E)', () => {
|
|||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Økt #46 — `plugin-health` dogfood. Four defects, all found by running the
|
||||
// command as written and comparing against a fasit registered BEFORE the run.
|
||||
//
|
||||
// F1 (M-BUG-21, third arm): the arg loop ended in
|
||||
// `else if (!args[i].startsWith('-')) targetPath = args[i]` with no
|
||||
// unknown-flag branch, so `--output-file /tmp/x.json` was dropped silently and
|
||||
// `/tmp/x.json` became the scan target. Worse than in drift-cli: scanning a
|
||||
// non-existent path yields "No plugins found" (info) and exit 0 — an
|
||||
// apparently GREEN answer, not an error.
|
||||
//
|
||||
// F3 (stderr-only class): default mode wrote the report to STDERR only, so
|
||||
// commands/plugin-health.md ("... 2>/dev/null" + "Read stdout output (JSON)")
|
||||
// captured zero bytes. There was no --output-file at all (ux-rules rule 2).
|
||||
//
|
||||
// F5/F7: per-plugin data (name/commandCount/agentCount) and the grade formula
|
||||
// never left scan(); cross-plugin findings were flattened into `findings` with
|
||||
// no marker. commands/plugin-health.md mandates a
|
||||
// `| Plugin | Grade | Commands | Agents |` table plus a separate Cross-Plugin
|
||||
// section — both unbuildable, so the command had to fabricate them.
|
||||
//
|
||||
// F9: `.claude-plugin/marketplace.json` was reported as "Unknown file". It is
|
||||
// the documented location for a marketplace catalog
|
||||
// (code.claude.com/docs/en/plugin-marketplaces), and with `"source": "./"` one
|
||||
// repo is legitimately both plugin and marketplace.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const PLH_CLI = resolve(__dirname, '../../scanners/plugin-health-scanner.mjs');
|
||||
const PLH_RUN = { encoding: 'utf-8', timeout: 20000 };
|
||||
|
||||
function plhSpawn(args) {
|
||||
const res = spawnSync('node', [PLH_CLI, ...args], PLH_RUN);
|
||||
return { status: res.status, stdout: String(res.stdout || ''), stderr: String(res.stderr || '') };
|
||||
}
|
||||
|
||||
function plhExpectFailure(args) {
|
||||
const { status, stderr } = plhSpawn(args);
|
||||
return { status, stderr };
|
||||
}
|
||||
|
||||
describe('plugin-health-scanner argument validation (F1 / M-BUG-21 third arm)', () => {
|
||||
it('rejects an unknown flag instead of swallowing its value as the scan target', () => {
|
||||
const { status, stderr } = plhExpectFailure([TEST_PLUGIN, '--bogus', 'some-value', '--json']);
|
||||
assert.equal(status, 3, 'unknown flag must fail loudly, not scan "some-value"');
|
||||
assert.match(stderr, /unknown option/i);
|
||||
assert.match(stderr, /--bogus/);
|
||||
});
|
||||
|
||||
it('rejects --output-file without a value', () => {
|
||||
const { status, stderr } = plhExpectFailure([TEST_PLUGIN, '--output-file']);
|
||||
assert.equal(status, 3);
|
||||
assert.match(stderr, /--output-file/);
|
||||
assert.match(stderr, /requires a value/i);
|
||||
});
|
||||
|
||||
it('still accepts a bare path as the scan target', () => {
|
||||
const { status, stdout } = plhSpawn([TEST_PLUGIN, '--json']);
|
||||
assert.equal(status, 0);
|
||||
assert.equal(JSON.parse(stdout).files_scanned, 1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('plugin-health-scanner --output-file (F3 / ux-rules rule 2)', () => {
|
||||
let dir;
|
||||
beforeEach(async () => { dir = await mkdtemp(join(tmpdir(), 'ca-plh-out-')); });
|
||||
afterEach(async () => { await rm(dir, { recursive: true, force: true }); });
|
||||
|
||||
it('writes the payload to the file and keeps default-mode stdout empty', async () => {
|
||||
const out = join(dir, 'plh.json');
|
||||
const { status, stdout } = plhSpawn([TEST_PLUGIN, '--output-file', out]);
|
||||
assert.equal(status, 0);
|
||||
assert.equal(stdout, '', 'default mode must not print to stdout (ux-rules rule 1)');
|
||||
const payload = JSON.parse(await readFile(out, 'utf-8'));
|
||||
assert.equal(payload.scanner, 'PLH');
|
||||
assert.equal(payload.files_scanned, 1);
|
||||
});
|
||||
|
||||
it('carries humanizer fields the command renders (F4)', async () => {
|
||||
const out = join(dir, 'plh.json');
|
||||
plhSpawn([BROKEN_PLUGIN, '--output-file', out]);
|
||||
const payload = JSON.parse(await readFile(out, 'utf-8'));
|
||||
assert.ok(payload.findings.length > 0, 'broken-plugin must produce findings');
|
||||
for (const f of payload.findings) {
|
||||
assert.ok(f.userImpactCategory, `finding ${f.id} missing userImpactCategory`);
|
||||
assert.ok(f.userActionLanguage, `finding ${f.id} missing userActionLanguage`);
|
||||
}
|
||||
});
|
||||
|
||||
it('exposes per-plugin rows with grade, score and component counts (F5)', async () => {
|
||||
const out = join(dir, 'plh.json');
|
||||
plhSpawn([TEST_PLUGIN, '--output-file', out]);
|
||||
const payload = JSON.parse(await readFile(out, 'utf-8'));
|
||||
assert.ok(Array.isArray(payload.plugins), 'payload must carry a plugins array');
|
||||
assert.equal(payload.plugins.length, 1);
|
||||
const p = payload.plugins[0];
|
||||
assert.equal(p.name, 'test-plugin');
|
||||
assert.ok(typeof p.commandCount === 'number');
|
||||
assert.ok(typeof p.agentCount === 'number');
|
||||
assert.ok(typeof p.score === 'number');
|
||||
assert.match(p.grade, /^[ABCDF]$/);
|
||||
});
|
||||
|
||||
it('separates cross-plugin findings from per-plugin findings (F7)', async () => {
|
||||
const out = join(dir, 'plh.json');
|
||||
plhSpawn([DUP_NAME, '--output-file', out]);
|
||||
const payload = JSON.parse(await readFile(out, 'utf-8'));
|
||||
assert.ok(Array.isArray(payload.cross_plugin_findings), 'payload must carry cross_plugin_findings');
|
||||
assert.ok(payload.cross_plugin_findings.length > 0, 'duplicate-plugin-name must yield a namespace collision');
|
||||
for (const f of payload.cross_plugin_findings) {
|
||||
assert.equal(f.crossPlugin, true, 'cross-plugin findings must be marked');
|
||||
}
|
||||
const perPlugin = payload.findings.filter(f => f.crossPlugin !== true);
|
||||
assert.ok(
|
||||
perPlugin.length + payload.cross_plugin_findings.length === payload.findings.length,
|
||||
'cross_plugin_findings must be a subset of findings, not a parallel universe'
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves --raw and --json byte-stable (no new keys on the frozen envelope)', () => {
|
||||
const raw = JSON.parse(plhSpawn([TEST_PLUGIN, '--raw']).stdout);
|
||||
const json = JSON.parse(plhSpawn([TEST_PLUGIN, '--json']).stdout);
|
||||
for (const env of [raw, json]) {
|
||||
assert.deepEqual(
|
||||
Object.keys(env).sort(),
|
||||
['counts', 'duration_ms', 'files_scanned', 'findings', 'scanner', 'status'],
|
||||
'frozen v5.0.0 envelope must not gain keys'
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('.claude-plugin/marketplace.json is not an unknown file (F9)', () => {
|
||||
let dir;
|
||||
beforeEach(async () => { dir = await mkdtemp(join(tmpdir(), 'ca-plh-mp-')); });
|
||||
afterEach(async () => { await rm(dir, { recursive: true, force: true }); });
|
||||
|
||||
async function writeMarketplacePlugin(root, extraFiles = {}) {
|
||||
await mkdir(join(root, '.claude-plugin'), { recursive: true });
|
||||
await writeFile(
|
||||
join(root, '.claude-plugin', 'plugin.json'),
|
||||
JSON.stringify({ name: 'mp-plugin', description: 'd', version: '1.0.0' })
|
||||
);
|
||||
await writeFile(
|
||||
join(root, '.claude-plugin', 'marketplace.json'),
|
||||
JSON.stringify({ name: 'cat', owner: { name: 'x' }, plugins: [] })
|
||||
);
|
||||
await writeFile(join(root, 'CLAUDE.md'), '# mp-plugin\n');
|
||||
for (const [name, body] of Object.entries(extraFiles)) {
|
||||
await writeFile(join(root, '.claude-plugin', name), body);
|
||||
}
|
||||
}
|
||||
|
||||
it('does not flag marketplace.json (documented catalog location)', async () => {
|
||||
resetCounter();
|
||||
await writeMarketplacePlugin(dir);
|
||||
const result = await scan(dir);
|
||||
const unknown = result.findings.filter(f => /Unknown file/i.test(f.title || ''));
|
||||
assert.equal(unknown.length, 0,
|
||||
`marketplace.json is documented; got: ${unknown.map(f => f.file).join(' | ')}`);
|
||||
});
|
||||
|
||||
it('still flags a genuinely unexpected file', async () => {
|
||||
resetCounter();
|
||||
await writeMarketplacePlugin(dir, { 'notes.txt': 'scratch' });
|
||||
const result = await scan(dir);
|
||||
const unknown = result.findings.filter(f => /Unknown file/i.test(f.title || ''));
|
||||
assert.equal(unknown.length, 1);
|
||||
assert.match(unknown[0].file, /notes\.txt$/);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue