docs(config-audit): straggler sweep for v5.0.0 — sync all badge counts

Reconcile README/CLAUDE.md/commands/agents to filesystem truth ahead of v5.0.0
release. Self-audit --check-readme now passes (counts: scanners 12, commands 18,
tests 635, knowledge 8, agents 6, hooks 4).

Self-audit (scanners/self-audit.mjs):
- Exclude plugin-health-scanner.mjs from countScannerShape (it is a "standalone"
  scanner per README/CLAUDE.md taxonomy; orchestrated scanners stay at 12)
- countTestCases: spawn `node --test` and parse the `tests N` line so the badge
  reflects test cases (635), not test files (36). countTestFiles kept as
  fallback when subprocess fails.

README.md:
- Badges: scanners 9→12, commands 17→18, tests 543→635
- Body counts updated: 8 quality scanners → 12 deterministic scanners; 8 quality
  areas → 10 (incl. Plugin Hygiene from N6); 9 Node.js scanners → 12
- Scanner table extended with CPS / DIS / COL rows; TOK row reflects the v5
  Pattern E/F/N1 expansion (sonnet-era removed)
- CLI table adds manifest, whats-active, --accurate-tokens, --with-telemetry-recipe
- Knowledge table adds opus-4.7-patterns.md and cache-telemetry-recipe.md
- Scanner Lib table notes WEIGHTS export, severity-weighted scoring, tokenizer-api
- Action Engines table adds manifest.mjs, whats-active.mjs, token-hotspots-cli.mjs
- Test count text 486→635, file count 27→36 (12 lib + 23 scanner + 1 hook)
- Tokens command: 4-pattern phrasing → 6 patterns + --accurate-tokens
- Adds /config-audit manifest and /config-audit whats-active to command tables

CLAUDE.md:
- Posture row: 8 → 10 quality areas
- Tokens row: 4 patterns (incl. sonnet-era) → 6 patterns + --accurate-tokens
- Adds /config-audit manifest entry
- Scanner table: TOK description rewritten; CPS, DIS, COL rows added
- Scanner Lib table: tokenizer-api.mjs added; v5 annotations on severity, output,
  scoring, active-config-reader
- Action Engines table: manifest.mjs added; token-hotspots-cli.mjs flags expanded
- Knowledge table: cache-telemetry-recipe.md added; configuration-best-practices
  notes Opus-4.7 cache-stability rewrite
- Finding ID examples extended with CA-TOK-005, CA-CPS-001, CA-DIS-001, CA-COL-001
- Test count text 543→635, file count 31→36

commands/help.md: tokens/manifest added to Core
commands/posture.md: 8 → 10 quality areas
commands/config-audit.md: argument-hint adds tokens/manifest; router adds tokens
  and manifest; "Running 8 configuration scanners" → 12
agents/feature-gap-agent.md: 8 → 10 quality areas

No production code paths changed beyond self-audit's badge-counting heuristic.
All 635 tests still green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-05-01 09:34:43 +02:00
commit 5bf500e1a8
7 changed files with 95 additions and 35 deletions

View file

@ -11,6 +11,8 @@
import { resolve, dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { readdir, readFile, stat } from 'node:fs/promises';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { runAllScanners } from './scan-orchestrator.mjs';
import { scan as scanPluginHealth } from './plugin-health-scanner.mjs';
import { scoreByArea } from './lib/scoring.mjs';
@ -18,15 +20,24 @@ import { gradeFromPassRate } from './lib/severity.mjs';
import { loadSuppressions, applySuppressions } from './lib/suppression.mjs';
import { parseJson } from './lib/yaml-parser.mjs';
const execFileAsync = promisify(execFile);
const __dirname = dirname(fileURLToPath(import.meta.url));
const PLUGIN_ROOT = resolve(__dirname, '..');
// Scanner-shape detection: files in scanners/ that export `scan` and are not
// support modules. Matches the detection rule from v5 plan Step 16.
//
// `plugin-health-scanner.mjs` is excluded from the main scanner count: it has
// `export async function scan` but it runs standalone (not via scan-orchestrator)
// and is documented under "Standalone Scanner" in README/CLAUDE.md. The badge
// `scanners-12` reflects the orchestrated scanners that contribute to posture
// scoring.
const SCANNER_EXCLUDES = new Set([
'scan-orchestrator.mjs',
'self-audit.mjs',
'whats-active.mjs',
'plugin-health-scanner.mjs',
]);
function isScannerShape(name, content) {
@ -74,6 +85,29 @@ async function countTestFiles(testsRoot) {
return count;
}
// Run the test suite in a subprocess and parse the ` tests N` line emitted
// by node:test. Used for badge accuracy under --check-readme. Slow (~15s on
// the full plugin) but produces the canonical case count rather than an
// approximation. Returns null on failure so the caller can fall back to
// file count without crashing the audit.
async function countTestCases(pluginRoot) {
try {
const { stdout } = await execFileAsync(
process.execPath,
['--test', 'tests/**/*.test.mjs'],
{ cwd: pluginRoot, timeout: 60000, maxBuffer: 10 * 1024 * 1024 },
);
const match = stdout.match(/^[^\n]*tests\s+(\d+)\s*$/m);
return match ? Number(match[1]) : null;
} catch (err) {
// node --test exits non-zero when tests fail; the count line is still
// present on stdout. Re-parse it from the captured output.
const stdout = err?.stdout || '';
const match = stdout.match(/^[^\n]*tests\s+(\d+)\s*$/m);
return match ? Number(match[1]) : null;
}
}
async function countHookEntries(hooksJsonPath) {
let content;
try { content = await readFile(hooksJsonPath, 'utf-8'); } catch { return 0; }
@ -114,12 +148,13 @@ function parseBadgeNumber(readme, kind) {
* @returns {Promise<{passed: boolean, mismatches: Array<{kind:string, expected:number, foundInReadme:number}>, counts: object, badges: object}>}
*/
export async function checkReadmeBadges(pluginDir) {
const testCases = await countTestCases(pluginDir);
const counts = {
scanners: await countScannerShape(join(pluginDir, 'scanners')),
commands: await countMdFiles(join(pluginDir, 'commands')),
agents: await countMdFiles(join(pluginDir, 'agents')),
hooks: await countHookEntries(join(pluginDir, 'hooks', 'hooks.json')),
tests: await countTestFiles(join(pluginDir, 'tests')),
tests: testCases ?? await countTestFiles(join(pluginDir, 'tests')),
knowledge: await countMdFiles(join(pluginDir, 'knowledge')),
};
let readme = '';