fix(migration): config-audit SC2 gate (re-seed + back-compat exclusion) + generic validator
50-config-audit-sc2.sh: re-seeds snapshot-default-output in-clone (UPDATE_SNAPSHOT=1), then runs the SC2 gate = full 'find tests -name *.test.mjs' MINUS the 6-file machine-locked v5.0.0 byte-stability surface. 730 tests across 46 files pass standalone. templates/validate-plugin.generic.sh: ported, parameterized structure validator for the two test-less plugins (okr, human-friendly-style) — STRUCTURE OK on valid plugin.json + non-empty surface + parseable frontmatter; STRUCTURE FAIL otherwise. Reuses the claude-design/tests/validate-plugin.sh pattern. Brief-correction (operator-ratified 2026-06-17): plan F4 named only json-backcompat + raw-backcompat (2 files). The verified machine-locked surface is 6 — the v5.0.0 fixtures embed the original absolute path AND the claude_md/plugin_hygiene scanners key off a plugins/ ancestor, so findings drift by path (behavioral, not a string rewrite). Dropped clean-room SC2 coverage = config-audit's humanizer/posture-humanizer/scan-orchestrator-humanizer prose-snapshot surface, recorded in the script header + plugin-map.json (not silent). The 3 other v5.0.0-referencing tests (posture, scoring-humanizer, scenario-read-test) are path-independent and stay in the gate. config-audit backlog (out of migration scope): normalize the v5.0.0 fixtures + path-agnostic scanners, then re-include. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
1769657a91
commit
690bbbd68d
4 changed files with 279 additions and 1 deletions
|
|
@ -0,0 +1,93 @@
|
|||
#!/usr/bin/env bash
|
||||
# Step 7 — Resolve the config-audit SC2 blocker for standalone extraction.
|
||||
#
|
||||
# config-audit declares its runner as `node --test 'tests/**/*.test.mjs'` (52 files, CLAUDE.md:109).
|
||||
# Two distinct, directly-verified portability defects block a clean-room SC2 gate at a new clone path:
|
||||
#
|
||||
# (a) REBASABLE — tests/snapshot-default-output.test.mjs asserts byte-equal CLI stdout whose deep
|
||||
# per-file paths are NOT normalized (normalizeScanOrchestrator scrubs only meta.target/timestamp/
|
||||
# duration_ms), so it breaks at a fresh clone path. FIX: re-seed in-clone via the test's own
|
||||
# intended re-approval seam — `UPDATE_SNAPSHOT=1 node --test tests/snapshot-default-output.test.mjs`
|
||||
# (seam documented at its line 33). After re-seeding it asserts byte-equal against the clone's
|
||||
# own path and passes.
|
||||
#
|
||||
# (b) FROZEN / MACHINE-LOCKED — a family of tests assert byte/structure equality against the
|
||||
# tests/snapshots/v5.0.0/ fixtures, which deliberately embed the ORIGINAL capture machine's
|
||||
# absolute path + a sibling marketplace + deleted plugins. At a fresh clone path they break in
|
||||
# TWO ways (both verified directly, 2026-06-17): a literal embedded `path:` mismatch, AND a
|
||||
# BEHAVIORAL drift — the claude_md / plugin_hygiene scanners key off whether a `plugins/<name>/`
|
||||
# ancestor exists in the absolute path, so the clone produces different findingCount/score than
|
||||
# the monorepo capture (e.g. posture-humanizer). drift-cli's baseline diff additionally leaks the
|
||||
# clone path into humanized prose, which trips lint-default-output's tier1/tier3 prose gate.
|
||||
# Regenerating these would defeat their byte-stability purpose and "normalize the path" is not a
|
||||
# string rewrite (the scan BEHAVIOR differs by path) — so the correct migration-scope fix is to
|
||||
# EXCLUDE the machine-locked surface by name (deterministic enumeration, no invented env var).
|
||||
#
|
||||
# OPERATOR-RATIFIED 2026-06-17 (brief-correction): plan F4 named only json-backcompat +
|
||||
# raw-backcompat (2 files). The verified machine-locked surface is SIX files (the 2 + the 4 that
|
||||
# still fail at clone path). Dropped clean-room SC2 coverage = config-audit's humanizer / posture-
|
||||
# humanizer / scan-orchestrator-humanizer prose-snapshot surface (NOT silent — recorded here +
|
||||
# in plugin-map.json's standalone_caveat). The three OTHER v5.0.0-referencing tests
|
||||
# (posture, scoring-humanizer, scenario-read-test) assert path-INDEPENDENT aspects, pass at the
|
||||
# clone path, and remain IN the gate. config-audit backlog (out of migration scope): normalize
|
||||
# the v5.0.0 fixtures' embedded paths + make the scanners path-agnostic, then re-include.
|
||||
#
|
||||
# SC2 gate (config-audit) := full `find tests -name '*.test.mjs'` MINUS the six machine-locked tests
|
||||
# below, run AFTER the in-clone snapshot re-seed. NULL push (D8). Prints "config-audit: SC2 PASS (...)"
|
||||
# + exit 0 on success; non-zero on any failure (escalate — never mask).
|
||||
#
|
||||
# Usage: 50-config-audit-sc2.sh
|
||||
set -uo pipefail
|
||||
unset NODE_TEST_CONTEXT 2>/dev/null || true # un-nest the internal `node --test` (Node 25 count suppression)
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WORK="${WORK:-/tmp/polyrepo-migration}"
|
||||
KEY="config-audit"
|
||||
DEST="$WORK/$KEY"
|
||||
CR="/tmp/claude-${KEY}-sc2"
|
||||
|
||||
# The machine-locked v5.0.0 byte-stability surface excluded from the SC2 gate (operator-ratified).
|
||||
# Anchored to each test file's basename; scan-orchestrator-humanizer is matched WITHOUT catching the
|
||||
# portable scan-orchestrator.test.mjs, and posture-humanizer WITHOUT catching posture.test.mjs.
|
||||
EXCLUDE_RE='(json-backcompat|raw-backcompat|cli-humanizer|posture-humanizer|scan-orchestrator-humanizer|lint-default-output)\.test\.mjs$'
|
||||
|
||||
# --- 1. Prep the extract (idempotent), reusing the Step 3-5 drivers (same pattern as 40-validate-standalone.sh) ---
|
||||
if [ ! -d "$DEST/.git" ]; then
|
||||
WORK="$WORK" bash "$SCRIPT_DIR/10-extract.sh" "$KEY" >/dev/null || { echo "config-audit: SC2 FAIL (extract error)"; exit 1; }
|
||||
fi
|
||||
WORK="$WORK" bash "$SCRIPT_DIR/20-rehome-config.sh" "$KEY" >/dev/null || { echo "config-audit: SC2 FAIL (rehome error)"; exit 1; }
|
||||
WORK="$WORK" node "$SCRIPT_DIR/30-fix-references.mjs" "$KEY" >/dev/null || { echo "config-audit: SC2 FAIL (fix-references error)"; exit 1; }
|
||||
|
||||
# --- 2. Clean room (no marketplace parent) ---
|
||||
rm -rf "$CR"
|
||||
cp -R "$DEST" "$CR"
|
||||
|
||||
# --- 3. Re-seed the rebasable snapshot at the clone's own path (intended re-approval seam) ---
|
||||
if [ ! -f "$CR/tests/snapshot-default-output.test.mjs" ]; then
|
||||
echo "config-audit: SC2 FAIL (snapshot test missing from extract)"; exit 1
|
||||
fi
|
||||
( cd "$CR" && UPDATE_SNAPSHOT=1 node --test tests/snapshot-default-output.test.mjs ) >/dev/null 2>&1 \
|
||||
|| { echo "config-audit: SC2 FAIL (snapshot re-seed error)"; exit 1; }
|
||||
|
||||
# --- 4. Build the gate: full suite MINUS the machine-locked v5.0.0 back-compat tests (excluded by name) ---
|
||||
GATE_FILES="$(cd "$CR" && find tests -name '*.test.mjs' | grep -vE "$EXCLUDE_RE" | sort)"
|
||||
if [ -z "$GATE_FILES" ]; then echo "config-audit: SC2 FAIL (no gate files enumerated)"; exit 1; fi
|
||||
|
||||
# Defense-in-depth: none of the machine-locked tests may leak into the gate.
|
||||
if printf '%s\n' "$GATE_FILES" | grep -qE "$EXCLUDE_RE"; then
|
||||
echo "config-audit: SC2 FAIL (machine-locked exclusion leaked into the gate)"; exit 1
|
||||
fi
|
||||
GATE_COUNT="$(printf '%s\n' "$GATE_FILES" | wc -l | tr -d '[:space:]')"
|
||||
|
||||
# --- 5. Run the gate in the clean room ---
|
||||
OUT="$(cd "$CR" && node --test $GATE_FILES 2>&1)"; STATUS=$?
|
||||
TESTS="$(printf '%s\n' "$OUT" | grep -oE 'tests [0-9]+' | grep -oE '[0-9]+' | tail -1)"
|
||||
|
||||
if [ "$STATUS" -ne 0 ]; then
|
||||
echo "config-audit: SC2 FAIL (gate exit $STATUS)"
|
||||
printf '%s\n' "$OUT" | tail -25
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "config-audit: SC2 PASS (${TESTS:-?} tests across ${GATE_COUNT} files, full suite minus the 6-file v5.0.0 byte-stability surface, standalone-safe)"
|
||||
exit 0
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
// Step 7 test — the SC2 resolver + generic validator are their own test surface:
|
||||
// 1. The config-audit SC2 gate PASSes a standalone extract (re-seed + back-compat exclusion).
|
||||
// 2. The gate excludes json-backcompat + raw-backcompat by NAME and re-seeds (not excludes) the snapshot.
|
||||
// 3. The generic validator PASSes okr.
|
||||
// 4. The generic validator FAILs a plugin whose plugin.json was deleted.
|
||||
// Pattern: 40-validate-standalone.test.mjs (spawn the script, assert exit code + stdout).
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { mkdtempSync, rmSync, cpSync, readFileSync } from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const here = path.dirname(fileURLToPath(import.meta.url));
|
||||
const REPO = path.resolve(here, '..', '..', '..'); // migration -> marketplace-polyrepo-migration -> docs -> repo root
|
||||
const SC2 = path.join(here, '50-config-audit-sc2.sh');
|
||||
const GENERIC = path.join(here, 'templates', 'validate-plugin.generic.sh');
|
||||
const WORK = process.env.WORK || '/tmp/polyrepo-migration';
|
||||
|
||||
function run(cmd, args, timeout = 600000) {
|
||||
// Strip the parent test-runner context so the script's own `node --test` runs un-nested
|
||||
// (otherwise Node emits its IPC subtest format and the test-count label is suppressed).
|
||||
const env = { ...process.env, WORK };
|
||||
delete env.NODE_TEST_CONTEXT;
|
||||
return spawnSync(cmd, args, { encoding: 'utf8', env, timeout });
|
||||
}
|
||||
|
||||
// The verified machine-locked v5.0.0 byte-stability surface (operator-ratified 2026-06-17):
|
||||
// plan F4 named 2 files; the real surface is these 6. The other v5.0.0-referencing tests
|
||||
// (posture, scoring-humanizer, scenario-read-test) are path-independent and stay IN the gate.
|
||||
const MACHINE_LOCKED = [
|
||||
'json-backcompat', 'raw-backcompat', 'cli-humanizer',
|
||||
'posture-humanizer', 'scan-orchestrator-humanizer', 'lint-default-output',
|
||||
];
|
||||
|
||||
test('config-audit SC2 gate PASSes a standalone extract (re-seed + machine-locked exclusion)', () => {
|
||||
const r = run('bash', [SC2]);
|
||||
assert.equal(r.status, 0, `expected SC2 PASS exit 0:\n${r.stdout}\n${r.stderr}`);
|
||||
assert.match(r.stdout, /config-audit: SC2 PASS/);
|
||||
assert.match(r.stdout, /minus the 6-file v5\.0\.0 byte-stability surface/);
|
||||
});
|
||||
|
||||
test('the SC2 gate excludes the full machine-locked surface by name and re-seeds the snapshot', () => {
|
||||
const src = readFileSync(SC2, 'utf8');
|
||||
for (const name of MACHINE_LOCKED) {
|
||||
assert.ok(src.includes(name), `EXCLUDE_RE must name the machine-locked test: ${name}`);
|
||||
}
|
||||
assert.match(src, /grep -vE "\$EXCLUDE_RE"/,
|
||||
'the gate must filter the enumerated test list through EXCLUDE_RE');
|
||||
assert.match(src, /UPDATE_SNAPSHOT=1/,
|
||||
'the rebasable snapshot must be re-seeded in-clone, never excluded');
|
||||
});
|
||||
|
||||
test('generic validator PASSes okr', () => {
|
||||
const tmp = mkdtempSync(path.join(os.tmpdir(), 'okr-ok-'));
|
||||
try {
|
||||
const root = path.join(tmp, 'okr');
|
||||
cpSync(path.join(REPO, 'plugins', 'okr'), root, { recursive: true });
|
||||
const r = run('bash', [GENERIC, 'okr', root], 60000);
|
||||
assert.equal(r.status, 0, `expected okr STRUCTURE OK exit 0:\n${r.stdout}\n${r.stderr}`);
|
||||
assert.match(r.stdout, /okr: STRUCTURE OK/);
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('generic validator FAILs a plugin with a deleted plugin.json', () => {
|
||||
const tmp = mkdtempSync(path.join(os.tmpdir(), 'okr-broken-'));
|
||||
try {
|
||||
const root = path.join(tmp, 'okr');
|
||||
cpSync(path.join(REPO, 'plugins', 'okr'), root, { recursive: true });
|
||||
rmSync(path.join(root, '.claude-plugin', 'plugin.json'), { force: true });
|
||||
const r = run('bash', [GENERIC, 'okr', root], 60000);
|
||||
assert.notEqual(r.status, 0, `expected STRUCTURE FAIL (non-zero), got 0:\n${r.stdout}`);
|
||||
assert.match(r.stdout, /okr: STRUCTURE FAIL/);
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
|
@ -33,7 +33,8 @@
|
|||
"blob_strip": false,
|
||||
"blob_strip_safe": null,
|
||||
"test_cmd": "node --test 'tests/**/*.test.mjs'",
|
||||
"standalone_caveat": "SC2 gate = full tests/**/*.test.mjs MINUS json-backcompat + raw-backcompat (frozen back-compat snapshots embed the original machine's absolute path + sibling marketplace + deleted plugins); re-seed snapshot-default-output via UPDATE_SNAPSHOT=1 in-clone (Step 7)"
|
||||
"sc2_gate": "50-config-audit-sc2.sh",
|
||||
"standalone_caveat": "SC2 gate = full tests/**/*.test.mjs MINUS the 6-file machine-locked v5.0.0 byte-stability surface (json-backcompat, raw-backcompat, cli-humanizer, posture-humanizer, scan-orchestrator-humanizer, lint-default-output), re-seeding snapshot-default-output via UPDATE_SNAPSHOT=1 in-clone. Operator-ratified 2026-06-17 brief-correction: plan F4 named only 2 files; the verified surface is 6 (the v5.0.0 fixtures embed the original abs path AND the claude_md/plugin_hygiene scanners key off a plugins/ ancestor, so findings drift by path — not a string rewrite). DROPPED clean-room SC2 coverage = config-audit's humanizer/posture-humanizer/scan-orchestrator-humanizer prose-snapshot surface (recorded, not silent). config-audit backlog (out of migration scope): normalize the v5.0.0 fixtures' embedded paths + make scanners path-agnostic, then re-include. The 3 other v5.0.0-referencing tests (posture, scoring-humanizer, scenario-read-test) are path-independent and stay in the gate."
|
||||
},
|
||||
"graceful-handoff": {
|
||||
"paths": ["plugins/graceful-handoff/"],
|
||||
|
|
|
|||
|
|
@ -0,0 +1,104 @@
|
|||
#!/usr/bin/env bash
|
||||
# validate-plugin.generic.sh — generic, parameterized plugin structure validator (Step 7).
|
||||
#
|
||||
# A ported, plugin-agnostic copy of plugins/claude-design/tests/validate-plugin.sh, stripped of
|
||||
# claude-design's bespoke checks (.coverage.md, SKILL description length, forbidden command names,
|
||||
# operator-private grep). It gives the two test-less plugins — okr + human-friendly-style — a runnable
|
||||
# SC2 gate for the standalone-extraction harness (40-validate-standalone.sh vendors this file into the
|
||||
# clean room and runs `bash validate-plugin.generic.sh <key>`).
|
||||
#
|
||||
# Checks (all generic):
|
||||
# (a) .claude-plugin/plugin.json is valid JSON with non-empty name/version/description [HARD]
|
||||
# (b) at least one user-facing surface dir is present + non-empty
|
||||
# (commands | agents | skills | output-styles | hooks) [HARD]
|
||||
# (c) every commands/*.md, agents/*.md, output-styles/*.md and skills/*/SKILL.md parses:
|
||||
# line 1 is the `---` frontmatter delimiter and the block carries a `name:` key [HARD]
|
||||
#
|
||||
# Usage: validate-plugin.generic.sh <key> [plugin_root]
|
||||
# <key> label used in the verdict line (e.g. "okr: STRUCTURE OK")
|
||||
# [plugin_root] plugin root to validate; defaults to $PWD (the clean-room cwd)
|
||||
#
|
||||
# Exit codes: 0 = STRUCTURE OK; 1 = STRUCTURE FAIL (>=1 hard check failed); 2 = usage error.
|
||||
set -uo pipefail
|
||||
|
||||
KEY="${1:-}"
|
||||
[ -n "$KEY" ] || { echo "usage: validate-plugin.generic.sh <key> [plugin_root]" >&2; exit 2; }
|
||||
PLUGIN_ROOT="${2:-$PWD}"
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
pass() { printf ' + %s\n' "$1"; PASS=$((PASS + 1)); }
|
||||
fail() { printf ' - %s\n' "$1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
echo "=== $KEY structure validation (root: $PLUGIN_ROOT) ==="
|
||||
|
||||
# --- (a) plugin.json valid JSON + required fields ---
|
||||
echo "--- (a) plugin.json ---"
|
||||
PJ=""
|
||||
for cand in "$PLUGIN_ROOT/.claude-plugin/plugin.json" "$PLUGIN_ROOT/plugin.json"; do
|
||||
[ -f "$cand" ] && { PJ="$cand"; break; }
|
||||
done
|
||||
if [ -z "$PJ" ]; then
|
||||
fail "plugin.json missing (.claude-plugin/plugin.json)"
|
||||
elif ! node -e "JSON.parse(require('fs').readFileSync(process.argv[1],'utf8'))" "$PJ" 2>/dev/null; then
|
||||
fail "plugin.json is invalid JSON"
|
||||
else
|
||||
pass "plugin.json is valid JSON"
|
||||
for field in name version description; do
|
||||
if node -e "const p=JSON.parse(require('fs').readFileSync(process.argv[1],'utf8'));if(typeof p[process.argv[2]]!=='string'||p[process.argv[2]]==='')process.exit(1)" "$PJ" "$field" 2>/dev/null; then
|
||||
pass "plugin.json has '$field'"
|
||||
else
|
||||
fail "plugin.json missing or empty '$field'"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# --- (b) at least one non-empty user-facing surface dir ---
|
||||
echo "--- (b) user-facing surface ---"
|
||||
SURFACE=0
|
||||
for d in commands agents skills output-styles hooks; do
|
||||
dir="$PLUGIN_ROOT/$d"
|
||||
[ -d "$dir" ] || continue
|
||||
if find "$dir" -type f 2>/dev/null | grep -q .; then
|
||||
pass "surface dir '$d/' present and non-empty"
|
||||
SURFACE=$((SURFACE + 1))
|
||||
fi
|
||||
done
|
||||
if [ "$SURFACE" -eq 0 ]; then
|
||||
fail "no non-empty user-facing surface dir (commands/agents/skills/output-styles/hooks)"
|
||||
fi
|
||||
|
||||
# --- (c) frontmatter parses on every surface markdown file ---
|
||||
echo "--- (c) frontmatter ---"
|
||||
check_frontmatter() {
|
||||
local f="$1"
|
||||
local rel="${f#"$PLUGIN_ROOT"/}"
|
||||
if [ "$(head -n 1 "$f")" != "---" ]; then
|
||||
fail "$rel: missing frontmatter delimiter on line 1"
|
||||
return
|
||||
fi
|
||||
local fm
|
||||
fm="$(awk 'NR==1{next} /^---$/{exit} {print}' "$f")"
|
||||
if printf '%s\n' "$fm" | grep -qE '^name:'; then
|
||||
pass "$rel: frontmatter has 'name:'"
|
||||
else
|
||||
fail "$rel: frontmatter missing 'name:'"
|
||||
fi
|
||||
}
|
||||
FM_SEEN=0
|
||||
for f in "$PLUGIN_ROOT"/commands/*.md "$PLUGIN_ROOT"/agents/*.md \
|
||||
"$PLUGIN_ROOT"/output-styles/*.md "$PLUGIN_ROOT"/skills/*/SKILL.md; do
|
||||
[ -f "$f" ] || continue
|
||||
FM_SEEN=$((FM_SEEN + 1))
|
||||
check_frontmatter "$f"
|
||||
done
|
||||
[ "$FM_SEEN" -eq 0 ] && pass "no frontmatter-bearing surface files to check"
|
||||
|
||||
echo "=== Summary: pass=$PASS fail=$FAIL ==="
|
||||
if [ "$FAIL" -gt 0 ]; then
|
||||
echo "$KEY: STRUCTURE FAIL"
|
||||
exit 1
|
||||
fi
|
||||
echo "$KEY: STRUCTURE OK"
|
||||
exit 0
|
||||
Loading…
Add table
Add a link
Reference in a new issue