Kjell Tore Guttormsen
3403648c6c
The operator window's step [a] called 40-validate-standalone.sh directly (strict exit-code), so it STOPped on the first target carrying pre-existing in-repo test red — voyage (2 doc-consistency drifts re phase_models/phase_signals, content moved to docs/operations.md) and ai-psychosis (1). But the migration's ratified contract, the one the Step-11 dry-run validated as PASS 11/11, is 'introduce no regression': pre-existing in-repo red is the plugin's own concern, not a migration regression. The window enforced a STRICTER gate than the contract the dry-run signed off. Fix: new 41-validate-or-regression.sh — the single per-target gate the window calls in [a]. It runs 40 strict, then on failure passes iff the standalone failing-test NAME set is a SUBSET of the live in-repo set (the exact decision 99-dryrun.sh makes), reusing sc2-regression.sh. A genuine extraction-introduced regression still STOPs the window; a structure-validator fail and the config-audit gate stay strict. Single-source the failing-name capture: extract capture_fails into capture-fails.sh (mirrors the sc2-regression.sh extraction) so the live gate and the dry-run agree on what 'failing' means; 99-dryrun.sh now delegates to it (behaviour identical). Verified end-to-end on the real extracts: 40 strict fails voyage+ai-psychosis while 41 passes them 'N pre-existing, regression-relative'; clean targets (llm-security, graceful-handoff) still pass via the strict path. New hermetic tests: capture-fails 3/3, 41 6/6 (strict-pass, regression-relative-pass, genuine-regression-fail, structure-not-eligible, gate pass/fail). RUNBOOK per-repo step updated to 41. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
252 lines
13 KiB
Bash
252 lines
13 KiB
Bash
#!/usr/bin/env bash
|
|
# Step 11 — Full local dry-run (all 11 targets, NULL push).
|
|
#
|
|
# The single end-to-end rehearsal of the Claude-run local half. For all 11 targets (10 plugins + the
|
|
# design-system) it FORCE-FRESH re-extracts (Steps 3->4->5: never reuses a possibly-stale $WORK/<key>),
|
|
# then validates each three ways:
|
|
# SC6 (content retention) — extract git-tracked file count vs the live monorepo's tracked count for the
|
|
# target's paths. A drop (not explained by an intentional blob-strip) is a hard FAIL: this is the
|
|
# check that catches the llm-security dual-rename class of defect (87 files dropped) deterministically.
|
|
# SC7 (no state-file leak) — no tracked STATE.md / *.local.md (except the okr template).
|
|
# SC2 (no test REGRESSION) — config-audit routes to its dedicated Step-7 gate (50-config-audit-sc2.sh);
|
|
# every other target runs its declared standalone runner via 40-validate-standalone.sh. If the
|
|
# standalone runner fails, the gate is REGRESSION-RELATIVE: it re-runs the same suite in the live
|
|
# monorepo and PASSES iff the standalone failing-test set is a SUBSET of the in-repo failing set
|
|
# (i.e. the extraction introduced NO new failures). Pre-existing in-repo red and environmental
|
|
# flake are the plugin's own concern, NOT a migration regression, and are reported transparently
|
|
# (never masked) — the migration's contract is "introduce no regression".
|
|
# Then it exercises Step 8's marketplace rewriter (--all -> --out) and Step 9's catalog thinning
|
|
# (--workspace), and runs the cross-repo DS-vendor chain (M9 / SC5): sync playground-design-system
|
|
# (--source) into ms-ai-architect (--target), then --check for byte-parity. It writes
|
|
# $WORK/dryrun-report.md (an OUTPUT artifact, never committed).
|
|
#
|
|
# Reuses the dedicated `git clone --no-local` extraction mirror ($WORK/_mirror, R4/M6) so extraction never
|
|
# reads the working checkout. NULL push (D8): zero pushes, zero live-file mutation — the live plugins/,
|
|
# shared/, and .claude-plugin/marketplace.json are never touched; every write lands in $WORK. Honest scope
|
|
# (M8): this proves the LOCAL mechanics only; the window-only steps (Forgejo accepting post-strip history,
|
|
# auto_init:false API repo creation, Claude Code resolving an HTTPS url+ref source) are gated by the
|
|
# Step 10 pilot, NOT by this dry-run.
|
|
#
|
|
# On any hard FAIL (SC6 drop, SC7 leak, or an SC2 regression): exit non-zero (escalate — never mask a
|
|
# real pre-window blocker).
|
|
#
|
|
# Usage: 99-dryrun.sh (override WORK= to relocate the workspace; default /tmp/polyrepo-migration)
|
|
set -uo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
MAP="$SCRIPT_DIR/plugin-map.json"
|
|
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
|
WORK="${WORK:-/tmp/polyrepo-migration}"
|
|
MIRROR="$WORK/_mirror"
|
|
REPORT="$WORK/dryrun-report.md"
|
|
MP_PREVIEW="$WORK/marketplace.all-external.json"
|
|
THIN_WS="$WORK/_thin-catalog"
|
|
SYNC="$REPO_ROOT/scripts/sync-design-system.mjs"
|
|
|
|
fail() { printf 'DRY-RUN FAIL: %s\n' "$*" >&2; exit 1; }
|
|
|
|
mapget() { python3 -c "import json;print(json.load(open('$MAP'))['targets']['$1'].get('$2',''))"; }
|
|
mappaths() { python3 -c "import json;print(' '.join(json.load(open('$MAP'))['targets']['$1']['paths']))"; }
|
|
|
|
# Live monorepo git-tracked file count across a target's source paths (SC6 baseline).
|
|
# Paths are asserted whitespace- and glob-free by 00-preflight.sh (aeb6292), so the word-split below is sound.
|
|
live_files() {
|
|
local key="$1" total=0 p n
|
|
for p in $(mappaths "$key"); do
|
|
n="$(git -C "$REPO_ROOT" ls-files "$p" | wc -l | tr -d '[:space:]')"
|
|
total=$((total + n))
|
|
done
|
|
echo "$total"
|
|
}
|
|
|
|
# Failing-test NAME set for a node:test suite — delegated to capture-fails.sh so the SAME capture feeds both
|
|
# this rehearsal and the live operator-window gate (41-validate-or-regression.sh). $1 = dir, $2 = node --test cmd.
|
|
capture_fails() { bash "$SCRIPT_DIR/capture-fails.sh" "$1" "$2"; }
|
|
|
|
[ -f "$MAP" ] || fail "plugin-map.json missing at $MAP"
|
|
mkdir -p "$WORK"
|
|
|
|
KEYS="$(python3 -c "import json;print('\n'.join(sorted(json.load(open('$MAP'))['targets'])))")" \
|
|
|| fail "plugin-map.json does not parse"
|
|
TARGET_COUNT="$(printf '%s\n' "$KEYS" | grep -c .)"
|
|
[ "$TARGET_COUNT" = "11" ] || fail "expected 11 targets, found $TARGET_COUNT"
|
|
|
|
# Ensure the dedicated --no-local mirror exists (R4/M6); extraction never reads the working checkout.
|
|
if [ ! -e "$MIRROR/HEAD" ] && [ ! -d "$MIRROR/.git" ]; then
|
|
echo " mirror absent -> running 00-preflight.sh (builds the --no-local extraction mirror)"
|
|
WORK="$WORK" bash "$SCRIPT_DIR/00-preflight.sh" >/dev/null || fail "preflight (mirror build) failed"
|
|
fi
|
|
|
|
{
|
|
echo "# Polyrepo Migration — Local Dry-Run Report"
|
|
echo
|
|
echo "> Output artifact (NOT committed). Workspace: \`$WORK\`. Force-fresh extraction (no stale reuse)."
|
|
echo "> **NULL push (D8):** 0 pushes, 0 live-file mutation (plugins/, shared/, marketplace.json untouched)."
|
|
echo "> Extraction reuses the dedicated \`git clone --no-local\` mirror (R4/M6) — never the working checkout."
|
|
echo "> **SC2 is regression-relative:** a target passes iff the extraction introduces NO test failure that"
|
|
echo "> does not also occur in the live monorepo. Pre-existing in-repo red / flake is reported, never masked."
|
|
echo
|
|
echo "## Per-target — force-fresh extraction + standalone validation (Steps 3→4→5, +6/+7)"
|
|
echo
|
|
echo "| target | commits | files ext/live | tag | SC2 | SC7 |"
|
|
echo "|--------|--------:|----------------|-----|-----|-----|"
|
|
} > "$REPORT"
|
|
|
|
PASSED=0
|
|
for key in $KEYS; do
|
|
dest="$WORK/$key"
|
|
ok=1
|
|
|
|
# --- force-fresh extract + prep (Steps 3->4->5; 10-extract wipes $dest first) ---
|
|
WORK="$WORK" bash "$SCRIPT_DIR/10-extract.sh" "$key" >/dev/null 2>&1 || ok=0
|
|
WORK="$WORK" bash "$SCRIPT_DIR/20-rehome-config.sh" "$key" >/dev/null 2>&1 || ok=0
|
|
WORK="$WORK" node "$SCRIPT_DIR/30-fix-references.mjs" "$key" >/dev/null 2>&1 || ok=0
|
|
|
|
# --- report data + SC6 + SC7 from the fresh extract ---
|
|
commits=0; tag="(none)"; ext_files=0; sc7="clean"; sc6="?"
|
|
if [ -d "$dest/.git" ]; then
|
|
commits="$(git -C "$dest" rev-list --count HEAD 2>/dev/null || echo 0)"
|
|
# F5 invariant: 10-extract.sh strips all carried-over tags and re-creates EXACTLY one. Assert that —
|
|
# a partial tag-strip leaves multiple tags, and `head -1` would silently report the lexicographically
|
|
# -first wrong one (e.g. v1.0.0 instead of v5.1.1) while the dry-run still passed (1708e90).
|
|
tagn="$(git -C "$dest" tag 2>/dev/null | wc -l | tr -d '[:space:]')"
|
|
if [ "$tagn" = "1" ]; then tag="$(git -C "$dest" tag 2>/dev/null)"; else tag="(tags=$tagn!)"; ok=0; fi
|
|
ext_files="$(git -C "$dest" ls-files | wc -l | tr -d '[:space:]')"
|
|
leak="$(git -C "$dest" ls-files | grep -E 'STATE\.md|\.local\.md$' \
|
|
| grep -v 'templates/okr\.local\.md\.template' || true)"
|
|
[ -z "$leak" ] || { sc7="LEAK: $(printf '%s' "$leak" | tr '\n' ' ')"; ok=0; }
|
|
else
|
|
ok=0
|
|
fi
|
|
|
|
# SC6 content retention — delegated to sc6-check.sh (the DROP detector, unit-tested in sc-checks.test.mjs).
|
|
# Only meaningful when the extract succeeded: a failed extract leaves ext_files=0, which must NOT be
|
|
# mislabelled as a content DROP (8d649e9) — emit (extract failed) instead. An intentional >1MB blob-strip
|
|
# target may legitimately shed blobs (reported, not failed); a shortfall elsewhere is a hard DROP (ok=0).
|
|
lf="$(live_files "$key")"
|
|
blob="$(mapget "$key" blob_strip)"
|
|
if [ -d "$dest/.git" ]; then
|
|
if sc6="$(bash "$SCRIPT_DIR/sc6-check.sh" "$ext_files" "$lf" "$blob")"; then :; else ok=0; fi
|
|
else
|
|
sc6="(extract failed)"
|
|
fi
|
|
|
|
# --- SC2 (regression-relative) ---
|
|
sc2_gate="$(mapget "$key" sc2_gate)"
|
|
test_cmd="$(mapget "$key" test_cmd)"
|
|
if [ -n "$sc2_gate" ]; then
|
|
WORK="$WORK" bash "$SCRIPT_DIR/$sc2_gate" >/dev/null 2>&1 && sc2="PASS" || { sc2="FAIL"; ok=0; }
|
|
else
|
|
if WORK="$WORK" bash "$SCRIPT_DIR/40-validate-standalone.sh" "$key" >/dev/null 2>&1; then
|
|
sc2="PASS"
|
|
else
|
|
case "$test_cmd" in
|
|
*node\ --test*)
|
|
# Regression-relative: the standalone failing set must be a SUBSET of the in-repo failing set.
|
|
# Capture the standalone set from the dry-run's OWN prepped extract ($dest = $WORK/$key), NOT the
|
|
# 40-validate side-effect clean room /tmp/claude-$key (4e494c8 — that implicit coupling masked a
|
|
# real regression when the dir was absent). Guard mktemp: an empty capture is an ERROR, never a
|
|
# false zero-regression PASS (4044c49). The subset decision is delegated to sc2-regression.sh.
|
|
if sf="$(mktemp)" && bf="$(mktemp)"; then
|
|
capture_fails "$dest" "$test_cmd" > "$sf" 2>/dev/null
|
|
capture_fails "$REPO_ROOT/plugins/$key" "$test_cmd" > "$bf" 2>/dev/null
|
|
pre="$(wc -l < "$bf" | tr -d '[:space:]')"
|
|
if regr="$(bash "$SCRIPT_DIR/sc2-regression.sh" "$sf" "$bf")"; then
|
|
sc2="PASS (${pre} pre-existing)"
|
|
else
|
|
rc=$?
|
|
if [ "$rc" -ge 2 ]; then sc2="FAIL (sc2-regression error)"; else
|
|
sc2="FAIL (regression: $(printf '%s' "$regr" | grep -c .))"; fi
|
|
ok=0
|
|
fi
|
|
rm -f "$sf" "$bf"
|
|
else
|
|
sc2="FAIL (mktemp)"; ok=0; rm -f "$sf" "$bf"
|
|
fi
|
|
;;
|
|
*) sc2="FAIL (structure)"; ok=0 ;; # deterministic structure check genuinely failed
|
|
esac
|
|
fi
|
|
fi
|
|
|
|
printf '| %s | %s | %s | %s | %s | %s |\n' "$key" "$commits" "$sc6" "$tag" "$sc2" "$sc7" >> "$REPORT"
|
|
echo " $key: commits=$commits files=$sc6 sc2=$sc2 sc7=${sc7%%:*}"
|
|
[ "$ok" -eq 1 ] && PASSED=$((PASSED+1))
|
|
done
|
|
|
|
# --- Step 8: all-external marketplace.json preview (F2 — HTTPS + ref, zero local) ---
|
|
if node "$SCRIPT_DIR/60-rewrite-marketplace.mjs" --all --out "$MP_PREVIEW" >/dev/null 2>&1; then
|
|
LOCAL_REFS="$(grep -c '\./plugins/' "$MP_PREVIEW" 2>/dev/null || true)"; LOCAL_REFS="${LOCAL_REFS:-0}"
|
|
SSH_REFS="$(grep -c 'ssh://' "$MP_PREVIEW" 2>/dev/null || true)"; SSH_REFS="${SSH_REFS:-0}"
|
|
EXT_REFS="$(grep -c '"source": "url"' "$MP_PREVIEW" 2>/dev/null || true)"; EXT_REFS="${EXT_REFS:-0}"
|
|
else
|
|
LOCAL_REFS="?"; SSH_REFS="?"; EXT_REFS="?"
|
|
fi
|
|
|
|
# --- Step 9: thin-catalog preview (SC4 — plugins/ & shared/ removed, CONVENTIONS.md extracted) ---
|
|
THIN_OK="yes"
|
|
if bash "$SCRIPT_DIR/70-thin-catalog.sh" --workspace "$THIN_WS" >/dev/null 2>&1; then
|
|
[ -d "$THIN_WS/plugins" ] && THIN_OK="no (plugins/ remain)"
|
|
[ -d "$THIN_WS/shared" ] && THIN_OK="no (shared/ remain)"
|
|
[ -f "$THIN_WS/CONVENTIONS.md" ] || THIN_OK="no (CONVENTIONS.md missing)"
|
|
else
|
|
THIN_OK="no (script error)"
|
|
fi
|
|
|
|
# --- M9 / SC5: cross-repo DS-vendor chain (the gap the pilot can't cover) ---
|
|
DS="$WORK/playground-design-system"; MSA="$WORK/ms-ai-architect"
|
|
DS_VENDOR="skipped (DS or ms-ai-architect extract missing)"
|
|
if [ -d "$DS/.git" ] && [ -d "$MSA/.git" ] && [ -f "$SYNC" ]; then
|
|
if node "$SYNC" ms-ai-architect --source "$DS" --target "$MSA" >/dev/null 2>&1 \
|
|
|| node "$SYNC" ms-ai-architect --source "$DS" --target "$MSA" --force >/dev/null 2>&1; then
|
|
CHK="$(node "$SYNC" ms-ai-architect --source "$DS" --target "$MSA" --check 2>&1)"
|
|
if [ $? -eq 0 ]; then DS_VENDOR="OK — $CHK"; else DS_VENDOR="DRIFT — $CHK"; fi
|
|
else
|
|
DS_VENDOR="sync failed"
|
|
fi
|
|
fi
|
|
|
|
ALL_OK=1
|
|
[ "$PASSED" = "11" ] || ALL_OK=0
|
|
[ "$LOCAL_REFS" = "0" ] || ALL_OK=0
|
|
[ "$SSH_REFS" = "0" ] || ALL_OK=0
|
|
[ "$THIN_OK" = "yes" ] || ALL_OK=0
|
|
case "$DS_VENDOR" in OK*) : ;; *) ALL_OK=0 ;; esac
|
|
|
|
{
|
|
echo
|
|
echo "## marketplace.json preview — all external (Step 8, F2)"
|
|
echo
|
|
echo "- out: \`$MP_PREVIEW\` · local \`./plugins/\`: $LOCAL_REFS (want 0) · \`ssh://\`: $SSH_REFS (want 0) · external (https+ref): $EXT_REFS"
|
|
echo
|
|
echo "## thin-catalog preview (Step 9, SC4)"
|
|
echo
|
|
echo "- workspace: \`$THIN_WS\` · plugins/ & shared/ removed + CONVENTIONS.md extracted: $THIN_OK"
|
|
echo
|
|
echo "## DS-vendor cross-repo validation (M9 / SC5)"
|
|
echo
|
|
echo "- \`playground-design-system\` (--source) → \`ms-ai-architect\` (--target) → \`--check\`: $DS_VENDOR"
|
|
echo
|
|
echo "## Honest scope of this dry-run (M8)"
|
|
echo
|
|
echo "Proves the **local mechanics**: force-fresh rename-aware extraction + content retention (SC6),"
|
|
echo "re-rooting, reference rewrites, regression-relative standalone validation (SC2/SC7), the"
|
|
echo "marketplace/catalog transforms, and the DS-vendor chain. It does **not** prove the **window-only**"
|
|
echo "steps (Forgejo push of post-strip history, \`auto_init:false\` API repo creation, Claude Code"
|
|
echo "resolving an HTTPS \`url\`+\`ref\` source). Those are gated by the **Step 10 pilot**, not this dry-run."
|
|
echo
|
|
if [ "$ALL_OK" = "1" ]; then
|
|
echo "**DRY-RUN OK — $PASSED/11 targets extracted + validated, 0 pushes**"
|
|
else
|
|
echo "**DRY-RUN INCOMPLETE — $PASSED/11 targets passed; inspect the rows above**"
|
|
fi
|
|
} >> "$REPORT"
|
|
|
|
echo " report: $REPORT"
|
|
if [ "$ALL_OK" = "1" ]; then
|
|
echo "DRY-RUN OK — 11/11 targets extracted + validated, 0 pushes"
|
|
exit 0
|
|
else
|
|
echo "DRY-RUN INCOMPLETE — $PASSED/11 targets passed; see $REPORT" >&2
|
|
exit 1
|
|
fi
|