Kjell Tore Guttormsen
3403648c6c
The operator window's step [a] called 40-validate-standalone.sh directly (strict exit-code), so it STOPped on the first target carrying pre-existing in-repo test red — voyage (2 doc-consistency drifts re phase_models/phase_signals, content moved to docs/operations.md) and ai-psychosis (1). But the migration's ratified contract, the one the Step-11 dry-run validated as PASS 11/11, is 'introduce no regression': pre-existing in-repo red is the plugin's own concern, not a migration regression. The window enforced a STRICTER gate than the contract the dry-run signed off. Fix: new 41-validate-or-regression.sh — the single per-target gate the window calls in [a]. It runs 40 strict, then on failure passes iff the standalone failing-test NAME set is a SUBSET of the live in-repo set (the exact decision 99-dryrun.sh makes), reusing sc2-regression.sh. A genuine extraction-introduced regression still STOPs the window; a structure-validator fail and the config-audit gate stay strict. Single-source the failing-name capture: extract capture_fails into capture-fails.sh (mirrors the sc2-regression.sh extraction) so the live gate and the dry-run agree on what 'failing' means; 99-dryrun.sh now delegates to it (behaviour identical). Verified end-to-end on the real extracts: 40 strict fails voyage+ai-psychosis while 41 passes them 'N pre-existing, regression-relative'; clean targets (llm-security, graceful-handoff) still pass via the strict path. New hermetic tests: capture-fails 3/3, 41 6/6 (strict-pass, regression-relative-pass, genuine-regression-fail, structure-not-eligible, gate pass/fail). RUNBOOK per-repo step updated to 41. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
84 lines
4.4 KiB
Bash
84 lines
4.4 KiB
Bash
#!/usr/bin/env bash
|
|
# Step 6b — per-target SC2/SC7 gate with the migration's REGRESSION-RELATIVE contract baked in.
|
|
#
|
|
# WHY THIS EXISTS: 40-validate-standalone.sh is STRICT — any standalone test failure exits non-zero. The
|
|
# migration's RATIFIED contract — the one the Step-11 dry-run (99-dryrun.sh) validated and signed off as
|
|
# "PASS 11/11" — is weaker and correct: a target passes iff the extraction introduces NO NEW failure.
|
|
# Pre-existing in-repo red (voyage's 2 doc-consistency drifts re phase_models/phase_signals; ai-psychosis's
|
|
# 1) is the PLUGIN's own concern, not a migration regression. The operator window (run-operator-window.sh
|
|
# step [a]) must enforce THAT contract, not a stricter one — otherwise it STOPs on the first target carrying
|
|
# pre-existing red even though the dry-run blessed it (the bug this script fixes). This is the single
|
|
# per-target gate the window calls; it mirrors 99-dryrun.sh's SC2 decision exactly, reusing capture-fails.sh
|
|
# (the failing-name capture) + sc2-regression.sh (the subset decision). NULL push (D8): read-only validation.
|
|
#
|
|
# CONTRACT:
|
|
# - sc2_gate target (config-audit): the dedicated gate is deterministic — run it STRICT (PASS/FAIL).
|
|
# - else: run 40-validate-standalone.sh STRICT. PASS => exit 0.
|
|
# On FAIL, only a `node --test` suite is regression-eligible: its standalone failing-NAME set must be
|
|
# a SUBSET of the live in-repo failing-NAME set (regression-relative PASS, "PASS (N pre-existing)").
|
|
# A structure-validator (validate-plugin*.sh / bash -c) FAIL is a genuine structural defect — NOT
|
|
# softened. A genuine regression (standalone-only failure) or a missing capture => exit non-zero.
|
|
# Escalate, never mask: any real regression or structural failure exits non-zero so the window STOPs.
|
|
#
|
|
# Usage: 41-validate-or-regression.sh <target-key>
|
|
set -uo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
MAP="$SCRIPT_DIR/plugin-map.json"
|
|
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
|
WORK="${WORK:-/tmp/polyrepo-migration}"
|
|
|
|
[ -f "$MAP" ] || { echo "plugin-map.json missing at $MAP" >&2; exit 2; }
|
|
key="${1:?usage: 41-validate-or-regression.sh <target-key>}"
|
|
|
|
mapget() { python3 -c "import json;print(json.load(open('$MAP'))['targets']['$key'].get('$1',''))"; }
|
|
|
|
sc2_gate="$(mapget sc2_gate)"
|
|
test_cmd="$(mapget test_cmd)"
|
|
|
|
# --- gate target (config-audit): deterministic, strict by design ---
|
|
if [ -n "$sc2_gate" ]; then
|
|
if WORK="$WORK" bash "$SCRIPT_DIR/$sc2_gate" >/dev/null 2>&1; then
|
|
echo "$key: PASS ($sc2_gate)"; exit 0
|
|
fi
|
|
echo "$key: FAIL ($sc2_gate)" >&2; exit 1
|
|
fi
|
|
|
|
# --- strict standalone validation first (SC2 + SC7 + per-target structural checks) ---
|
|
if WORK="$WORK" bash "$SCRIPT_DIR/40-validate-standalone.sh" "$key" >/dev/null 2>&1; then
|
|
echo "$key: PASS (standalone strict)"; exit 0
|
|
fi
|
|
|
|
# Strict failed. Only a node:test suite can fail regression-relative-acceptably; a structure validator
|
|
# failing is a genuine structural defect — do not soften it.
|
|
case "$test_cmd" in
|
|
*node\ --test*) : ;;
|
|
*) echo "$key: FAIL (standalone strict; structure validator — not regression-eligible)" >&2; exit 1 ;;
|
|
esac
|
|
|
|
dest="$WORK/$key"
|
|
[ -d "$dest/.git" ] || { echo "$key: FAIL (no prepped extract at $dest — run 40-validate-standalone.sh first)" >&2; exit 1; }
|
|
|
|
# Regression-relative: the standalone failing-NAME set must be a SUBSET of the live in-repo failing-NAME set.
|
|
# Capture the standalone set from the PREPPED EXTRACT ($WORK/$key), NOT 40's /tmp/claude-$key side-effect
|
|
# clean room (4e494c8: that coupling masked a real regression when the dir was absent). The subset decision
|
|
# is delegated to sc2-regression.sh; a missing capture FILE there is a hard error, never a false PASS.
|
|
sf="$(mktemp)" && bf="$(mktemp)" || { echo "$key: FAIL (mktemp)" >&2; exit 1; }
|
|
trap 'rm -f "$sf" "$bf"' EXIT
|
|
bash "$SCRIPT_DIR/capture-fails.sh" "$dest" "$test_cmd" > "$sf" 2>/dev/null
|
|
bash "$SCRIPT_DIR/capture-fails.sh" "$REPO_ROOT/plugins/$key" "$test_cmd" > "$bf" 2>/dev/null
|
|
pre="$(wc -l < "$bf" | tr -d '[:space:]')"
|
|
|
|
if regr="$(bash "$SCRIPT_DIR/sc2-regression.sh" "$sf" "$bf")"; then
|
|
echo "$key: PASS (${pre} pre-existing, regression-relative)"
|
|
exit 0
|
|
else
|
|
rc=$?
|
|
if [ "$rc" -ge 2 ]; then
|
|
echo "$key: FAIL (sc2-regression error — missing capture file)" >&2
|
|
else
|
|
echo "$key: FAIL (regression: $(printf '%s' "$regr" | grep -c .) new failure(s) absent from in-repo baseline):" >&2
|
|
printf '%s\n' "$regr" >&2
|
|
fi
|
|
exit 1
|
|
fi
|