Kjell Tore Guttormsen
3403648c6c
The operator window's step [a] called 40-validate-standalone.sh directly (strict exit-code), so it STOPped on the first target carrying pre-existing in-repo test red — voyage (2 doc-consistency drifts re phase_models/phase_signals, content moved to docs/operations.md) and ai-psychosis (1). But the migration's ratified contract, the one the Step-11 dry-run validated as PASS 11/11, is 'introduce no regression': pre-existing in-repo red is the plugin's own concern, not a migration regression. The window enforced a STRICTER gate than the contract the dry-run signed off. Fix: new 41-validate-or-regression.sh — the single per-target gate the window calls in [a]. It runs 40 strict, then on failure passes iff the standalone failing-test NAME set is a SUBSET of the live in-repo set (the exact decision 99-dryrun.sh makes), reusing sc2-regression.sh. A genuine extraction-introduced regression still STOPs the window; a structure-validator fail and the config-audit gate stay strict. Single-source the failing-name capture: extract capture_fails into capture-fails.sh (mirrors the sc2-regression.sh extraction) so the live gate and the dry-run agree on what 'failing' means; 99-dryrun.sh now delegates to it (behaviour identical). Verified end-to-end on the real extracts: 40 strict fails voyage+ai-psychosis while 41 passes them 'N pre-existing, regression-relative'; clean targets (llm-security, graceful-handoff) still pass via the strict path. New hermetic tests: capture-fails 3/3, 41 6/6 (strict-pass, regression-relative-pass, genuine-regression-fail, structure-not-eligible, gate pass/fail). RUNBOOK per-repo step updated to 41. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
20 lines
1.2 KiB
Bash
Executable file
20 lines
1.2 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Failing-test NAME set for a node:test suite, via the stable TAP reporter (locale-independent).
|
|
# Extracted from 99-dryrun.sh's inline capture_fails (mirrors the sc2-regression.sh extraction, 5d112cb) so
|
|
# the SAME capture feeds BOTH the dry-run rehearsal (99-dryrun.sh) and the live operator-window gate
|
|
# (41-validate-or-regression.sh) — a single source of truth prevents the gate and the rehearsal from
|
|
# disagreeing on what "failing" means (the exact class of drift that let the strict window STOP on red the
|
|
# dry-run had blessed).
|
|
#
|
|
# Prints sorted-unique failing test names, one per line. Empty output = the suite reported no `not ok` lines.
|
|
# The caller distinguishes "ran clean" from "did not run": sc2-regression.sh treats a missing capture FILE
|
|
# as a hard error, but an empty-yet-present file is a legitimate zero-failure set. No pipefail: a no-match
|
|
# grep (zero failures) must exit 0, not leak a spurious non-zero up to the caller.
|
|
#
|
|
# Usage: capture-fails.sh <dir> <node --test command>
|
|
set -u
|
|
dir="${1:?dir}"
|
|
cmd="${2:?node --test command}"
|
|
tapcmd="${cmd/node --test/node --test --test-reporter=tap}"
|
|
( cd "$dir" && eval "$tapcmd" ) 2>&1 \
|
|
| grep -E '^not ok ' | sed -E 's/^not ok [0-9]+ - //; s/ #.*$//' | sort -u
|