repo-mailbox/scripts/plan-column-eval.sh
Kjell Tore Guttormsen 20fcd67f4a test(board): red eval for the PLAN column, before the column exists
scripts/plan-column-eval.sh is the gate for a display-only PLAN column
(operator decision 2026-09-17): '-' without a root PLAN.md, otherwise the
whole days PLAN.md's mtime lies behind the last commit.

Seven rows: three fixtures with three different values ('-', '0d', and an
Nd computed from the fixture's own timestamps); docs/PLAN.md and a lowercase
plan.md read '-' (the case trap is asserted armed on APFS, so the byte-exact
name check is real); --row carries plan= equal to the table cell; the column
is kept out of the sort, the --plan blocks and SUM (two repos whose PLAN
values are swapped between runs must keep their order); a replay with many
commits, the destination in docs/*.local.md and a release tag reads '-' at
every step; every scanned repo has a value; the legend says the column does
not judge whether a criterion can be falsified and that touch/checkout reset
the mtime.

Kept outside the five suites and npm test on purpose: committed red checks
inside board-selftest.sh would turn main red and break the README count
check. The build session moves them in once green.

Today: plan-column-eval 9 passed, 24 failed (fixture ground truth green,
every column check red because the column is missing). Suites unchanged:
coord 257, board 427, route 73, orders 116, guard 54; npm test 12/12.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 15:36:06 +02:00

232 lines
12 KiB
Bash
Executable file

#!/bin/bash
# plan-column-eval.sh - the RED gate for board.sh's PLAN column, written
# before the column exists. It is NOT one of the five selftest suites and is
# deliberately not wired into `npm test`: a committed red check inside
# board-selftest.sh would turn the whole suite red on main and break the
# README number check, so "red for the right reason" could no longer be told
# apart from "red because something else broke". The session that builds the
# column moves these checks into board-selftest.sh (and updates the counts)
# once they are green.
#
# What the column is (operator decision 2026-09-17): a DISPLAY-ONLY reading of
# the filesystem, one value per scanned repo:
# - no file named exactly PLAN.md in the repo root
# Nd PLAN.md exists; N = whole days its mtime lies BEHIND the last commit
# (0d when PLAN.md is as new as, or newer than, the last commit)
# It never reads PLAN.md's content, never enters a sort, a --plan block or a
# sum, and does not judge whether a criterion can be falsified.
#
# Table cells are located by the header, never by a fixed offset: the column's
# position is the builder's choice. The locator splits header and row on runs
# of 2+ spaces, which relies on two properties the table already has - every
# cell is followed by at least two spaces, and NESTE (free prose) is last.
#
# ASCII only, bash 3.2 safe, never touches ~/repos or ~/.claude/coord.
set -u
export LC_ALL=C
DIR="$(cd "$(dirname "$0")" && pwd)"
BOARD="$DIR/board.sh"
ROOT="$(mktemp -d)"
CLAUDE_COORD_DIR="$(mktemp -d)"
export CLAUDE_COORD_DIR
cleanup() { /bin/rm -rf "$ROOT" "$CLAUDE_COORD_DIR" 2>/dev/null; }
trap cleanup EXIT
PASS=0; FAIL=0
check() { if [ "$2" -eq 0 ]; then PASS=$((PASS+1)); echo " ok - $1"; else FAIL=$((FAIL+1)); echo " FAIL - $1"; fi; }
NOW="$(date +%s)"
DAY=86400
mkrepo() { mkdir -p "$1" && git -C "$1" init -q 2>/dev/null; }
state() {
{
echo "# STATE - $2"
echo "## NESTE - START HER"
echo "<!-- board: status=$3; blocked-on=-; next-cost=Sonnet 5/high -->"
echo "Next step for $2."
} > "$1/STATE.md"
}
# commit_at <dir> <epoch> <msg>: commit everything with a fixed committer date.
commit_at() {
git -C "$1" add -A >/dev/null 2>&1
GIT_AUTHOR_DATE="@$2 +0000" GIT_COMMITTER_DATE="@$2 +0000" \
git -C "$1" -c user.email=t@t -c user.name=t commit -qm "$3" >/dev/null 2>&1
}
touch_at() { touch -t "$(date -r "$2" +%Y%m%d%H%M.%S)" "$1"; }
last_commit() { git -C "$1" log -1 --format=%ct 2>/dev/null; }
mtime() { stat -f %m "$1"; }
# table_cell <table-output> <repo>: the PLAN cell of <repo>'s row, read under
# the nearest preceding header line. Empty when the header has no PLAN column.
table_cell() {
printf '%s\n' "$1" | awk -v repo="$2" '
function split2(s, arr, n) { gsub(/ +/, "\t", s); sub(/^\t/, "", s); return split(s, arr, "\t") }
/^REPO / { n = split2($0, h); col = 0; for (i = 1; i <= n; i++) if (h[i] == "PLAN") col = i; next }
$1 == repo && col > 0 { split2($0, r); print r[col]; exit }'
}
row_plan() { printf '%s\n' "$1" | sed -n 's/^plan=//p'; }
board() { "$BOARD" --roots "$ROOT" "$@" 2>/dev/null; }
echo "plan-column-eval (root: $ROOT, mailbox: $CLAUDE_COORD_DIR)"
# --- Fixtures ---------------------------------------------------------------
# p-none: no PLAN.md anywhere.
mkrepo "$ROOT/p-none"; state "$ROOT/p-none" p-none planned
commit_at "$ROOT/p-none" $((NOW - 3 * DAY)) init
# p-fresh: PLAN.md newer than the last commit -> 0d. The file is EMPTY: a value
# computed from content could not produce 0d here, so this also shows that
# the column is a filesystem reading.
mkrepo "$ROOT/p-fresh"; state "$ROOT/p-fresh" p-fresh planned
commit_at "$ROOT/p-fresh" $((NOW - 2 * DAY)) init
: > "$ROOT/p-fresh/PLAN.md"
touch_at "$ROOT/p-fresh/PLAN.md" "$NOW"
# p-stale: PLAN.md 5 days and 3 hours older than the last commit -> 5d. Its
# content looks like a value on purpose ("PLAN=99d"); it must not be read.
mkrepo "$ROOT/p-stale"; state "$ROOT/p-stale" p-stale planned
printf '# PLAN\nPLAN=99d\n' > "$ROOT/p-stale/PLAN.md"
P_STALE_CT=$((NOW - 3600))
commit_at "$ROOT/p-stale" "$P_STALE_CT" init
touch_at "$ROOT/p-stale/PLAN.md" $((P_STALE_CT - 5 * DAY - 3 * 3600))
# p-docs: PLAN.md in docs/ only - the wrong place by convention.
mkrepo "$ROOT/p-docs"; state "$ROOT/p-docs" p-docs planned
mkdir -p "$ROOT/p-docs/docs"; echo "# PLAN" > "$ROOT/p-docs/docs/PLAN.md"
commit_at "$ROOT/p-docs" $((NOW - DAY)) init
# p-lower: lowercase plan.md in the root. On a case-insensitive filesystem
# (APFS default) `[ -f PLAN.md ]` is TRUE here, so a board that asks the
# filesystem alone would report a plan that is not there. The name has to be
# compared byte for byte against what the directory actually stores (e.g. a
# directory listing matched exactly against PLAN.md).
mkrepo "$ROOT/p-lower"; state "$ROOT/p-lower" p-lower planned
echo "# plan" > "$ROOT/p-lower/plan.md"
commit_at "$ROOT/p-lower" $((NOW - DAY)) init
# --- Ground truth for the fixtures (these must be green today) -------------
[ -z "$(git -C "$ROOT/p-stale" status --porcelain)" ]
check "fixture: p-stale is committed clean" $?
P_STALE_EXP="$(( ($(last_commit "$ROOT/p-stale") - $(mtime "$ROOT/p-stale/PLAN.md")) / DAY ))d"
[ "$P_STALE_EXP" != "0d" ]
check "fixture: p-stale's expected value is computed from its timestamps and is not 0d ($P_STALE_EXP)" $?
[ "$(mtime "$ROOT/p-fresh/PLAN.md")" -ge "$(last_commit "$ROOT/p-fresh")" ]
check "fixture: p-fresh's PLAN.md is newer than its last commit" $?
if [ -f "$ROOT/p-lower/PLAN.md" ]; then LOWER_ARMED=0; else LOWER_ARMED=1; fi
check "fixture: the case trap is armed here - [ -f PLAN.md ] is true for plan.md (case-insensitive fs)" "$LOWER_ARMED"
ls -1 "$ROOT/p-lower" | grep -qx 'PLAN.md'; rc=$?
[ "$rc" -ne 0 ]; check "fixture: a byte-exact directory listing does NOT see PLAN.md in p-lower" $?
TABLE="$(board)"
N_REPOS="$(printf '%s\n' "$TABLE" | sed -n 's/^BOARD .*(\([0-9][0-9]*\) repo).*/\1/p' | head -1)"
[ "${N_REPOS:-0}" -eq 5 ]; check "fixture: board scanned the 5 fixture repos (header says ${N_REPOS:-nothing})" $?
# --- 1. Three different values from three repos ----------------------------
printf '%s\n' "$TABLE" | grep -q '^REPO .* PLAN '
check "1: the table header carries a PLAN column" $?
[ "$(table_cell "$TABLE" p-none)" = "-" ]; check "1: no root PLAN.md -> '-'" $?
[ "$(table_cell "$TABLE" p-fresh)" = "0d" ]; check "1: PLAN.md newer than the last commit -> '0d'" $?
[ "$(table_cell "$TABLE" p-stale)" = "$P_STALE_EXP" ]
check "1: PLAN.md older than the last commit -> '$P_STALE_EXP' (from the fixture's own timestamps)" $?
V3="$(table_cell "$TABLE" p-none) $(table_cell "$TABLE" p-fresh) $(table_cell "$TABLE" p-stale)"
[ "$(printf '%s\n' $V3 | sort -u | wc -l | tr -d ' ')" -eq 3 ]
check "1: the three fixtures show three DIFFERENT values ($V3)" $?
# --- 2. Wrong place and wrong case are not a plan ---------------------------
[ "$(table_cell "$TABLE" p-docs)" = "-" ]; check "2: docs/PLAN.md only -> '-'" $?
[ "$(table_cell "$TABLE" p-lower)" = "-" ]; check "2: lowercase plan.md in the root -> '-' (byte-exact name)" $?
# --- 3. --row carries plan= with the table's value ---------------------------
for r in p-none p-fresh p-stale p-docs p-lower; do
RV="$(row_plan "$(board --row "$r")")"
TV="$(table_cell "$TABLE" "$r")"
[ -n "$RV" ] && [ "$RV" = "$TV" ]
check "3: --row $r has plan=${RV:-<missing>} equal to the table cell (${TV:-<missing>})" $?
done
[ "$(board --row p-none | grep -c '^plan=')" -eq 1 ]
check "3: --row emits exactly one plan= line" $?
[ "$(board --row p-none | tail -1 | cut -d= -f1)" = "neste" ]
check "3: neste= is still the last --row line" $?
# --- 4. Not in the sort, not in --plan blocks, not summed --------------------
# Two repos identical in every sorted property; run 1 gives s-a the stale plan
# and s-b the fresh one, run 2 swaps them. Any sort key built on PLAN would
# reverse them in one of the two runs.
SROOT="$(mktemp -d)"
for r in s-a s-b; do
mkrepo "$SROOT/$r"; state "$SROOT/$r" "$r" planned
echo "<!-- route: path=known; verification=strong; reversibility=cheap; scope=local; rationale=x -->" >> "$SROOT/$r/STATE.md"
: > "$SROOT/$r/PLAN.md"
commit_at "$SROOT/$r" $((NOW - DAY)) init
touch_at "$SROOT/$r/STATE.md" $((NOW - 2 * DAY))
done
touch_at "$SROOT/s-a/PLAN.md" $((NOW - 9 * DAY)); touch_at "$SROOT/s-b/PLAN.md" "$NOW"
T1="$("$BOARD" --roots "$SROOT" 2>/dev/null)"; P1="$("$BOARD" --roots "$SROOT" --plan 2>/dev/null)"
touch_at "$SROOT/s-a/PLAN.md" "$NOW"; touch_at "$SROOT/s-b/PLAN.md" $((NOW - 9 * DAY))
T2="$("$BOARD" --roots "$SROOT" 2>/dev/null)"; P2="$("$BOARD" --roots "$SROOT" --plan 2>/dev/null)"
A1="$(table_cell "$T1" s-a)"; A2="$(table_cell "$T2" s-a)"
[ -n "$A1" ] && [ -n "$A2" ] && [ "$A1" != "$A2" ]
check "4: precondition - the swap really changed s-a's PLAN value (${A1:-<missing>} -> ${A2:-<missing>})" $?
O1="$(printf '%s\n' "$T1" | awk '$1 ~ /^s-/ {print $1}' | tr '\n' ' ')"
O2="$(printf '%s\n' "$T2" | awk '$1 ~ /^s-/ {print $1}' | tr '\n' ' ')"
[ -n "$A1" ] && [ "$O1" = "$O2" ]; check "4: table order is identical across the swap ($O1| $O2)" $?
Q1="$(printf '%s\n' "$P1" | sed -n 's/^repo=//p' | tr '\n' ' ')"
Q2="$(printf '%s\n' "$P2" | sed -n 's/^repo=//p' | tr '\n' ' ')"
[ -n "$A1" ] && [ -n "$Q1" ] && [ "$Q1" = "$Q2" ]; check "4: --plan order is identical across the swap ($Q1| $Q2)" $?
if printf '%s\n' "$P1" | grep -qi '^plan='; then rc=1; else rc=0; fi
[ -n "$A1" ] && [ "$rc" -eq 0 ]; check "4: --plan blocks carry no plan= key" $?
S1="$(printf '%s\n' "$T1" | grep '^SUM:')"; S2="$(printf '%s\n' "$T2" | grep '^SUM:')"
if printf '%s' "$S1" | grep -qi 'plan'; then rc=1; else rc=0; fi
[ -n "$A1" ] && [ "$rc" -eq 0 ] && [ "$S1" = "$S2" ]; check "4: the SUM line neither mentions PLAN nor moves with it" $?
/bin/rm -rf "$SROOT" 2>/dev/null
# --- 5. Replay: the shape C failed on ----------------------------------------
# Many commits, no root PLAN.md, the destination written in docs/*.local.md,
# and a release tag late in the window. A heuristic keyed on "activity without
# a release" went silent at the tag; this column must read '-' at every step.
LROOT="$(mktemp -d)"
L="$LROOT/loop-repo"
mkrepo "$L"; state "$L" loop-repo in-progress
mkdir -p "$L/docs"; echo "destination lives here" > "$L/docs/destination.local.md"
BASE=$((NOW - 60 * DAY)); HITS=0; STEPS=0
i=0
while [ "$i" -lt 6 ]; do
echo "step $i" > "$L/work.txt"
commit_at "$L" $((BASE + i * 10 * DAY)) "step $i"
[ "$i" -eq 4 ] && git -C "$L" -c tag.gpgSign=false tag v9.9.0 >/dev/null 2>&1
LT="$("$BOARD" --roots "$LROOT" 2>/dev/null)"
STEPS=$((STEPS + 1))
[ "$(table_cell "$LT" loop-repo)" = "-" ] && HITS=$((HITS + 1))
i=$((i + 1))
done
[ -n "$(git -C "$L" tag -l v9.9.0)" ]; check "5: fixture - the release tag exists in the replay" $?
[ "$HITS" -eq "$STEPS" ]; check "5: '-' at every step of the replay, before and after the tag ($HITS of $STEPS)" $?
/bin/rm -rf "$LROOT" 2>/dev/null
# --- 6. Denominator: one value per scanned repo ------------------------------
WITH=0
for r in p-none p-fresh p-stale p-docs p-lower; do
[ -n "$(table_cell "$TABLE" "$r")" ] && WITH=$((WITH + 1))
done
[ "$WITH" -eq "${N_REPOS:-0}" ]; check "6: every scanned repo has a PLAN value ($WITH of ${N_REPOS:-?})" $?
printf '%s\n' "$TABLE" | grep -q '^undersoekt: '
check "6: the scan's own denominator line is still printed" $?
# --- 7. Legend -----------------------------------------------------------------
LEG="$(printf '%s\n' "$TABLE" | awk '/^PLAN = /{on=1} on && /^[A-Z]+ = / && !/^PLAN = /{on=0} on')"
[ -n "$LEG" ]; check "7: the legend has a 'PLAN = ' entry" $?
printf '%s' "$LEG" | grep -q 'IKKE' && printf '%s' "$LEG" | grep -qi 'felle'
check "7: the legend says the column does NOT judge whether the criterion can be falsified" $?
printf '%s' "$LEG" | grep -q 'touch' && printf '%s' "$LEG" | grep -q 'checkout'
check "7: the legend says touch/checkout reset the mtime" $?
if printf '%s' "$LEG" | LC_ALL=C grep -q '[^ -~]'; then rc=1; else rc=0; fi
[ -n "$LEG" ] && [ "$rc" -eq 0 ]; check "7: the legend is ASCII like the rest of the table" $?
echo ""
echo "plan-column-eval: $PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ] || exit 1
exit 0