feat(route): give next-cost a writer, and make the rubric the only copy
The board line's next-cost field had a reader (board.sh) and no writer, so
its value was retyped by hand every session and drifted into several
competing spellings. Cleaning the data could not fix that: the cause was the
missing write path.
route.sh is that writer. Four scored traits of the next task -- path,
verification, reversibility, scope -- plus a required rationale, run through
the operator's model rubric moved here as the single copy. The row table is a
closed set of six values, so a seventh spelling cannot enter circulation, and
route-selftest.sh section 6 runs the round trip (route emits -> board parses)
inside one repo rather than across two.
Two spellings of one decision come out of one table: the rubric name for the
board line, the CLI alias for the command the operator pastes. Effort levels
are the set pinned in this marketplace; model aliases are gated against the
installed claude rather than hardcoded.
Three things worth naming:
- Escalation is asymmetric. Any single trait escalates; the cheapest row needs
all four at the cheap end. Underkill costs one session, overkill costs quota
every session -- but a wrong architecture call costs more than either.
- The Fable rows fire only from an explicit judgement flag, never inferred
from the last-session record. "The session did not finish" also covers
context exhaustion and operator interrupts, which say nothing about the
model, and Fable runs without an advisor.
- The trait block is a single-line HTML comment because board.sh's NESTE
extractor skips only lines that START with '<!--'. Measured first: a YAML
block or a multi-line comment silently replaces the repo's next step on the
board with "next_task:". Pinned by section 7.
board.sh is untouched as a program; its header now points at route.sh for the
value set so this does not reopen the two-specs defect 305f168 closed.
Selftests: coord 136, board 30, route 47 (new), node 7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017peNgsxVt1BR4BTuMwiPoX
This commit is contained in:
parent
305f1687b7
commit
0d1183800e
8 changed files with 808 additions and 11 deletions
|
|
@ -27,6 +27,10 @@
|
|||
# any spelling, but this field is compared across repos by eye, so
|
||||
# one form is the whole point - and a second spelling documented
|
||||
# here is how a field with no write path drifts.
|
||||
# The field now HAS a write path: route.sh emits it, and the set of
|
||||
# legal values is that script's row table - not this comment, which
|
||||
# shows the form only. `route.sh --help` is the authority; the two
|
||||
# ends are pinned together by route-selftest.sh section 6.
|
||||
#
|
||||
# ATTENTION AXIS, NOT A TOPIC AXIS. This status vocabulary is deliberately NOT
|
||||
# the vocabulary a cross-repo TOPIC register uses. A topic register answers
|
||||
|
|
|
|||
356
scripts/route-selftest.sh
Executable file
356
scripts/route-selftest.sh
Executable file
|
|
@ -0,0 +1,356 @@
|
|||
#!/bin/bash
|
||||
# route-selftest.sh - prove route.sh against the closed rubric row table, and
|
||||
# prove that what it emits is what board.sh parses back. Re-run after any edit
|
||||
# to route.sh or to the row table. ASCII only, bash 3.2 safe.
|
||||
#
|
||||
# The highest-value section here is 6, the round trip: route.sh WRITES the
|
||||
# next-cost value and board.sh READS it, and until now this repo owned only the
|
||||
# reader. A field with a reader and no writer drifts by construction - that is
|
||||
# the defect that put several competing spellings in circulation. One suite
|
||||
# now pins both ends, so a spelling change that breaks the parser fails here
|
||||
# instead of in the operator's eye three weeks later.
|
||||
set -u
|
||||
export LC_ALL=C
|
||||
|
||||
DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROUTE="$DIR/route.sh"
|
||||
BOARD="$DIR/board.sh"
|
||||
|
||||
PASS=0; FAIL=0
|
||||
check() { if [ "$2" -eq 0 ]; then PASS=$((PASS+1)); echo " ok - $1"; else FAIL=$((FAIL+1)); echo " FAIL - $1"; fi; }
|
||||
|
||||
# Multibyte building blocks (octal escapes keep this source ASCII).
|
||||
EMDASH="$(printf '\342\200\224')"
|
||||
HAND="$(printf '\360\237\221\211')"
|
||||
|
||||
R="$ROUTE"
|
||||
# Shorthand: run route.sh with the four traits + a rationale, print one field.
|
||||
# $1..$4 traits, $5 field name.
|
||||
field() {
|
||||
"$R" --path "$1" --verification "$2" --reversibility "$3" --scope "$4" \
|
||||
--rationale "selftest" 2>/dev/null | sed -n "s/^$5=//p"
|
||||
}
|
||||
|
||||
echo "route-selftest"
|
||||
|
||||
# --- 1. Every rubric row is reachable -------------------------------------
|
||||
# A row that no trait combination can produce is dead policy. All six must fire.
|
||||
|
||||
got="$(field known strong cheap local next-cost)"
|
||||
[ "$got" = "Sonnet 5/high" ]; check "row 1: known/strong/cheap/local -> Sonnet 5/high" $?
|
||||
|
||||
got="$(field known weak cheap local next-cost)"
|
||||
[ "$got" = "Sonnet 5/xhigh" ]; check "row 2: weak verification -> Sonnet 5/xhigh" $?
|
||||
|
||||
got="$(field partial strong cheap local next-cost)"
|
||||
[ "$got" = "Opus 5/high" ]; check "row 3: path=partial -> Opus 5/high" $?
|
||||
|
||||
got="$(field known strong cheap cross-cutting next-cost)"
|
||||
[ "$got" = "Opus 5/xhigh" ]; check "row 4: scope=cross-cutting -> Opus 5/xhigh" $?
|
||||
|
||||
got="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --opus-xhigh-failed 2>/dev/null | sed -n 's/^next-cost=//p')"
|
||||
[ "$got" = "Fable 5/high" ]; check "row 5: opus-xhigh-failed -> Fable 5/high" $?
|
||||
|
||||
got="$("$R" --path undetermined --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --opus-xhigh-failed 2>/dev/null | sed -n 's/^next-cost=//p')"
|
||||
[ "$got" = "Fable 5/xhigh" ]; check "row 6: failed + undetermined -> Fable 5/xhigh" $?
|
||||
|
||||
# --- 2. Escalation is asymmetric ------------------------------------------
|
||||
# One trait escalates; a downgrade needs ALL of them. Underkill costs one
|
||||
# session, overkill costs quota every session - so the cheap row must be hard
|
||||
# to reach and the expensive rows easy.
|
||||
|
||||
got="$(field known strong one-way local next-cost)"
|
||||
[ "$got" = "Opus 5/xhigh" ]; check "one-way alone escalates to row 4" $?
|
||||
|
||||
got="$(field known strong costly local next-cost)"
|
||||
[ "$got" = "Opus 5/high" ]; check "costly alone escalates to row 3" $?
|
||||
|
||||
got="$(field known strong cheap multi-file next-cost)"
|
||||
[ "$got" = "Opus 5/high" ]; check "multi-file alone escalates to row 3" $?
|
||||
|
||||
got="$(field known none cheap local next-cost)"
|
||||
[ "$got" = "Sonnet 5/xhigh" ]; check "verification=none alone escalates to row 2" $?
|
||||
|
||||
got="$(field undetermined strong cheap local next-cost)"
|
||||
[ "$got" = "Opus 5/high" ]; check "path=undetermined stops at row 3, not row 4" $?
|
||||
|
||||
# --- 3. The emitted vocabulary is CLOSED ----------------------------------
|
||||
# Every one of the 3*3*3*3 trait combinations, in both history states, must
|
||||
# emit one of exactly six strings. This is what structurally prevents a
|
||||
# seventh spelling from ever entering circulation: not a convention, an
|
||||
# enumeration. The board line's drift was possible only because the field had
|
||||
# no writer with a closed range.
|
||||
VOCAB="|Sonnet 5/high|Sonnet 5/xhigh|Opus 5/high|Opus 5/xhigh|Fable 5/high|Fable 5/xhigh|"
|
||||
bad=0; n=0
|
||||
for p in known partial undetermined; do
|
||||
for v in strong weak none; do
|
||||
for r in cheap costly one-way; do
|
||||
for s in local multi-file cross-cutting; do
|
||||
for h in "" "--opus-xhigh-failed"; do
|
||||
n=$((n+1))
|
||||
out="$("$R" --path "$p" --verification "$v" --reversibility "$r" --scope "$s" \
|
||||
--rationale x $h 2>/dev/null | sed -n 's/^next-cost=//p')"
|
||||
case "$VOCAB" in *"|$out|"*) ;; *) bad=$((bad+1)); echo " out-of-vocab: $p/$v/$r/$s [$out]" ;; esac
|
||||
done
|
||||
done
|
||||
done
|
||||
done
|
||||
done
|
||||
[ "$n" -eq 162 ] && [ "$bad" -eq 0 ]
|
||||
check "all 162 trait combinations emit one of the 6 rubric rows" $?
|
||||
|
||||
# --- 4. Every trait is REQUIRED -------------------------------------------
|
||||
# verification carries the most signal and is the one most often left out, so
|
||||
# a missing trait must be a hard error, never a silent default. A default here
|
||||
# would be indistinguishable from a scored value when the log is read back.
|
||||
"$R" --verification strong --reversibility cheap --scope local --rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "missing --path exits 2" $?
|
||||
"$R" --path known --reversibility cheap --scope local --rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "missing --verification exits 2" $?
|
||||
"$R" --path known --verification strong --scope local --rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "missing --reversibility exits 2" $?
|
||||
"$R" --path known --verification strong --reversibility cheap --rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "missing --scope exits 2" $?
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "missing --rationale exits 2" $?
|
||||
|
||||
# An unscored trait must not be smuggled in as empty text either.
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale "" >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "empty --rationale exits 2" $?
|
||||
|
||||
# --- 5. Invalid trait values are rejected ---------------------------------
|
||||
"$R" --path maybe --verification strong --reversibility cheap --scope local \
|
||||
--rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "invalid --path value exits 2" $?
|
||||
"$R" --path known --verification medium --reversibility cheap --scope local \
|
||||
--rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "invalid --verification value exits 2" $?
|
||||
"$R" --path known --verification strong --reversibility cheap --scope global \
|
||||
--rationale x >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "invalid --scope value exits 2" $?
|
||||
"$R" --nonsense >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "unknown argument exits 2" $?
|
||||
|
||||
# --- 6. ROUND TRIP: what route WRITES, board READS ------------------------
|
||||
# The reason this repo owns the calculator at all. Splice route.sh's next-cost
|
||||
# into a board line, run the real board.sh over it, and require the KOST column
|
||||
# to show the same string back. Runs for all six rows.
|
||||
ROOT="$(mktemp -d)"
|
||||
MBOX="$(mktemp -d)"
|
||||
cleanup() { /bin/rm -rf "$ROOT" "$MBOX" 2>/dev/null; }
|
||||
trap cleanup EXIT
|
||||
|
||||
rt_bad=0
|
||||
rt_case() { # $1 repo name, $2 next-cost value
|
||||
mkdir -p "$ROOT/$1" && git -C "$ROOT/$1" init -q 2>/dev/null
|
||||
{ echo "# STATE - $1"
|
||||
printf '## %s NESTE %s START HER\n' "$HAND" "$EMDASH"
|
||||
echo "<!-- board: status=in-progress; blocked-on=-; next-cost=$2 -->"
|
||||
echo ""
|
||||
echo "**1.** en helt vanlig prosalinje her."
|
||||
} > "$ROOT/$1/STATE.md"
|
||||
}
|
||||
i=0
|
||||
for combo in "known strong cheap local" "known weak cheap local" \
|
||||
"partial strong cheap local" "known strong cheap cross-cutting"; do
|
||||
set -- $combo
|
||||
i=$((i+1))
|
||||
cost="$(field "$1" "$2" "$3" "$4" next-cost)"
|
||||
rt_case "rt-$i" "$cost"
|
||||
done
|
||||
for h in 5 6; do
|
||||
if [ "$h" = "5" ]; then pp=known; else pp=undetermined; fi
|
||||
cost="$("$R" --path "$pp" --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --opus-xhigh-failed 2>/dev/null | sed -n 's/^next-cost=//p')"
|
||||
rt_case "rt-$h" "$cost"
|
||||
done
|
||||
|
||||
OUT="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
|
||||
for want in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
|
||||
"Fable 5/high" "Fable 5/xhigh"; do
|
||||
printf '%s' "$OUT" | grep -q "$want" || { rt_bad=$((rt_bad+1)); echo " board lost: [$want]"; }
|
||||
done
|
||||
[ "$rt_bad" -eq 0 ]; check "round trip: board.sh parses back all 6 emitted values" $?
|
||||
|
||||
# board.sh renders KOST with %-14s; a longer value shoves the whole row right
|
||||
# even though it parsed fine. Measure the widest string the table can emit -
|
||||
# not whatever the loop above happened to leave behind.
|
||||
widest=0
|
||||
for v in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
|
||||
"Fable 5/high" "Fable 5/xhigh"; do
|
||||
[ "${#v}" -gt "$widest" ] && widest="${#v}"
|
||||
done
|
||||
[ "$widest" -le 14 ]; check "widest emitted next-cost ($widest) fits the KOST column" $?
|
||||
|
||||
# --- 7. The route line does not steal the NESTE column --------------------
|
||||
# Measured before writing route.sh: board.sh's NESTE extractor skips blanks,
|
||||
# lines STARTING with '<!--', and headings, then prints the first line left. A
|
||||
# YAML block or a MULTI-line comment therefore becomes the NESTE column and
|
||||
# silently replaces the operator's next step with "next_task:". A single-line
|
||||
# comment is the only form that survives, which is why the trait line reuses
|
||||
# the board line's one-line grammar instead of inventing a block format.
|
||||
mkdir -p "$ROOT/rt-neste" && git -C "$ROOT/rt-neste" init -q 2>/dev/null
|
||||
LINE="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale "monsteret finnes i handlers" 2>/dev/null | sed -n 's/^route-line=//p')"
|
||||
{ echo "# STATE - rt-neste"
|
||||
printf '## %s NESTE %s START HER\n' "$HAND" "$EMDASH"
|
||||
echo "<!-- board: status=in-progress; blocked-on=-; next-cost=Sonnet 5/high -->"
|
||||
echo "$LINE"
|
||||
echo ""
|
||||
echo "**1.** prosalinjen som skal overleve."
|
||||
} > "$ROOT/rt-neste/STATE.md"
|
||||
OUT2="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
|
||||
printf '%s' "$OUT2" | grep -q 'prosalinjen som skal overleve'
|
||||
check "route line under NESTE leaves the prose in the NESTE column" $?
|
||||
|
||||
printf '%s' "$LINE" | grep -q '^<!-- route: .* -->$'
|
||||
check "route line is a single-line HTML comment" $?
|
||||
|
||||
# --- 8. Rationale is untrusted text on a comment line ---------------------
|
||||
# It is free text typed by a session and it lands inside an HTML comment on one
|
||||
# line. A newline splits the line (and hands the next line to the NESTE
|
||||
# extractor); a literal '-->' closes the comment early and dumps the rest into
|
||||
# the rendered STATE.md. Same line-oriented sanitizing the send side does.
|
||||
LINE2="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale "$(printf 'first\nsecond')" 2>/dev/null | sed -n 's/^route-line=//p')"
|
||||
[ "$(printf '%s' "$LINE2" | wc -l | tr -d ' ')" -eq 0 ]
|
||||
check "newline in rationale does not split the route line" $?
|
||||
|
||||
LINE3="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale 'oops --> loose text' 2>/dev/null | sed -n 's/^route-line=//p')"
|
||||
[ "$(printf '%s' "$LINE3" | grep -c -- '-->')" -eq 1 ]
|
||||
check "'-->' in rationale cannot close the comment early" $?
|
||||
|
||||
# --- 8b. The last-session record ------------------------------------------
|
||||
# Rows 5 and 6 are history-dependent, so without a record of how the previous
|
||||
# session went they are dead policy. The record is also the cheap proxy for
|
||||
# whether the routing itself is any good: systematically high corrections on
|
||||
# row 1 means the cheap row is too easy to reach, systematically zero on row 4
|
||||
# means escalation fires too readily. It gets a WRITER here for the same
|
||||
# reason next-cost needed one - a format with only a reader drifts.
|
||||
LAST="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-model "Opus 5" --last-effort xhigh \
|
||||
--last-completed no --last-corrections 3 2>/dev/null | sed -n 's/^route-last=//p')"
|
||||
printf '%s' "$LAST" | grep -q '^<!-- route-last: model=Opus 5; effort=xhigh; completed=no; corrections=3 -->$'
|
||||
check "route-last line is emitted in the pinned single-line form" $?
|
||||
|
||||
# Omitted record must emit no line at all rather than a half-filled one: a
|
||||
# blank record read back later is indistinguishable from a real measurement.
|
||||
out_norec="$("$R" --path known --verification strong --reversibility cheap \
|
||||
--scope local --rationale x 2>/dev/null)"
|
||||
if printf '%s' "$out_norec" | grep -q '^route-last='; then rc=1; else rc=0; fi
|
||||
check "no route-last line when the record is omitted" "$rc"
|
||||
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-completed maybe >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "invalid --last-completed exits 2" $?
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-corrections three >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "non-numeric --last-corrections exits 2" $?
|
||||
|
||||
# The record is telemetry and must NOT silently fire the Fable rows. Only the
|
||||
# explicit flag does, because "an opus/xhigh session did not finish" also
|
||||
# covers context exhaustion, an operator interrupt and a block on another repo
|
||||
# - none of which mean the MODEL failed at this step. Fable runs without an
|
||||
# advisor, so this auto-escalation has to stay a judgement, not an inference.
|
||||
got="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-model "Opus 5" --last-effort xhigh --last-completed no \
|
||||
--last-corrections 4 2>/dev/null | sed -n 's/^next-cost=//p')"
|
||||
[ "$got" = "Sonnet 5/high" ]; check "a failed opus/xhigh record alone does NOT reach Fable" $?
|
||||
|
||||
# A record is all four fields or none. A partial one emits `corrections=` with
|
||||
# nothing after it, which reads back later exactly like a measured zero.
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-model "Opus 5" --last-effort xhigh >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "a partial last-session record exits 2" $?
|
||||
|
||||
# All three comment lines stacked under the heading must still leave the prose
|
||||
# in the NESTE column - that is the arrangement a real STATE.md ends up with.
|
||||
mkdir -p "$ROOT/rt-three" && git -C "$ROOT/rt-three" init -q 2>/dev/null
|
||||
{ echo "# STATE - rt-three"
|
||||
printf '## %s NESTE %s START HER\n' "$HAND" "$EMDASH"
|
||||
echo "<!-- board: status=in-progress; blocked-on=-; next-cost=Sonnet 5/high -->"
|
||||
echo "$LINE"
|
||||
echo "$LAST"
|
||||
echo ""
|
||||
# Kept well under board.sh's 38-char NESTE truncation: a longer line would
|
||||
# be cut mid-word and fail this grep for a reason that has nothing to do
|
||||
# with what is being tested.
|
||||
echo "**1.** tredje prosalinje."
|
||||
} > "$ROOT/rt-three/STATE.md"
|
||||
OUT3="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
|
||||
printf '%s' "$OUT3" | grep -q 'tredje prosalinje'
|
||||
check "board + route + route-last stacked still yield prose in NESTE" $?
|
||||
|
||||
# --- 9. Startup command and fallback --------------------------------------
|
||||
# Two spellings of ONE decision: the rubric name for the board line, the CLI
|
||||
# alias for the command the operator pastes. They must never disagree.
|
||||
cmd="$(field partial strong cheap local command)"
|
||||
[ "$cmd" = "claude --model opus --effort high" ]; check "command mirrors the row (opus/high)" $?
|
||||
|
||||
cmd="$(field known strong cheap local command)"
|
||||
[ "$cmd" = "claude --model sonnet --effort high" ]; check "command mirrors the row (sonnet/high)" $?
|
||||
|
||||
# The rubric requires ALWAYS naming one row cheaper as the quota fallback.
|
||||
fb="$(field partial strong cheap local fallback)"
|
||||
[ "$fb" = "Sonnet 5/xhigh" ]; check "fallback is exactly one row cheaper" $?
|
||||
|
||||
fb="$(field known strong cheap local fallback)"
|
||||
[ "$fb" = "Sonnet 5/high" ]; check "row 1 fallback floors at row 1, never below" $?
|
||||
|
||||
# The fallback needs its own pasteable command or the operator translates by
|
||||
# hand at exactly the moment they are under quota pressure.
|
||||
fbc="$(field partial strong cheap local fallback-command)"
|
||||
[ "$fbc" = "claude --model sonnet --effort xhigh" ]; check "fallback ships its own command" $?
|
||||
|
||||
# --- 10. The command carries no 'cd' --------------------------------------
|
||||
# One repo per terminal tab: a startup command prefixed with cd is wrong by
|
||||
# construction, and another directory means another tab.
|
||||
out="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x 2>/dev/null)"
|
||||
if printf '%s' "$out" | grep -q 'cd '; then rc=1; else rc=0; fi
|
||||
check "no emitted command contains a cd prefix" "$rc"
|
||||
|
||||
# --- 11. Effort and model values are the VERIFIED sets --------------------
|
||||
# Effort levels are pinned in this marketplace at
|
||||
# config-audit/scanners/settings-validator.mjs (low|medium|high|xhigh|max).
|
||||
# Model aliases are whatever the INSTALLED claude accepts - never hardcoded
|
||||
# without a gate, because an alias that stops resolving turns every emitted
|
||||
# command into a paste that fails.
|
||||
efforts="$(printf '%s' "$out" | sed -n 's/^command=claude --model [a-z]* --effort //p')"
|
||||
case "|low|medium|high|xhigh|max|" in *"|$efforts|"*) rc=0 ;; *) rc=1 ;; esac
|
||||
check "emitted effort is in the verified effort set" "$rc"
|
||||
|
||||
if command -v claude >/dev/null 2>&1; then
|
||||
HELPTXT="$(claude --help 2>&1)"
|
||||
miss=0
|
||||
for alias in fable opus sonnet; do
|
||||
printf '%s' "$HELPTXT" | grep -q "'$alias'" || \
|
||||
printf '%s' "$HELPTXT" | grep -q "$alias" || miss=$((miss+1))
|
||||
done
|
||||
[ "$miss" -eq 0 ]; check "installed claude still accepts fable/opus/sonnet aliases" $?
|
||||
else
|
||||
echo " skip - claude not on PATH, model alias gate not run"
|
||||
fi
|
||||
|
||||
# --- 12. One spec, and it is the row table --------------------------------
|
||||
# The last board fix closed a defect whose root cause was this repo's own
|
||||
# --help being the SECOND spec for next-cost. route.sh must not reopen it: its
|
||||
# help may describe the row table (it owns it now) but must not restate the
|
||||
# board line grammar, which board.sh --help owns.
|
||||
HELP="$("$R" --help 2>/dev/null)"; rc=$?
|
||||
[ "$rc" -eq 0 ]; check "--help exits 0" $?
|
||||
printf '%s' "$HELP" | grep -q 'Sonnet 5/high'
|
||||
check "--help shows the canonical rubric spelling" $?
|
||||
if printf '%s' "$HELP" | grep -qE '(sonnet|opus|fable) ?5?/(high|xhigh)'; then rc=1; else rc=0; fi
|
||||
check "--help shows no versionless model example" "$rc"
|
||||
if printf '%s' "$HELP" | grep -q 'blocked-on='; then rc=1; else rc=0; fi
|
||||
check "--help does not restate the board line grammar" "$rc"
|
||||
|
||||
echo ""
|
||||
echo "route-selftest: $PASS passed, $FAIL failed"
|
||||
[ "$FAIL" -eq 0 ] || exit 1
|
||||
exit 0
|
||||
239
scripts/route.sh
Executable file
239
scripts/route.sh
Executable file
|
|
@ -0,0 +1,239 @@
|
|||
#!/bin/bash
|
||||
# route.sh - score four traits of the NEXT task, get the model and effort to
|
||||
# run it with. A pure calculator: reads nothing, writes nothing, prints one
|
||||
# block of key=value lines on stdout. The session pastes the result into
|
||||
# STATE.md; this script never touches a file.
|
||||
#
|
||||
# WHY THIS EXISTS. The board line's next-cost field had a reader (board.sh)
|
||||
# and no writer, so its value was typed by hand every session and drifted into
|
||||
# several competing spellings. Cleaning the data could not fix that.
|
||||
# A writer with a CLOSED output range can: this script can only ever emit one
|
||||
# of six strings, so a seventh cannot enter circulation.
|
||||
#
|
||||
# THE ROW TABLE IS THE POLICY, and it is the operator's rubric verbatim -
|
||||
# moved here so there is one copy rather than one per repo:
|
||||
#
|
||||
# 1 Sonnet 5/high reading, summarizing, docs, mechanical refactor
|
||||
# 2 Sonnet 5/xhigh TDD cycle, known-root-cause bugfix, one-file change
|
||||
# 3 Opus 5/high multi-file feature, architecture choice, hard debugging
|
||||
# 4 Opus 5/xhigh long autonomous run, big refactor, cross-repo migration
|
||||
# 5 Fable 5/high Opus 5/xhigh has ACTUALLY failed on this same step
|
||||
# 6 Fable 5/xhigh open problem, no known solution path, longest horizon
|
||||
#
|
||||
# Cheapest first, so the rubric's "always name one row cheaper as the quota
|
||||
# fallback" is row minus one, floored at row 1, correct by construction.
|
||||
#
|
||||
# THE TRAITS describe the task, never how it feels. "Hard", "complex" and
|
||||
# "important" are deliberately absent: they are unfalsifiable and collapse to
|
||||
# a hunch, which is the thing being replaced.
|
||||
#
|
||||
# --path known | partial | undetermined
|
||||
# Is the solution route described, or must it be found?
|
||||
# --verification strong | weak | none
|
||||
# Will tests, types or a compiler catch the error?
|
||||
# --reversibility cheap | costly | one-way
|
||||
# --scope local | multi-file | cross-cutting
|
||||
# --rationale required free text - WHY these four scores
|
||||
#
|
||||
# All five are required. None has a default, and that is load-bearing:
|
||||
# verification carries the most signal and is the trait most often left out,
|
||||
# and a default would be indistinguishable from a real score when the log is
|
||||
# read back to find out whether the ROUTING was wrong or the SCORING was.
|
||||
#
|
||||
# SELECTION - first match wins, most expensive first:
|
||||
# row 6 history says Opus 5/xhigh failed here, and path=undetermined
|
||||
# row 5 history says Opus 5/xhigh failed here
|
||||
# row 4 reversibility=one-way OR scope=cross-cutting
|
||||
# row 3 path=partial|undetermined OR reversibility=costly OR scope=multi-file
|
||||
# row 2 verification=weak|none
|
||||
# row 1 otherwise
|
||||
#
|
||||
# Escalation is ASYMMETRIC on purpose: any single trait escalates, while
|
||||
# reaching row 1 needs all four at the cheap end. Underkill costs one session,
|
||||
# overkill costs quota every session - but a wrong architecture decision in a
|
||||
# published plugin costs more than either.
|
||||
#
|
||||
# WHERE IT DISAGREES WITH THE RUBRIC'S EXAMPLES. The rows are task-type labels;
|
||||
# the traits are a different classification over the same six outcomes. They
|
||||
# part company by one row on the two cheapest rows - documentation scores
|
||||
# known/none/cheap/local and lands on row 2 where the rubric's prose says row
|
||||
# 1; a TDD cycle scores known/strong/cheap/local and lands on row 1 where the
|
||||
# prose says row 2. This is left alone deliberately. Chasing the example
|
||||
# phrases would mean re-implementing description-matching, which is the guess
|
||||
# the traits exist to replace. It is a policy, not a theorem: --rationale is
|
||||
# how a misscore is found afterwards.
|
||||
#
|
||||
# THE SPECIFICATION CHECK THAT COMES FREE. path=undetermined with no planned
|
||||
# design phase means the TASK DESCRIPTION is underspecified - not that the
|
||||
# model should be upgraded. Rewrite the next step; upgrading the model to
|
||||
# compensate for a vague spec is the most expensive form of procrastination
|
||||
# available.
|
||||
#
|
||||
# Usage:
|
||||
# route.sh --path <v> --verification <v> --reversibility <v> --scope <v>
|
||||
# --rationale <text> [--opus-xhigh-failed]
|
||||
#
|
||||
# route.sh ... --last-model <name> --last-effort <level>
|
||||
# --last-completed <yes|no> --last-corrections <n>
|
||||
#
|
||||
# --opus-xhigh-failed is the ONLY way rows 5 and 6 can fire, and it means one
|
||||
# narrow thing: an Opus 5/xhigh session already ran at THIS SAME next step and
|
||||
# did not complete it. Not "a session failed" - context exhaustion, an operator
|
||||
# interrupt or a block on another repo are not this flag. It escalates to a
|
||||
# model that runs without an advisor, so when in doubt, leave it off.
|
||||
#
|
||||
# THE LAST-SESSION RECORD (the four --last-* fields, all or none) is what makes
|
||||
# any of this falsifiable. It records how the session that just ran actually
|
||||
# went, so the policy can later be judged against outcomes instead of against
|
||||
# how sensible it reads. --last-corrections is the cheap proxy: systematically
|
||||
# high counts on row 1 mean the cheap row is too easy to reach, systematically
|
||||
# zero on row 4 means escalation fires too readily.
|
||||
#
|
||||
# It is deliberately NOT wired to the Fable rows. Inferring "the model failed"
|
||||
# from "the session did not finish" would fire on context exhaustion and on
|
||||
# operator interrupts, which say nothing about the model. The record is
|
||||
# evidence; --opus-xhigh-failed is a judgement. Keeping them apart is why the
|
||||
# evidence stays worth reading.
|
||||
#
|
||||
# Exit 0 on a decision, 2 on any bad or missing argument. ASCII only,
|
||||
# bash 3.2 safe.
|
||||
set -u
|
||||
export LC_ALL=C
|
||||
|
||||
PATH_T=""; VERIF=""; REVERS=""; SCOPE=""; RATIONALE=""; RAT_SET=0; FAILED=0
|
||||
L_MODEL=""; L_EFFORT=""; L_DONE=""; L_CORR=""; L_SET=0
|
||||
|
||||
die() { echo "route: $1" >&2; exit 2; }
|
||||
|
||||
need() { [ $# -ge 2 ] || die "$1 requires a value"; }
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
# bash 3.2: `shift 2` past the end of $# is a no-op -> would loop forever.
|
||||
--path) need "$@"; PATH_T="$2"; shift 2 ;;
|
||||
--verification) need "$@"; VERIF="$2"; shift 2 ;;
|
||||
--reversibility) need "$@"; REVERS="$2"; shift 2 ;;
|
||||
--scope) need "$@"; SCOPE="$2"; shift 2 ;;
|
||||
--rationale) need "$@"; RATIONALE="$2"; RAT_SET=1; shift 2 ;;
|
||||
--opus-xhigh-failed) FAILED=1; shift ;;
|
||||
--last-model) need "$@"; L_MODEL="$2"; L_SET=1; shift 2 ;;
|
||||
--last-effort) need "$@"; L_EFFORT="$2"; L_SET=1; shift 2 ;;
|
||||
--last-completed) need "$@"; L_DONE="$2"; L_SET=1; shift 2 ;;
|
||||
--last-corrections) need "$@"; L_CORR="$2"; L_SET=1; shift 2 ;;
|
||||
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) die "unknown argument: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Unknown values are rejected rather than tolerated. A silently accepted
|
||||
# typo would route on three traits and look exactly like a scored decision.
|
||||
case "$PATH_T" in
|
||||
known|partial|undetermined) ;;
|
||||
"") die "--path is required (known|partial|undetermined)" ;;
|
||||
*) die "--path: unknown value '$PATH_T' (known|partial|undetermined)" ;;
|
||||
esac
|
||||
case "$VERIF" in
|
||||
strong|weak|none) ;;
|
||||
"") die "--verification is required (strong|weak|none)" ;;
|
||||
*) die "--verification: unknown value '$VERIF' (strong|weak|none)" ;;
|
||||
esac
|
||||
case "$REVERS" in
|
||||
cheap|costly|one-way) ;;
|
||||
"") die "--reversibility is required (cheap|costly|one-way)" ;;
|
||||
*) die "--reversibility: unknown value '$REVERS' (cheap|costly|one-way)" ;;
|
||||
esac
|
||||
case "$SCOPE" in
|
||||
local|multi-file|cross-cutting) ;;
|
||||
"") die "--scope is required (local|multi-file|cross-cutting)" ;;
|
||||
*) die "--scope: unknown value '$SCOPE' (local|multi-file|cross-cutting)" ;;
|
||||
esac
|
||||
[ "$RAT_SET" -eq 1 ] || die "--rationale is required (why these four scores)"
|
||||
[ -n "$RATIONALE" ] || die "--rationale must not be empty"
|
||||
|
||||
# The last-session record is all four fields or none at all. A partial record
|
||||
# emits an empty value that reads back later exactly like a measured one, and
|
||||
# the whole purpose of the record is to be readable evidence months from now.
|
||||
if [ "$L_SET" -eq 1 ]; then
|
||||
[ -n "$L_MODEL" ] || die "--last-model is required with a last-session record"
|
||||
[ -n "$L_EFFORT" ] || die "--last-effort is required with a last-session record"
|
||||
[ -n "$L_DONE" ] || die "--last-completed is required with a last-session record"
|
||||
[ -n "$L_CORR" ] || die "--last-corrections is required with a last-session record"
|
||||
case "$L_DONE" in
|
||||
yes|no) ;;
|
||||
*) die "--last-completed: unknown value '$L_DONE' (yes|no)" ;;
|
||||
esac
|
||||
case "$L_CORR" in
|
||||
""|*[!0-9]*) die "--last-corrections must be a whole number, got '$L_CORR'" ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# --- Selection: first match wins, most expensive first ---------------------
|
||||
if [ "$FAILED" -eq 1 ] && [ "$PATH_T" = "undetermined" ]; then
|
||||
ROW=6; RULE="opus-xhigh-failed + path=undetermined"
|
||||
elif [ "$FAILED" -eq 1 ]; then
|
||||
ROW=5; RULE="opus-xhigh-failed"
|
||||
elif [ "$REVERS" = "one-way" ]; then
|
||||
ROW=4; RULE="reversibility=one-way"
|
||||
elif [ "$SCOPE" = "cross-cutting" ]; then
|
||||
ROW=4; RULE="scope=cross-cutting"
|
||||
elif [ "$PATH_T" != "known" ]; then
|
||||
ROW=3; RULE="path=$PATH_T"
|
||||
elif [ "$REVERS" = "costly" ]; then
|
||||
ROW=3; RULE="reversibility=costly"
|
||||
elif [ "$SCOPE" = "multi-file" ]; then
|
||||
ROW=3; RULE="scope=multi-file"
|
||||
elif [ "$VERIF" != "strong" ]; then
|
||||
ROW=2; RULE="verification=$VERIF"
|
||||
else
|
||||
ROW=1; RULE="no escalating trait (all four at the cheap end)"
|
||||
fi
|
||||
|
||||
# --- The row table: one decision, two spellings ---------------------------
|
||||
# The rubric name goes in the board line so repos compare by eye; the CLI alias
|
||||
# goes in the command the operator pastes. Emitting both from one table is the
|
||||
# point - two hand-maintained spellings of one decision is how they disagree.
|
||||
# Aliases are gated against the installed claude by route-selftest.sh section
|
||||
# 11, never assumed here.
|
||||
row_name() {
|
||||
case "$1" in
|
||||
1) echo "Sonnet 5/high" ;; 2) echo "Sonnet 5/xhigh" ;;
|
||||
3) echo "Opus 5/high" ;; 4) echo "Opus 5/xhigh" ;;
|
||||
5) echo "Fable 5/high" ;; 6) echo "Fable 5/xhigh" ;;
|
||||
esac
|
||||
}
|
||||
row_cmd() {
|
||||
case "$1" in
|
||||
1) echo "claude --model sonnet --effort high" ;;
|
||||
2) echo "claude --model sonnet --effort xhigh" ;;
|
||||
3) echo "claude --model opus --effort high" ;;
|
||||
4) echo "claude --model opus --effort xhigh" ;;
|
||||
5) echo "claude --model fable --effort high" ;;
|
||||
6) echo "claude --model fable --effort xhigh" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
FB=$((ROW - 1)); [ "$FB" -ge 1 ] || FB=1
|
||||
|
||||
# The rationale is free text from a session and lands inside an HTML comment on
|
||||
# ONE line. A newline would split the line and hand the remainder to board.sh's
|
||||
# NESTE extractor as the repo's next step; a literal '-->' would close the
|
||||
# comment early and spill the rest into the rendered STATE.md. Same
|
||||
# line-oriented sanitizing the send side applies to its own fields.
|
||||
RAT_CLEAN="$(printf '%s' "$RATIONALE" | tr '\r\n' ' ' | tr -d '\000-\037' \
|
||||
| sed 's/-->/-- >/g')"
|
||||
|
||||
# rationale goes LAST in the line: it is the only field that may contain a
|
||||
# ';', so anything after it would be unparseable.
|
||||
echo "row=$ROW"
|
||||
echo "rule=$RULE"
|
||||
echo "next-cost=$(row_name "$ROW")"
|
||||
echo "command=$(row_cmd "$ROW")"
|
||||
echo "fallback=$(row_name "$FB")"
|
||||
echo "fallback-command=$(row_cmd "$FB")"
|
||||
echo "route-line=<!-- route: path=$PATH_T; verification=$VERIF; reversibility=$REVERS; scope=$SCOPE; rationale=$RAT_CLEAN -->"
|
||||
if [ "$L_SET" -eq 1 ]; then
|
||||
LM="$(printf '%s' "$L_MODEL" | tr '\r\n' ' ' | tr -d '\000-\037' | sed 's/-->/-- >/g')"
|
||||
LE="$(printf '%s' "$L_EFFORT" | tr '\r\n' ' ' | tr -d '\000-\037' | sed 's/-->/-- >/g')"
|
||||
echo "route-last=<!-- route-last: model=$LM; effort=$LE; completed=$L_DONE; corrections=$L_CORR -->"
|
||||
fi
|
||||
exit 0
|
||||
Loading…
Add table
Add a link
Reference in a new issue