repo-mailbox/scripts/route-selftest.sh
Kjell Tore Guttormsen dd8f3ce042 feat(route)!: Fable rows are a hand-written override, never a route.sh outcome
The operator removed the "Fable can only be suggested after a failed Opus
5/xhigh session" policy on 2026-08-06 - it stood in the way too often. Until
now that policy was implemented as code: --opus-xhigh-failed was the ONLY
way route.sh's SELECTION chain could reach rows 5-6 (Fable), so the rubric
still enforced a policy the operator had already dropped.

Given a choice between (A) deleting the branches and the flag outright, so
the calculator's output range closes at row 4, or (C) keeping a path that
renders a Fable row on explicit instruction with rule=operator-override, the
operator chose A (AskUserQuestion) - CLAUDE.md is explicit that the rubric
stays the only deterministic lookup and a Fable choice is now always a
deviation from it, recorded in STATE as an override rather than produced
here. board.sh is untouched and still parses "Fable 5/high"/"Fable 5/xhigh"
written by hand into a board line (route-selftest.sh section 6 now pins that
half directly, since route.sh can no longer produce the strings itself).

TDD: every affected selftest check was rewritten to fail against the
unmodified route.sh first (72->68/69, confirmed red), then route.sh was
edited to match. --opus-xhigh-failed is gone outright - passing it now exits
2 like any other unknown argument, not silently accepted as a no-op.

route-selftest.sh: 73 -> 69 checks (three checks tested command shapes
row_advisor() can no longer produce; the two row-5/row-6 reachability checks
in section 1 collapsed into one "the flag is gone" check). coord 191, board
142 unaffected. Suite total 406 -> 402.

Also folds in a standalone fix already pushed this session: section 14 was
gating the wrong CLI fact (whether "claude --advisor fable" is rejected,
which row_advisor() never depends on) instead of the one it actually rests
on (whether opus/sonnet can advise a FABLE main model). Re-pointed and
verified against the installed CC 2.1.226.

Verified before committing: grepped every repo under ~/repos for a route
line carrying --opus-xhigh-failed (none - one repo has it in prose only,
not in its <!-- route: --> comment) and ran board.sh --plan/--brief over the
real tree to confirm no repo's command line broke. Sent a follow-up
coord-send to catalog superseding an earlier now-stale "406" stat-line
correction with the current 402.

skills/route/SKILL.md: usage block, "last-session record" framing, and the
closing --opus-xhigh-failed paragraph rewritten to match. CLAUDE.md, README
and CHANGELOG updated (checks 73->69, badge 406->402, new 0.21.0 entry).
Version bumped 0.20.3 -> 0.21.0 across plugin.json, package.json and all
three skill frontmatters (breaking CLI removal at 0.x -> minor, per v0.20.0
precedent).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ap1WKHCDPcSfpjo4ds2dQX
2026-08-09 22:03:05 +02:00

536 lines
28 KiB
Bash
Executable file

#!/bin/bash
# route-selftest.sh - prove route.sh against the closed rubric row table, and
# prove that what it emits is what board.sh parses back. Re-run after any edit
# to route.sh or to the row table. ASCII only, bash 3.2 safe.
#
# The highest-value section here is 6, the round trip: route.sh WRITES the
# next-cost value and board.sh READS it, and until now this repo owned only the
# reader. A field with a reader and no writer drifts by construction - that is
# the defect that put several competing spellings in circulation. One suite
# now pins both ends, so a spelling change that breaks the parser fails here
# instead of in the operator's eye three weeks later.
set -u
export LC_ALL=C
DIR="$(cd "$(dirname "$0")" && pwd)"
ROUTE="$DIR/route.sh"
BOARD="$DIR/board.sh"
PASS=0; FAIL=0
check() { if [ "$2" -eq 0 ]; then PASS=$((PASS+1)); echo " ok - $1"; else FAIL=$((FAIL+1)); echo " FAIL - $1"; fi; }
# Multibyte building blocks (octal escapes keep this source ASCII).
EMDASH="$(printf '\342\200\224')"
HAND="$(printf '\360\237\221\211')"
R="$ROUTE"
# Shorthand: run route.sh with the four traits + a rationale, print one field.
# $1..$4 traits, $5 field name.
field() {
"$R" --path "$1" --verification "$2" --reversibility "$3" --scope "$4" \
--rationale "selftest" 2>/dev/null | sed -n "s/^$5=//p"
}
echo "route-selftest"
# --- 1. Every calculator row is reachable -----------------------------------
# A row that no trait combination can produce is dead policy. All four rows
# this calculator can output must fire from traits alone.
got="$(field known strong cheap local next-cost)"
[ "$got" = "Sonnet 5/high" ]; check "row 1: known/strong/cheap/local -> Sonnet 5/high" $?
got="$(field known weak cheap local next-cost)"
[ "$got" = "Sonnet 5/xhigh" ]; check "row 2: weak verification -> Sonnet 5/xhigh" $?
got="$(field partial strong cheap local next-cost)"
[ "$got" = "Opus 5/high" ]; check "row 3: path=partial -> Opus 5/high" $?
got="$(field known strong cheap cross-cutting next-cost)"
[ "$got" = "Opus 5/xhigh" ]; check "row 4: scope=cross-cutting -> Opus 5/xhigh" $?
# Rows 5-6 (Fable) are the operator's hand-written override (policy decision
# 2026-08-06), never a rubric outcome - the calculator's output range is
# closed at row 4. The flag that used to gate them into reach is gone
# outright, not merely disarmed: a caller passing it gets the same "unknown
# argument" as any other typo.
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --opus-xhigh-failed >/dev/null 2>&1
[ $? -eq 2 ]; check "--opus-xhigh-failed is gone: unknown argument, not a route to Fable" $?
# --- 2. Escalation is asymmetric ------------------------------------------
# One trait escalates; a downgrade needs ALL of them. Underkill costs one
# session, overkill costs quota every session - so the cheap row must be hard
# to reach and the expensive rows easy.
got="$(field known strong one-way local next-cost)"
[ "$got" = "Opus 5/xhigh" ]; check "one-way alone escalates to row 4" $?
got="$(field known strong costly local next-cost)"
[ "$got" = "Opus 5/high" ]; check "costly alone escalates to row 3" $?
got="$(field known strong cheap multi-file next-cost)"
[ "$got" = "Opus 5/high" ]; check "multi-file alone escalates to row 3" $?
got="$(field known none cheap local next-cost)"
[ "$got" = "Sonnet 5/xhigh" ]; check "verification=none alone escalates to row 2" $?
got="$(field undetermined strong cheap local next-cost)"
[ "$got" = "Opus 5/high" ]; check "path=undetermined stops at row 3, not row 4" $?
# --- 3. The emitted vocabulary is CLOSED ----------------------------------
# Every one of the 3*3*3*3 trait combinations must emit one of exactly FOUR
# strings - the calculator's whole output range now that Fable is reached
# only by a hand-written operator override, never by this script. This is
# what structurally prevents a fifth spelling from ever entering circulation:
# not a convention, an enumeration. The board line's drift was possible only
# because the field had no writer with a closed range.
VOCAB="|Sonnet 5/high|Sonnet 5/xhigh|Opus 5/high|Opus 5/xhigh|"
bad=0; n=0
for p in known partial undetermined; do
for v in strong weak none; do
for r in cheap costly one-way; do
for s in local multi-file cross-cutting; do
n=$((n+1))
out="$("$R" --path "$p" --verification "$v" --reversibility "$r" --scope "$s" \
--rationale x 2>/dev/null | sed -n 's/^next-cost=//p')"
case "$VOCAB" in *"|$out|"*) ;; *) bad=$((bad+1)); echo " out-of-vocab: $p/$v/$r/$s [$out]" ;; esac
done
done
done
done
[ "$n" -eq 81 ] && [ "$bad" -eq 0 ]
check "all 81 trait combinations emit one of the 4 calculator rows" $?
# --- 4. Every trait is REQUIRED -------------------------------------------
# verification carries the most signal and is the one most often left out, so
# a missing trait must be a hard error, never a silent default. A default here
# would be indistinguishable from a scored value when the log is read back.
"$R" --verification strong --reversibility cheap --scope local --rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "missing --path exits 2" $?
"$R" --path known --reversibility cheap --scope local --rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "missing --verification exits 2" $?
"$R" --path known --verification strong --scope local --rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "missing --reversibility exits 2" $?
"$R" --path known --verification strong --reversibility cheap --rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "missing --scope exits 2" $?
"$R" --path known --verification strong --reversibility cheap --scope local >/dev/null 2>&1
[ $? -eq 2 ]; check "missing --rationale exits 2" $?
# An unscored trait must not be smuggled in as empty text either.
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale "" >/dev/null 2>&1
[ $? -eq 2 ]; check "empty --rationale exits 2" $?
# --- 5. Invalid trait values are rejected ---------------------------------
"$R" --path maybe --verification strong --reversibility cheap --scope local \
--rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "invalid --path value exits 2" $?
"$R" --path known --verification medium --reversibility cheap --scope local \
--rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "invalid --verification value exits 2" $?
"$R" --path known --verification strong --reversibility cheap --scope global \
--rationale x >/dev/null 2>&1
[ $? -eq 2 ]; check "invalid --scope value exits 2" $?
"$R" --nonsense >/dev/null 2>&1
[ $? -eq 2 ]; check "unknown argument exits 2" $?
# --- 6. ROUND TRIP: what route WRITES, board READS ------------------------
# The reason this repo owns the calculator at all. Splice route.sh's next-cost
# into a board line, run the real board.sh over it, and require the KOST column
# to show the same string back. Runs for all six rows.
ROOT="$(mktemp -d)"
MBOX="$(mktemp -d)"
cleanup() { /bin/rm -rf "$ROOT" "$MBOX" 2>/dev/null; }
trap cleanup EXIT
rt_bad=0
rt_case() { # $1 repo name, $2 next-cost value
mkdir -p "$ROOT/$1" && git -C "$ROOT/$1" init -q 2>/dev/null
{ echo "# STATE - $1"
printf '## %s NESTE %s START HER\n' "$HAND" "$EMDASH"
echo "<!-- board: status=in-progress; blocked-on=-; next-cost=$2 -->"
echo ""
echo "**1.** en helt vanlig prosalinje her."
} > "$ROOT/$1/STATE.md"
}
i=0
for combo in "known strong cheap local" "known weak cheap local" \
"partial strong cheap local" "known strong cheap cross-cutting"; do
set -- $combo
i=$((i+1))
cost="$(field "$1" "$2" "$3" "$4" next-cost)"
rt_case "rt-$i" "$cost"
done
# Rows 5-6 are never emitted by route.sh any more (policy decision
# 2026-08-06), but board.sh must still parse them back when the operator
# hand-writes a Fable board line - that is exactly the path that replaces the
# removed rubric outcome, so the literal strings are spliced in directly here
# rather than produced by "$R".
rt_case "rt-5" "Fable 5/high"
rt_case "rt-6" "Fable 5/xhigh"
OUT="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
for want in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
"Fable 5/high" "Fable 5/xhigh"; do
printf '%s' "$OUT" | grep -q "$want" || { rt_bad=$((rt_bad+1)); echo " board lost: [$want]"; }
done
[ "$rt_bad" -eq 0 ]; check "round trip: board.sh parses back all 6 emitted values" $?
# board.sh renders KOST with %-14s; a longer value shoves the whole row right
# even though it parsed fine. Measure the widest string the table can emit -
# not whatever the loop above happened to leave behind.
widest=0
for v in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
"Fable 5/high" "Fable 5/xhigh"; do
[ "${#v}" -gt "$widest" ] && widest="${#v}"
done
[ "$widest" -le 14 ]; check "widest emitted next-cost ($widest) fits the KOST column" $?
# --- 7. The route line does not steal the NESTE column --------------------
# Measured before writing route.sh: board.sh's NESTE extractor skips blanks,
# lines STARTING with '<!--', and headings, then prints the first line left. A
# YAML block or a MULTI-line comment therefore becomes the NESTE column and
# silently replaces the operator's next step with "next_task:". A single-line
# comment is the only form that survives, which is why the trait line reuses
# the board line's one-line grammar instead of inventing a block format.
mkdir -p "$ROOT/rt-neste" && git -C "$ROOT/rt-neste" init -q 2>/dev/null
LINE="$("$R" --path known --verification strong --reversibility cheap --scope local \
--rationale "monsteret finnes i handlers" 2>/dev/null | sed -n 's/^route-line=//p')"
{ echo "# STATE - rt-neste"
printf '## %s NESTE %s START HER\n' "$HAND" "$EMDASH"
echo "<!-- board: status=in-progress; blocked-on=-; next-cost=Sonnet 5/high -->"
echo "$LINE"
echo ""
echo "**1.** prosalinjen som skal overleve."
} > "$ROOT/rt-neste/STATE.md"
OUT2="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
printf '%s' "$OUT2" | grep -q 'prosalinjen som skal overleve'
check "route line under NESTE leaves the prose in the NESTE column" $?
printf '%s' "$LINE" | grep -q '^<!-- route: .* -->$'
check "route line is a single-line HTML comment" $?
# --- 8. Rationale is untrusted text on a comment line ---------------------
# It is free text typed by a session and it lands inside an HTML comment on one
# line. A newline splits the line (and hands the next line to the NESTE
# extractor); a literal '-->' closes the comment early and dumps the rest into
# the rendered STATE.md. Same line-oriented sanitizing the send side does.
LINE2="$("$R" --path known --verification strong --reversibility cheap --scope local \
--rationale "$(printf 'first\nsecond')" 2>/dev/null | sed -n 's/^route-line=//p')"
[ "$(printf '%s' "$LINE2" | wc -l | tr -d ' ')" -eq 0 ]
check "newline in rationale does not split the route line" $?
LINE3="$("$R" --path known --verification strong --reversibility cheap --scope local \
--rationale 'oops --> loose text' 2>/dev/null | sed -n 's/^route-line=//p')"
[ "$(printf '%s' "$LINE3" | grep -c -- '-->')" -eq 1 ]
check "'-->' in rationale cannot close the comment early" $?
# --- 8b. The last-session record ------------------------------------------
# The record is the cheap proxy for whether the routing itself is any good:
# systematically high corrections on row 1 means the cheap row is too easy to
# reach, systematically zero on row 4 means escalation fires too readily. It
# gets a WRITER here for the same reason next-cost needed one - a format with
# only a reader drifts. It is pure telemetry - it never changes what the
# calculator outputs, Fable rows included, which are unreachable through it
# entirely now.
LAST="$("$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-model "Opus 5" --last-effort xhigh \
--last-completed no --last-corrections 3 2>/dev/null | sed -n 's/^route-last=//p')"
printf '%s' "$LAST" | grep -q '^<!-- route-last: model=Opus 5; effort=xhigh; completed=no; corrections=3 -->$'
check "route-last line is emitted in the pinned single-line form" $?
# Omitted record must emit no line at all rather than a half-filled one: a
# blank record read back later is indistinguishable from a real measurement.
out_norec="$("$R" --path known --verification strong --reversibility cheap \
--scope local --rationale x 2>/dev/null)"
if printf '%s' "$out_norec" | grep -q '^route-last='; then rc=1; else rc=0; fi
check "no route-last line when the record is omitted" "$rc"
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-completed maybe >/dev/null 2>&1
[ $? -eq 2 ]; check "invalid --last-completed exits 2" $?
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-corrections three >/dev/null 2>&1
[ $? -eq 2 ]; check "non-numeric --last-corrections exits 2" $?
# The record is read back by the NEXT session as evidence months from now, so
# its model and effort are compared, not just displayed. Leaving them as free
# text would rebuild the exact reader-versus-writer drift this script exists to
# kill, one field over. Both are closed sets: the row table's three model names
# and the verified effort levels.
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-model "opus 5" --last-effort xhigh \
--last-completed no --last-corrections 1 >/dev/null 2>&1
[ $? -eq 2 ]; check "--last-model rejects a non-rubric spelling" $?
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-model "Opus 5" --last-effort extreme \
--last-completed no --last-corrections 1 >/dev/null 2>&1
[ $? -eq 2 ]; check "--last-effort rejects a value outside the verified set" $?
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-model "Fable 5" --last-effort medium \
--last-completed yes --last-corrections 0 >/dev/null 2>&1
[ $? -eq 0 ]; check "--last-model/-effort accept every legal value" $?
# The record is telemetry and must NOT silently change what the calculator
# outputs - a "completed=no" record describes what happened, and covers
# context exhaustion, an operator interrupt and a block on another repo just
# as much as an actual model failure. Reading it as an inference would revive
# exactly the escalation path the removed --opus-xhigh-failed flag used to
# gate deliberately.
got="$("$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-model "Opus 5" --last-effort xhigh --last-completed no \
--last-corrections 4 2>/dev/null | sed -n 's/^next-cost=//p')"
[ "$got" = "Sonnet 5/high" ]; check "a failed-session record alone does not change the routing outcome" $?
# A record is all four fields or none. A partial one emits `corrections=` with
# nothing after it, which reads back later exactly like a measured zero.
"$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x --last-model "Opus 5" --last-effort xhigh >/dev/null 2>&1
[ $? -eq 2 ]; check "a partial last-session record exits 2" $?
# All three comment lines stacked under the heading must still leave the prose
# in the NESTE column - that is the arrangement a real STATE.md ends up with.
mkdir -p "$ROOT/rt-three" && git -C "$ROOT/rt-three" init -q 2>/dev/null
{ echo "# STATE - rt-three"
printf '## %s NESTE %s START HER\n' "$HAND" "$EMDASH"
echo "<!-- board: status=in-progress; blocked-on=-; next-cost=Sonnet 5/high -->"
echo "$LINE"
echo "$LAST"
echo ""
# Kept well under board.sh's 38-char NESTE truncation: a longer line would
# be cut mid-word and fail this grep for a reason that has nothing to do
# with what is being tested.
echo "**1.** tredje prosalinje."
} > "$ROOT/rt-three/STATE.md"
OUT3="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
printf '%s' "$OUT3" | grep -q 'tredje prosalinje'
check "board + route + route-last stacked still yield prose in NESTE" $?
# --- 9. Startup command and fallback --------------------------------------
# Two spellings of ONE decision: the rubric name for the board line, the CLI
# alias for the command the operator pastes. They must never disagree.
# These assert the WHOLE string, advisor included, so section 14's rule cannot
# be widened without a deliberate edit here - the exact-match is the tripwire.
cmd="$(field partial strong cheap local command)"
[ "$cmd" = "claude --model opus --effort high" ]; check "command mirrors the row (opus/high)" $?
cmd="$(field known strong cheap local command)"
[ "$cmd" = "claude --model sonnet --effort high --advisor opus" ]; check "command mirrors the row (sonnet/high)" $?
# The rubric requires ALWAYS naming one row cheaper as the quota fallback.
fb="$(field partial strong cheap local fallback)"
[ "$fb" = "Sonnet 5/xhigh" ]; check "fallback is exactly one row cheaper" $?
fb="$(field known strong cheap local fallback)"
[ "$fb" = "Sonnet 5/high" ]; check "row 1 fallback floors at row 1, never below" $?
# The fallback needs its own pasteable command or the operator translates by
# hand at exactly the moment they are under quota pressure.
fbc="$(field partial strong cheap local fallback-command)"
[ "$fbc" = "claude --model sonnet --effort xhigh --advisor opus" ]; check "fallback ships its own command" $?
# --- 10. The command carries no 'cd' --------------------------------------
# One repo per terminal tab: a startup command prefixed with cd is wrong by
# construction, and another directory means another tab.
out="$("$R" --path known --verification strong --reversibility cheap --scope local \
--rationale x 2>/dev/null)"
if printf '%s' "$out" | grep -q 'cd '; then rc=1; else rc=0; fi
check "no emitted command contains a cd prefix" "$rc"
# --- 11. Effort and model values are the VERIFIED sets --------------------
# Effort levels are pinned in this marketplace at
# config-audit/scanners/settings-validator.mjs (low|medium|high|xhigh|max).
# Model aliases are whatever the INSTALLED claude accepts - never hardcoded
# without a gate, because an alias that stops resolving turns every emitted
# command into a paste that fails.
# Capture the effort TOKEN only. Anything may legitimately follow it on the
# command line (--advisor does, since section 14), and a match that swallowed
# the tail would report a valid effort as invalid.
efforts="$(printf '%s' "$out" | sed -n 's/^command=claude --model [a-z]* --effort \([a-z]*\).*/\1/p')"
case "|low|medium|high|xhigh|max|" in *"|$efforts|"*) rc=0 ;; *) rc=1 ;; esac
check "emitted effort is in the verified effort set" "$rc"
if command -v claude >/dev/null 2>&1; then
# Match ONLY the quoted alias as --model documents it. An unanchored grep for
# the bare word would hit "opus" anywhere in the help text and pass even if
# --model stopped accepting the alias entirely - a gate that reports success
# without testing anything, which is worse than no gate.
HELPTXT="$(claude --help 2>&1)"
miss=""
for alias in fable opus sonnet; do
printf '%s' "$HELPTXT" | grep -q "'$alias'" || miss="$miss $alias"
done
[ -z "$miss" ]; check "installed claude documents the aliases route emits:${miss:- all three}" $?
else
echo " skip - claude not on PATH, model alias gate not run"
fi
# --- 12. One spec, and it is the row table --------------------------------
# The last board fix closed a defect whose root cause was this repo's own
# --help being the SECOND spec for next-cost. route.sh must not reopen it: its
# help may describe the row table (it owns it now) but must not restate the
# board line grammar, which board.sh --help owns.
HELP="$("$R" --help 2>/dev/null)"; rc=$?
[ "$rc" -eq 0 ]; check "--help exits 0" $?
printf '%s' "$HELP" | grep -q 'Sonnet 5/high'
check "--help shows the canonical rubric spelling" $?
if printf '%s' "$HELP" | grep -qE '(sonnet|opus|fable) ?5?/(high|xhigh)'; then rc=1; else rc=0; fi
check "--help shows no versionless model example" "$rc"
if printf '%s' "$HELP" | grep -q 'blocked-on='; then rc=1; else rc=0; fi
check "--help does not restate the board line grammar" "$rc"
# --- 13. Where --last-effort comes from -----------------------------------
# The record exists to make the policy falsifiable, which it only is if the
# effort field is MEASURED. Two earlier sources both measured the wrong thing:
# the previous board line holds what was PRESCRIBED, and asking the operator
# launders that same prescription through a human who is reading it off the
# startup command they typed. $CLAUDE_EFFORT is what the session actually
# resolved - it is documented as the CURRENT effort level and is exported into
# every tool-use context, which is why a Bash call can read it at all.
#
# The trap this section exists to pin: skill frontmatter can set `effort`, and
# frontmatter overrides the session level while that skill is active. An
# `effort:` field in route's own SKILL.md would therefore make the reading
# report the SKILL's effort and not the session's - a measurement quietly
# measuring itself, with nothing in the output to show it happened.
SKILL="$DIR/../skills/route/SKILL.md"
[ -f "$SKILL" ]; check "route SKILL.md is where the selftest expects it" $?
# Frontmatter only: the body must be free to DISCUSS effort at length.
FM="$(awk 'NR==1 && /^---$/ {f=1; next} f && /^---$/ {exit} f {print}' "$SKILL" 2>/dev/null)"
if printf '%s' "$FM" | grep -q '^effort:'; then rc=1; else rc=0; fi
check "route SKILL.md declares no effort: frontmatter field" "$rc"
grep -q 'CLAUDE_EFFORT' "$SKILL"
check "route SKILL.md names CLAUDE_EFFORT as the --last-effort source" $?
grep -q 'board line' "$SKILL"
check "route SKILL.md still warns off the previous board line" $?
# route.sh carried the claim as a documented premise. It was true when written
# and is not any more, so it must not survive as a comment that reads like a
# measured fact three weeks from now.
FLAT="$(tr '\n' ' ' < "$R" | sed 's/#//g' | tr -s ' ')"
if printf '%s' "$FLAT" | grep -q 'is not observable from inside'; then rc=1; else rc=0; fi
check "route.sh no longer claims effort is unobservable from inside" "$rc"
grep -q 'CLAUDE_EFFORT' "$R"
check "route.sh names the source the caller should measure from" $?
# --- 14. The advisor: capability gap first, stakes second -----------------
# The advisor is a second, stronger model consulted mid-task, so a session
# carrying one costs more than a session without. It must therefore fire where
# there is a NEED and nowhere else, or it decays into the always-on
# advisorModel setting it exists to replace. Two independent needs qualify,
# and they turn out to be almost disjoint.
#
# 1. THE MAIN MODEL IS SONNET (rows 1-2). Here opus is a genuine capability
# lift, not a peer: opus judgement at sonnet cost. This is the load-bearing
# half, because every fallback-command is one row cheaper and the cheapest
# rows are Sonnet - so this is what makes the QUOTA FALLBACK safe to take.
# Pinned below by the row-3-falls-back-to-row-2 case.
#
# 2. A MISTAKE IS EXPENSIVE TO UNDO (reversibility costly|one-way). On an Opus
# row the advisor is a peer review rather than a lift, which is worth
# paying for when being wrong is not cheap to reverse.
#
# The two barely overlap: costly forces row 3 and one-way forces row 4, so a
# Sonnet row ALWAYS has reversibility=cheap and trigger 2 can never reach it.
# Rule 1 covers the Sonnet rows, rule 2 covers the Opus rows.
#
# What is deliberately NOT a trigger: verification=none on its own. The set it
# would add beyond rule 2 is exactly {verification=none AND
# reversibility=cheap} - mistakes that are cheap to reverse. It survives on the
# Sonnet rows only via rule 1, which is about the model, not the trait.
adv="$(field known strong cheap local command)"
printf '%s' "$adv" | grep -q -- '--advisor opus'
check "row 1 (Sonnet/high) always carries an advisor" $?
adv="$(field known weak cheap local command)"
printf '%s' "$adv" | grep -q -- '--advisor opus'
check "row 2 (Sonnet/xhigh) always carries an advisor" $?
adv="$(field known strong costly local command)"
printf '%s' "$adv" | grep -q -- '--advisor opus'
check "reversibility=costly carries an advisor onto an Opus row" $?
adv="$(field known strong one-way local command)"
printf '%s' "$adv" | grep -q -- '--advisor opus'
check "reversibility=one-way carries an advisor onto an Opus row" $?
# The Opus rows reached WITHOUT expensive stakes are the whole no-advisor set.
adv="$(field partial strong cheap local command)"
if printf '%s' "$adv" | grep -q -- '--advisor'; then rc=1; else rc=0; fi
check "row 3 via path=partial at cheap stakes takes NO advisor" "$rc"
adv="$(field known strong cheap multi-file command)"
if printf '%s' "$adv" | grep -q -- '--advisor'; then rc=1; else rc=0; fi
check "row 3 via scope=multi-file at cheap stakes takes NO advisor" "$rc"
adv="$(field known strong cheap cross-cutting command)"
if printf '%s' "$adv" | grep -q -- '--advisor'; then rc=1; else rc=0; fi
check "row 4 via scope=cross-cutting at cheap stakes takes NO advisor" "$rc"
# Rows 5-6 (Fable) are unreachable through this calculator now - section 1
# pins that directly, and section 3's closed vocabulary covers every command
# this script can ever emit. There is nothing left here for "Fable takes no
# advisor" to test: that fact still holds (gated against the installed claude
# by the CLI check below), but row_advisor()'s branch for it can no longer be
# exercised through the public interface.
adv="$(field known strong one-way local fallback-command)"
printf '%s' "$adv" | grep -q -- '--advisor opus'
check "row 4 fallback to row 3 keeps the advisor at one-way stakes" $?
# THE POINT OF RULE 1, pinned. Row 3 at cheap stakes takes no advisor, but its
# quota fallback is row 2 - a Sonnet row - which does. Dropping a row under
# quota pressure must not silently drop the safety net with it.
adv="$(field partial strong cheap local fallback-command)"
printf '%s' "$adv" | grep -q -- '--advisor opus'
check "an Opus row with no advisor still falls back to an advised Sonnet row" $?
adv="$(field known strong cheap cross-cutting fallback-command)"
if printf '%s' "$adv" | grep -q -- '--advisor'; then rc=1; else rc=0; fi
check "row 4 falling back to row 3 at cheap stakes stays unadvised" "$rc"
# Closed vocabulary, same reason as the row table: one spelling in circulation.
# fable is not merely weaker here, it is REJECTED as an advisor outright.
allout="$("$R" --path known --verification strong --reversibility one-way \
--scope cross-cutting --rationale x 2>/dev/null)"
bad="$(printf '%s' "$allout" | grep -o -- '--advisor [a-z0-9-]*' | grep -v -- '--advisor opus')"
[ -z "$bad" ]; check "the only advisor value ever emitted is opus:${bad:- none other}" $?
if command -v claude >/dev/null 2>&1; then
# A real gate that spends NO tokens. Advisor validation runs BEFORE the
# empty-prompt check, so `-p ""` reaches the validator and then exits: a
# valid advisor fails on the missing prompt, an invalid one fails on itself.
# --help cannot gate this - it short-circuits before option validation, so
# even an unknown flag exits 0 and the gate would pass without testing.
# Row 1-2's trigger is "sonnet main + opus advisor" specifically, so the
# main model is pinned rather than left to the ambient default.
ADVOUT="$(claude --model sonnet --advisor opus -p "" 2>&1 | head -3)"
if printf '%s' "$ADVOUT" | grep -q 'unknown option'; then rc=1; else rc=0; fi
check "installed claude accepts the --advisor flag at all" "$rc"
if printf '%s' "$ADVOUT" | grep -q 'cannot advise'; then rc=1; else rc=0; fi
check "installed claude accepts opus advising a sonnet main" "$rc"
# row_advisor() never emits '--advisor fable' - it depends on the opposite
# direction: whether opus can advise a FABLE main model. That is the fact
# rows 5-6's suppression rests on, so that is what must be gated. (Measured
# at CC 2.1.226: fable is now itself accepted as an advisor for opus/sonnet
# mains - a capability-ordering change worth flagging to the operator, but
# not what this check exists to pin.)
FABOUT="$(claude --model fable --advisor opus -p "" 2>&1 | head -3)"
printf '%s' "$FABOUT" | grep -q 'cannot advise'
check "installed claude still rejects opus advising a fable main" $?
else
echo " skip - claude not on PATH, advisor flag gate not run"
fi
echo ""
echo "route-selftest: $PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ] || exit 1
exit 0