fix(route): --last-model learns Fable 5.1, and the set stays closed
Fable 5.1 shipped 2026-09-01. The closed set at route.sh:212 refused it,
so a session that actually ran it could not record what it ran: the
record was either omitted or LIED, and a lied record reads back months
later as a measurement rather than as the gap it is.
TDD, test first. The exact failure the new check produced before the fix:
route: --last-model: 'Fable 5.1' is not a row-table model (Sonnet 5|Opus 5|Fable 5)
Two boundary decisions, both taken here and both written into the code:
(a) "Fable 5" is KEPT alongside the point release. The reason is not
backward compatibility with existing route-last lines - measured
across the machine, exactly 1 of 45 carries it. It is that route.sh's
OWN row table spells rows 5-6 "Fable 5/high" and "Fable 5/xhigh", so
dropping the value would make the script refuse to record a name its
own spec writes.
(b) The set is WIDENED, never replaced by form validation. A pattern like
"<family> <digits>[.<digits>]" would still catch a misspelled family
and a drifted case, and would stop catching a version that does not
exist: "Fable 5.2" and "Opus 7" would both pass and read back as
evidence that a model ran when it never shipped. This field is
telemetry read as evidence, so a silently-accepted lie is worse than
a loud refusal. The cost is real and was paid before the choice was
made, so the die message now names the repair instead of leaving the
caller to approximate to a value already in the list.
Both edit sites, never one: the usage block (route.sh:116) and the case
itself. Fixing the case alone is the two-copies-of-one-policy defect this
repo names repeatedly.
Selftest, 69 -> 73 checks:
- Fable 5.1 is accepted (was RED)
- route-last carries "Fable 5.1" verbatim into the line (was RED)
- Fable 5.2 / Fable 6 / Fable 5.10 are still REFUSED - the
check that makes "we did not switch to form validation"
machine-verified rather than prose (control)
- board parses back a hand-written Fable 5.1 next-cost (control)
MEASURED GAP, stated rather than closed: "Fable 5.1/xhigh" is 15
characters and overflows board.sh's %-14s KOST column, shifting the rest
of that row one column right. Parsing is unaffected. Widening the column
is a board.sh rendering change nobody ordered in this session, so it is
reported rather than fixed - which is also why the value is deliberately
absent from the widest-value loop in selftest section 6.
Not touched: the row table (still six rows, route.sh still emits only
1-4), and the note that a Fable session runs without an advisor because
the CLI does not enforce it.
Order: 20260901T185028Z-3612729538-from-.claude
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
85cd628c3e
commit
2bec7fe2fb
2 changed files with 74 additions and 3 deletions
|
|
@ -169,6 +169,16 @@ done
|
||||||
# rather than produced by "$R".
|
# rather than produced by "$R".
|
||||||
rt_case "rt-5" "Fable 5/high"
|
rt_case "rt-5" "Fable 5/high"
|
||||||
rt_case "rt-6" "Fable 5/xhigh"
|
rt_case "rt-6" "Fable 5/xhigh"
|
||||||
|
# A hand-written Fable 5.1 board line. next-cost extraction is free text, so
|
||||||
|
# board.sh parses the point release back unchanged - pinned here so a later
|
||||||
|
# narrowing of that extraction fails in this suite rather than in the
|
||||||
|
# operator's eye. MEASURED GAP, stated rather than closed: at 15 characters it
|
||||||
|
# overflows the %-14s KOST column and shifts the rest of that row one column
|
||||||
|
# right. That is a board.sh rendering change nobody ordered in this session, so
|
||||||
|
# it is reported to .claude, not fixed here - which is also why "Fable
|
||||||
|
# 5.1/xhigh" is deliberately absent from the widest-value loop below. Adding it
|
||||||
|
# there would go red, and the red would be the unfixed gap, not a broken test.
|
||||||
|
rt_case "rt-51" "Fable 5.1/xhigh"
|
||||||
|
|
||||||
OUT="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
|
OUT="$(CLAUDE_COORD_DIR="$MBOX" "$BOARD" --roots "$ROOT" 2>/dev/null)"
|
||||||
for want in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
|
for want in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
|
||||||
|
|
@ -176,6 +186,8 @@ for want in "Sonnet 5/high" "Sonnet 5/xhigh" "Opus 5/high" "Opus 5/xhigh" \
|
||||||
printf '%s' "$OUT" | grep -q "$want" || { rt_bad=$((rt_bad+1)); echo " board lost: [$want]"; }
|
printf '%s' "$OUT" | grep -q "$want" || { rt_bad=$((rt_bad+1)); echo " board lost: [$want]"; }
|
||||||
done
|
done
|
||||||
[ "$rt_bad" -eq 0 ]; check "round trip: board.sh parses back all 6 emitted values" $?
|
[ "$rt_bad" -eq 0 ]; check "round trip: board.sh parses back all 6 emitted values" $?
|
||||||
|
printf '%s' "$OUT" | grep -q 'Fable 5\.1/xhigh'
|
||||||
|
check "board parses back a hand-written Fable 5.1 next-cost" $?
|
||||||
|
|
||||||
# board.sh renders KOST with %-14s; a longer value shoves the whole row right
|
# board.sh renders KOST with %-14s; a longer value shoves the whole row right
|
||||||
# even though it parsed fine. Measure the widest string the table can emit -
|
# even though it parsed fine. Measure the widest string the table can emit -
|
||||||
|
|
@ -272,6 +284,43 @@ check "no route-last line when the record is omitted" "$rc"
|
||||||
--last-completed yes --last-corrections 0 >/dev/null 2>&1
|
--last-completed yes --last-corrections 0 >/dev/null 2>&1
|
||||||
[ $? -eq 0 ]; check "--last-model/-effort accept every legal value" $?
|
[ $? -eq 0 ]; check "--last-model/-effort accept every legal value" $?
|
||||||
|
|
||||||
|
# Fable 5.1 shipped 2026-09-01 and the closed set refused it, so a session that
|
||||||
|
# actually ran it could not record what it ran: the record was either omitted
|
||||||
|
# or LIED, and a lied record reads back months later as a measurement. The set
|
||||||
|
# is WIDENED, never replaced by form validation - the check below is what makes
|
||||||
|
# that choice machine-verified instead of prose. "Fable 5" stays legal for a
|
||||||
|
# reason stronger than the one STATE.md on this machine that still carries it:
|
||||||
|
# route.sh's OWN row table spells rows 5-6 "Fable 5/high" and "Fable 5/xhigh",
|
||||||
|
# so dropping it would make the script refuse to record a value its own spec
|
||||||
|
# names. The check above this one is what goes red if anyone drops it.
|
||||||
|
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||||
|
--rationale x --last-model "Fable 5.1" --last-effort xhigh \
|
||||||
|
--last-completed yes --last-corrections 0 >/dev/null 2>&1
|
||||||
|
[ $? -eq 0 ]; check "--last-model accepts the Fable 5.1 point release" $?
|
||||||
|
|
||||||
|
# The set is still CLOSED after being widened, and this is the whole cost of
|
||||||
|
# NOT switching to form validation. A pattern like "<family> <digits>[.<digits>]"
|
||||||
|
# would accept every line below, and would stop catching a version that does
|
||||||
|
# not exist - which reads back later as evidence that a model ran when it never
|
||||||
|
# shipped. That is the positive-looking null this repo refuses everywhere else.
|
||||||
|
fable_bad=0
|
||||||
|
for bad in "Fable 5.2" "Fable 6" "Fable 5.10"; do
|
||||||
|
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||||
|
--rationale x --last-model "$bad" --last-effort xhigh \
|
||||||
|
--last-completed yes --last-corrections 0 >/dev/null 2>&1
|
||||||
|
[ $? -eq 2 ] || { fable_bad=$((fable_bad+1)); echo " accepted a model that does not exist: [$bad]"; }
|
||||||
|
done
|
||||||
|
[ "$fable_bad" -eq 0 ]; check "the model set stays CLOSED after Fable 5.1 (no form validation)" $?
|
||||||
|
|
||||||
|
# Accepting the value is not the same as RECORDING it. The record is what the
|
||||||
|
# next session reads back, so the emitted line must carry the point release
|
||||||
|
# verbatim rather than collapsing it to the family name.
|
||||||
|
LAST51="$("$R" --path known --verification strong --reversibility cheap --scope local \
|
||||||
|
--rationale x --last-model "Fable 5.1" --last-effort xhigh \
|
||||||
|
--last-completed yes --last-corrections 0 2>/dev/null | sed -n 's/^route-last=//p')"
|
||||||
|
printf '%s' "$LAST51" | grep -q '^<!-- route-last: model=Fable 5.1; effort=xhigh; completed=yes; corrections=0 -->$'
|
||||||
|
check "route-last carries Fable 5.1 verbatim into the emitted line" $?
|
||||||
|
|
||||||
# The record is telemetry and must NOT silently change what the calculator
|
# The record is telemetry and must NOT silently change what the calculator
|
||||||
# outputs - a "completed=no" record describes what happened, and covers
|
# outputs - a "completed=no" record describes what happened, and covers
|
||||||
# context exhaustion, an operator interrupt and a block on another repo just
|
# context exhaustion, an operator interrupt and a block on another repo just
|
||||||
|
|
|
||||||
|
|
@ -113,7 +113,7 @@
|
||||||
# route.sh --path <v> --verification <v> --reversibility <v> --scope <v>
|
# route.sh --path <v> --verification <v> --reversibility <v> --scope <v>
|
||||||
# --rationale <text>
|
# --rationale <text>
|
||||||
#
|
#
|
||||||
# route.sh ... --last-model <Sonnet 5|Opus 5|Fable 5>
|
# route.sh ... --last-model <Sonnet 5|Opus 5|Fable 5|Fable 5.1>
|
||||||
# --last-effort <low|medium|high|xhigh|max>
|
# --last-effort <low|medium|high|xhigh|max>
|
||||||
# --last-completed <yes|no> --last-corrections <n>
|
# --last-completed <yes|no> --last-corrections <n>
|
||||||
#
|
#
|
||||||
|
|
@ -208,9 +208,31 @@ if [ "$L_SET" -eq 1 ]; then
|
||||||
# this record back as evidence months from now, so a drifted spelling
|
# this record back as evidence months from now, so a drifted spelling
|
||||||
# ("opus 5" for "Opus 5") rebuilds the reader-versus-writer drift this whole
|
# ("opus 5" for "Opus 5") rebuilds the reader-versus-writer drift this whole
|
||||||
# script exists to remove, one field over.
|
# script exists to remove, one field over.
|
||||||
|
# THE SET IS CLOSED, AND STAYS CLOSED - decided 2026-09-01 when Fable 5.1
|
||||||
|
# shipped and was refused here. Both boundary questions were live:
|
||||||
|
#
|
||||||
|
# (a) "Fable 5" is KEPT alongside the point release. The reason is not
|
||||||
|
# backward compatibility with the one STATE.md on this machine that still
|
||||||
|
# carries it (measured: 1 of 45 route-last lines) - it is that the row
|
||||||
|
# table above spells rows 5-6 "Fable 5/high" and "Fable 5/xhigh".
|
||||||
|
# Dropping the value would make this script refuse to record a name its
|
||||||
|
# own spec writes.
|
||||||
|
#
|
||||||
|
# (b) The set was WIDENED rather than replaced by form validation. A pattern
|
||||||
|
# like "<family> <digits>[.<digits>]" would still catch a misspelled
|
||||||
|
# family and a drifted case, and would stop catching A VERSION THAT DOES
|
||||||
|
# NOT EXIST: "Fable 5.2" and "Opus 7" would both pass and read back
|
||||||
|
# months later as evidence that a model ran when it never shipped. This
|
||||||
|
# field is telemetry read as evidence, so a silently-accepted lie is
|
||||||
|
# worse than a loud refusal.
|
||||||
|
#
|
||||||
|
# The cost of that choice is real and was paid before it was made: a session
|
||||||
|
# that genuinely ran Fable 5.1 could not record it, so its record was omitted
|
||||||
|
# or lied. The list must therefore be extended the day a model ships, and the
|
||||||
|
# die message says so rather than leaving the caller to guess.
|
||||||
case "$L_MODEL" in
|
case "$L_MODEL" in
|
||||||
"Sonnet 5"|"Opus 5"|"Fable 5") ;;
|
"Sonnet 5"|"Opus 5"|"Fable 5"|"Fable 5.1") ;;
|
||||||
*) die "--last-model: '$L_MODEL' is not a row-table model (Sonnet 5|Opus 5|Fable 5)" ;;
|
*) die "--last-model: '$L_MODEL' is not a row-table model (Sonnet 5|Opus 5|Fable 5|Fable 5.1) - a newly shipped model must be added to this list in route.sh, never approximated to a name that is already in it" ;;
|
||||||
esac
|
esac
|
||||||
case "$L_EFFORT" in
|
case "$L_EFFORT" in
|
||||||
low|medium|high|xhigh|max) ;;
|
low|medium|high|xhigh|max) ;;
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue