Kjell Tore Guttormsen
ac62a38b40
The last-session record exists to make the routing policy falsifiable, and it only is if --last-effort is measured. route.sh documented the opposite as settled fact: that the effort a session ran with is not observable from inside that session. That was true when written and is not now. Claude Code exports CLAUDE_EFFORT into every tool-use context as the session's current effort level, so a Bash call reads it directly. The premise had a cost. With effort unobservable, the record could only be completed by asking the operator at session end, which made it block on their presence -- all four fields or none. That is also the weaker measurement, and in the same way the previous board line is: the operator reads the effort off the startup command they typed, so both sources report what was PRESCRIBED rather than what was RUN. They come apart exactly when the record would be most interesting, which is what a session that silently ran xhigh under a board line saying high already showed. Reading it makes all four fields knowable from inside the ending session, so the record no longer waits on anyone. The skill does the reading; route.sh deliberately does NOT default from the variable, because a calculator that consults its environment is no longer deterministic from its arguments and the route->board round trip in selftest section 6 rests on that. Section 13 also pins the trap this opens: skill frontmatter overrides the session effort while that skill is active, so an effort: field in route's own SKILL.md would make the reading report the skill instead of the session -- a measurement quietly measuring itself, with nothing in the output to show it happened. Also corrects the neighbouring claim that the model is readable from the environment. There is no CLAUDE_MODEL; the session takes it from what it knows itself to be running as. route-selftest 50 -> 56. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gfa1nvwGXdST2MHvbs6htD
264 lines
13 KiB
Bash
Executable file
264 lines
13 KiB
Bash
Executable file
#!/bin/bash
|
|
# route.sh - score four traits of the NEXT task, get the model and effort to
|
|
# run it with. A pure calculator: reads nothing, writes nothing, prints one
|
|
# block of key=value lines on stdout. The session pastes the result into
|
|
# STATE.md; this script never touches a file.
|
|
#
|
|
# WHY THIS EXISTS. The board line's next-cost field had a reader (board.sh)
|
|
# and no writer, so its value was typed by hand every session and drifted into
|
|
# several competing spellings. Cleaning the data could not fix that.
|
|
# A writer with a CLOSED output range can: this script can only ever emit one
|
|
# of six strings, so a seventh cannot enter circulation.
|
|
#
|
|
# THE ROW TABLE IS THE POLICY, and it is the operator's rubric verbatim -
|
|
# moved here so there is one copy rather than one per repo:
|
|
#
|
|
# 1 Sonnet 5/high reading, summarizing, docs, mechanical refactor
|
|
# 2 Sonnet 5/xhigh TDD cycle, known-root-cause bugfix, one-file change
|
|
# 3 Opus 5/high multi-file feature, architecture choice, hard debugging
|
|
# 4 Opus 5/xhigh long autonomous run, big refactor, cross-repo migration
|
|
# 5 Fable 5/high Opus 5/xhigh has ACTUALLY failed on this same step
|
|
# 6 Fable 5/xhigh open problem, no known solution path, longest horizon
|
|
#
|
|
# Cheapest first, so the rubric's "always name one row cheaper as the quota
|
|
# fallback" is row minus one, floored at row 1, correct by construction.
|
|
#
|
|
# THE TRAITS describe the task, never how it feels. "Hard", "complex" and
|
|
# "important" are deliberately absent: they are unfalsifiable and collapse to
|
|
# a hunch, which is the thing being replaced.
|
|
#
|
|
# --path known | partial | undetermined
|
|
# Is the solution route described, or must it be found?
|
|
# --verification strong | weak | none
|
|
# Will tests, types or a compiler catch the error?
|
|
# --reversibility cheap | costly | one-way
|
|
# --scope local | multi-file | cross-cutting
|
|
# --rationale required free text - WHY these four scores
|
|
#
|
|
# All five are required. None has a default, and that is load-bearing:
|
|
# verification carries the most signal and is the trait most often left out,
|
|
# and a default would be indistinguishable from a real score when the log is
|
|
# read back to find out whether the ROUTING was wrong or the SCORING was.
|
|
#
|
|
# SELECTION - first match wins, most expensive first:
|
|
# row 6 history says Opus 5/xhigh failed here, and path=undetermined
|
|
# row 5 history says Opus 5/xhigh failed here
|
|
# row 4 reversibility=one-way OR scope=cross-cutting
|
|
# row 3 path=partial|undetermined OR reversibility=costly OR scope=multi-file
|
|
# row 2 verification=weak|none
|
|
# row 1 otherwise
|
|
#
|
|
# Escalation is ASYMMETRIC on purpose: any single trait escalates, while
|
|
# reaching row 1 needs all four at the cheap end. Underkill costs one session,
|
|
# overkill costs quota every session - but a wrong architecture decision in a
|
|
# published plugin costs more than either.
|
|
#
|
|
# WHERE IT DISAGREES WITH THE RUBRIC'S EXAMPLES. The rows are task-type labels;
|
|
# the traits are a different classification over the same six outcomes. They
|
|
# part company by one row on the two cheapest rows - documentation scores
|
|
# known/none/cheap/local and lands on row 2 where the rubric's prose says row
|
|
# 1; a TDD cycle scores known/strong/cheap/local and lands on row 1 where the
|
|
# prose says row 2. This is left alone deliberately. Chasing the example
|
|
# phrases would mean re-implementing description-matching, which is the guess
|
|
# the traits exist to replace. It is a policy, not a theorem: --rationale is
|
|
# how a misscore is found afterwards.
|
|
#
|
|
# THE SPECIFICATION CHECK THAT COMES FREE. path=undetermined with no planned
|
|
# design phase means the TASK DESCRIPTION is underspecified - not that the
|
|
# model should be upgraded. Rewrite the next step; upgrading the model to
|
|
# compensate for a vague spec is the most expensive form of procrastination
|
|
# available.
|
|
#
|
|
# Usage:
|
|
# route.sh --path <v> --verification <v> --reversibility <v> --scope <v>
|
|
# --rationale <text> [--opus-xhigh-failed]
|
|
#
|
|
# route.sh ... --last-model <Sonnet 5|Opus 5|Fable 5>
|
|
# --last-effort <low|medium|high|xhigh|max>
|
|
# --last-completed <yes|no> --last-corrections <n>
|
|
#
|
|
# --opus-xhigh-failed is the ONLY way rows 5 and 6 can fire, and it means one
|
|
# narrow thing: an Opus 5/xhigh session already ran at THIS SAME next step and
|
|
# did not complete it. Not "a session failed" - context exhaustion, an operator
|
|
# interrupt or a block on another repo are not this flag. It escalates to a
|
|
# model that runs without an advisor, so when in doubt, leave it off.
|
|
#
|
|
# THE LAST-SESSION RECORD (the four --last-* fields, all or none) is what makes
|
|
# any of this falsifiable. It records how the session that just ran actually
|
|
# went, so the policy can later be judged against outcomes instead of against
|
|
# how sensible it reads. --last-corrections is the cheap proxy: systematically
|
|
# high counts on row 1 mean the cheap row is too easy to reach, systematically
|
|
# zero on row 4 means escalation fires too readily.
|
|
#
|
|
# All four fields are closed sets or numbers, and required together, because the
|
|
# NEXT session reads the record back to decide --opus-xhigh-failed. Every one of
|
|
# them must be MEASURED by the caller: --last-effort comes from CLAUDE_EFFORT,
|
|
# which Claude Code exports into every tool-use context as the session's current
|
|
# effort level. It is deliberately NOT defaulted from that variable here - a
|
|
# calculator that reads its own environment stops being deterministic from its
|
|
# arguments, and section 6 of the selftest depends on that determinism. Reading
|
|
# the value off the previous board line would measure what was PRESCRIBED rather
|
|
# than what was RUN; so, less obviously, does asking the operator, who reads it
|
|
# off the startup command they typed. Omit the record rather than guess - a
|
|
# guessed value reads back as a measurement.
|
|
#
|
|
# It is deliberately NOT wired to the Fable rows. Inferring "the model failed"
|
|
# from "the session did not finish" would fire on context exhaustion and on
|
|
# operator interrupts, which say nothing about the model. The record is
|
|
# evidence; --opus-xhigh-failed is a judgement. Keeping them apart is why the
|
|
# evidence stays worth reading.
|
|
#
|
|
# Exit 0 on a decision, 2 on any bad or missing argument. ASCII only,
|
|
# bash 3.2 safe.
|
|
set -u
|
|
export LC_ALL=C
|
|
|
|
PATH_T=""; VERIF=""; REVERS=""; SCOPE=""; RATIONALE=""; RAT_SET=0; FAILED=0
|
|
L_MODEL=""; L_EFFORT=""; L_DONE=""; L_CORR=""; L_SET=0
|
|
|
|
die() { echo "route: $1" >&2; exit 2; }
|
|
|
|
need() { [ $# -ge 2 ] || die "$1 requires a value"; }
|
|
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
# bash 3.2: `shift 2` past the end of $# is a no-op -> would loop forever.
|
|
--path) need "$@"; PATH_T="$2"; shift 2 ;;
|
|
--verification) need "$@"; VERIF="$2"; shift 2 ;;
|
|
--reversibility) need "$@"; REVERS="$2"; shift 2 ;;
|
|
--scope) need "$@"; SCOPE="$2"; shift 2 ;;
|
|
--rationale) need "$@"; RATIONALE="$2"; RAT_SET=1; shift 2 ;;
|
|
--opus-xhigh-failed) FAILED=1; shift ;;
|
|
--last-model) need "$@"; L_MODEL="$2"; L_SET=1; shift 2 ;;
|
|
--last-effort) need "$@"; L_EFFORT="$2"; L_SET=1; shift 2 ;;
|
|
--last-completed) need "$@"; L_DONE="$2"; L_SET=1; shift 2 ;;
|
|
--last-corrections) need "$@"; L_CORR="$2"; L_SET=1; shift 2 ;;
|
|
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
|
*) die "unknown argument: $1" ;;
|
|
esac
|
|
done
|
|
|
|
# Unknown values are rejected rather than tolerated. A silently accepted
|
|
# typo would route on three traits and look exactly like a scored decision.
|
|
case "$PATH_T" in
|
|
known|partial|undetermined) ;;
|
|
"") die "--path is required (known|partial|undetermined)" ;;
|
|
*) die "--path: unknown value '$PATH_T' (known|partial|undetermined)" ;;
|
|
esac
|
|
case "$VERIF" in
|
|
strong|weak|none) ;;
|
|
"") die "--verification is required (strong|weak|none)" ;;
|
|
*) die "--verification: unknown value '$VERIF' (strong|weak|none)" ;;
|
|
esac
|
|
case "$REVERS" in
|
|
cheap|costly|one-way) ;;
|
|
"") die "--reversibility is required (cheap|costly|one-way)" ;;
|
|
*) die "--reversibility: unknown value '$REVERS' (cheap|costly|one-way)" ;;
|
|
esac
|
|
case "$SCOPE" in
|
|
local|multi-file|cross-cutting) ;;
|
|
"") die "--scope is required (local|multi-file|cross-cutting)" ;;
|
|
*) die "--scope: unknown value '$SCOPE' (local|multi-file|cross-cutting)" ;;
|
|
esac
|
|
[ "$RAT_SET" -eq 1 ] || die "--rationale is required (why these four scores)"
|
|
[ -n "$RATIONALE" ] || die "--rationale must not be empty"
|
|
|
|
# The last-session record is all four fields or none at all. A partial record
|
|
# emits an empty value that reads back later exactly like a measured one, and
|
|
# the whole purpose of the record is to be readable evidence months from now.
|
|
if [ "$L_SET" -eq 1 ]; then
|
|
[ -n "$L_MODEL" ] || die "--last-model is required with a last-session record"
|
|
[ -n "$L_EFFORT" ] || die "--last-effort is required with a last-session record"
|
|
[ -n "$L_DONE" ] || die "--last-completed is required with a last-session record"
|
|
[ -n "$L_CORR" ] || die "--last-corrections is required with a last-session record"
|
|
# Model and effort are closed sets here, not free text. The next session READS
|
|
# this record back to decide --opus-xhigh-failed, so a drifted spelling
|
|
# ("opus 5" for "Opus 5") rebuilds the reader-versus-writer drift this whole
|
|
# script exists to remove, one field over.
|
|
case "$L_MODEL" in
|
|
"Sonnet 5"|"Opus 5"|"Fable 5") ;;
|
|
*) die "--last-model: '$L_MODEL' is not a row-table model (Sonnet 5|Opus 5|Fable 5)" ;;
|
|
esac
|
|
case "$L_EFFORT" in
|
|
low|medium|high|xhigh|max) ;;
|
|
*) die "--last-effort: '$L_EFFORT' is not a verified effort level (low|medium|high|xhigh|max)" ;;
|
|
esac
|
|
case "$L_DONE" in
|
|
yes|no) ;;
|
|
*) die "--last-completed: unknown value '$L_DONE' (yes|no)" ;;
|
|
esac
|
|
case "$L_CORR" in
|
|
""|*[!0-9]*) die "--last-corrections must be a whole number, got '$L_CORR'" ;;
|
|
esac
|
|
fi
|
|
|
|
# --- Selection: first match wins, most expensive first ---------------------
|
|
if [ "$FAILED" -eq 1 ] && [ "$PATH_T" = "undetermined" ]; then
|
|
ROW=6; RULE="opus-xhigh-failed + path=undetermined"
|
|
elif [ "$FAILED" -eq 1 ]; then
|
|
ROW=5; RULE="opus-xhigh-failed"
|
|
elif [ "$REVERS" = "one-way" ]; then
|
|
ROW=4; RULE="reversibility=one-way"
|
|
elif [ "$SCOPE" = "cross-cutting" ]; then
|
|
ROW=4; RULE="scope=cross-cutting"
|
|
elif [ "$PATH_T" != "known" ]; then
|
|
ROW=3; RULE="path=$PATH_T"
|
|
elif [ "$REVERS" = "costly" ]; then
|
|
ROW=3; RULE="reversibility=costly"
|
|
elif [ "$SCOPE" = "multi-file" ]; then
|
|
ROW=3; RULE="scope=multi-file"
|
|
elif [ "$VERIF" != "strong" ]; then
|
|
ROW=2; RULE="verification=$VERIF"
|
|
else
|
|
ROW=1; RULE="no escalating trait (all four at the cheap end)"
|
|
fi
|
|
|
|
# --- The row table: one decision, two spellings ---------------------------
|
|
# The rubric name goes in the board line so repos compare by eye; the CLI alias
|
|
# goes in the command the operator pastes. Emitting both from one table is the
|
|
# point - two hand-maintained spellings of one decision is how they disagree.
|
|
# Aliases are gated against the installed claude by route-selftest.sh section
|
|
# 11, never assumed here.
|
|
row_name() {
|
|
case "$1" in
|
|
1) echo "Sonnet 5/high" ;; 2) echo "Sonnet 5/xhigh" ;;
|
|
3) echo "Opus 5/high" ;; 4) echo "Opus 5/xhigh" ;;
|
|
5) echo "Fable 5/high" ;; 6) echo "Fable 5/xhigh" ;;
|
|
esac
|
|
}
|
|
row_cmd() {
|
|
case "$1" in
|
|
1) echo "claude --model sonnet --effort high" ;;
|
|
2) echo "claude --model sonnet --effort xhigh" ;;
|
|
3) echo "claude --model opus --effort high" ;;
|
|
4) echo "claude --model opus --effort xhigh" ;;
|
|
5) echo "claude --model fable --effort high" ;;
|
|
6) echo "claude --model fable --effort xhigh" ;;
|
|
esac
|
|
}
|
|
|
|
FB=$((ROW - 1)); [ "$FB" -ge 1 ] || FB=1
|
|
|
|
# The rationale is free text from a session and lands inside an HTML comment on
|
|
# ONE line. A newline would split the line and hand the remainder to board.sh's
|
|
# NESTE extractor as the repo's next step; a literal '-->' would close the
|
|
# comment early and spill the rest into the rendered STATE.md. Same
|
|
# line-oriented sanitizing the send side applies to its own fields.
|
|
RAT_CLEAN="$(printf '%s' "$RATIONALE" | tr '\r\n' ' ' | tr -d '\000-\037' \
|
|
| sed 's/-->/-- >/g')"
|
|
|
|
# rationale goes LAST in the line: it is the only field that may contain a
|
|
# ';', so anything after it would be unparseable.
|
|
echo "row=$ROW"
|
|
echo "rule=$RULE"
|
|
echo "next-cost=$(row_name "$ROW")"
|
|
echo "command=$(row_cmd "$ROW")"
|
|
echo "fallback=$(row_name "$FB")"
|
|
echo "fallback-command=$(row_cmd "$FB")"
|
|
echo "route-line=<!-- route: path=$PATH_T; verification=$VERIF; reversibility=$REVERS; scope=$SCOPE; rationale=$RAT_CLEAN -->"
|
|
# No sanitizing needed on the record: all four fields are validated against
|
|
# closed sets above, so none of them can carry a newline or a '-->'.
|
|
if [ "$L_SET" -eq 1 ]; then
|
|
echo "route-last=<!-- route-last: model=$L_MODEL; effort=$L_EFFORT; completed=$L_DONE; corrections=$L_CORR -->"
|
|
fi
|
|
exit 0
|