fix(route): measure the effort a session ran with, instead of asking for it
The last-session record exists to make the routing policy falsifiable, and it only is if --last-effort is measured. route.sh documented the opposite as settled fact: that the effort a session ran with is not observable from inside that session. That was true when written and is not now. Claude Code exports CLAUDE_EFFORT into every tool-use context as the session's current effort level, so a Bash call reads it directly. The premise had a cost. With effort unobservable, the record could only be completed by asking the operator at session end, which made it block on their presence -- all four fields or none. That is also the weaker measurement, and in the same way the previous board line is: the operator reads the effort off the startup command they typed, so both sources report what was PRESCRIBED rather than what was RUN. They come apart exactly when the record would be most interesting, which is what a session that silently ran xhigh under a board line saying high already showed. Reading it makes all four fields knowable from inside the ending session, so the record no longer waits on anyone. The skill does the reading; route.sh deliberately does NOT default from the variable, because a calculator that consults its environment is no longer deterministic from its arguments and the route->board round trip in selftest section 6 rests on that. Section 13 also pins the trap this opens: skill frontmatter overrides the session effort while that skill is active, so an effort: field in route's own SKILL.md would make the reading report the skill instead of the session -- a measurement quietly measuring itself, with nothing in the output to show it happened. Also corrects the neighbouring claim that the model is readable from the environment. There is no CLAUDE_MODEL; the session takes it from what it knows itself to be running as. route-selftest 50 -> 56. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gfa1nvwGXdST2MHvbs6htD
This commit is contained in:
parent
6735298752
commit
ac62a38b40
5 changed files with 96 additions and 17 deletions
|
|
@ -371,6 +371,44 @@ check "--help shows no versionless model example" "$rc"
|
|||
if printf '%s' "$HELP" | grep -q 'blocked-on='; then rc=1; else rc=0; fi
|
||||
check "--help does not restate the board line grammar" "$rc"
|
||||
|
||||
# --- 13. Where --last-effort comes from -----------------------------------
|
||||
# The record exists to make the policy falsifiable, which it only is if the
|
||||
# effort field is MEASURED. Two earlier sources both measured the wrong thing:
|
||||
# the previous board line holds what was PRESCRIBED, and asking the operator
|
||||
# launders that same prescription through a human who is reading it off the
|
||||
# startup command they typed. $CLAUDE_EFFORT is what the session actually
|
||||
# resolved - it is documented as the CURRENT effort level and is exported into
|
||||
# every tool-use context, which is why a Bash call can read it at all.
|
||||
#
|
||||
# The trap this section exists to pin: skill frontmatter can set `effort`, and
|
||||
# frontmatter overrides the session level while that skill is active. An
|
||||
# `effort:` field in route's own SKILL.md would therefore make the reading
|
||||
# report the SKILL's effort and not the session's - a measurement quietly
|
||||
# measuring itself, with nothing in the output to show it happened.
|
||||
SKILL="$DIR/../skills/route/SKILL.md"
|
||||
[ -f "$SKILL" ]; check "route SKILL.md is where the selftest expects it" $?
|
||||
|
||||
# Frontmatter only: the body must be free to DISCUSS effort at length.
|
||||
FM="$(awk 'NR==1 && /^---$/ {f=1; next} f && /^---$/ {exit} f {print}' "$SKILL" 2>/dev/null)"
|
||||
if printf '%s' "$FM" | grep -q '^effort:'; then rc=1; else rc=0; fi
|
||||
check "route SKILL.md declares no effort: frontmatter field" "$rc"
|
||||
|
||||
grep -q 'CLAUDE_EFFORT' "$SKILL"
|
||||
check "route SKILL.md names CLAUDE_EFFORT as the --last-effort source" $?
|
||||
|
||||
grep -q 'board line' "$SKILL"
|
||||
check "route SKILL.md still warns off the previous board line" $?
|
||||
|
||||
# route.sh carried the claim as a documented premise. It was true when written
|
||||
# and is not any more, so it must not survive as a comment that reads like a
|
||||
# measured fact three weeks from now.
|
||||
FLAT="$(tr '\n' ' ' < "$R" | sed 's/#//g' | tr -s ' ')"
|
||||
if printf '%s' "$FLAT" | grep -q 'is not observable from inside'; then rc=1; else rc=0; fi
|
||||
check "route.sh no longer claims effort is unobservable from inside" "$rc"
|
||||
|
||||
grep -q 'CLAUDE_EFFORT' "$R"
|
||||
check "route.sh names the source the caller should measure from" $?
|
||||
|
||||
echo ""
|
||||
echo "route-selftest: $PASS passed, $FAIL failed"
|
||||
[ "$FAIL" -eq 0 ] || exit 1
|
||||
|
|
|
|||
|
|
@ -91,11 +91,16 @@
|
|||
# zero on row 4 means escalation fires too readily.
|
||||
#
|
||||
# All four fields are closed sets or numbers, and required together, because the
|
||||
# NEXT session reads the record back to decide --opus-xhigh-failed. Note what
|
||||
# the caller must supply rather than infer: the effort a session ran with is not
|
||||
# observable from inside that session, and reading it back off the previous
|
||||
# board line would measure what was PRESCRIBED, not what was RUN. Omit the
|
||||
# record rather than guess - a guessed value reads back as a measurement.
|
||||
# NEXT session reads the record back to decide --opus-xhigh-failed. Every one of
|
||||
# them must be MEASURED by the caller: --last-effort comes from CLAUDE_EFFORT,
|
||||
# which Claude Code exports into every tool-use context as the session's current
|
||||
# effort level. It is deliberately NOT defaulted from that variable here - a
|
||||
# calculator that reads its own environment stops being deterministic from its
|
||||
# arguments, and section 6 of the selftest depends on that determinism. Reading
|
||||
# the value off the previous board line would measure what was PRESCRIBED rather
|
||||
# than what was RUN; so, less obviously, does asking the operator, who reads it
|
||||
# off the startup command they typed. Omit the record rather than guess - a
|
||||
# guessed value reads back as a measurement.
|
||||
#
|
||||
# It is deliberately NOT wired to the Fable rows. Inferring "the model failed"
|
||||
# from "the session did not finish" would fire on context exhaustion and on
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue