fix(route): measure the effort a session ran with, instead of asking for it

The last-session record exists to make the routing policy falsifiable, and
it only is if --last-effort is measured. route.sh documented the opposite as
settled fact: that the effort a session ran with is not observable from
inside that session. That was true when written and is not now. Claude Code
exports CLAUDE_EFFORT into every tool-use context as the session's current
effort level, so a Bash call reads it directly.

The premise had a cost. With effort unobservable, the record could only be
completed by asking the operator at session end, which made it block on their
presence -- all four fields or none. That is also the weaker measurement, and
in the same way the previous board line is: the operator reads the effort off
the startup command they typed, so both sources report what was PRESCRIBED
rather than what was RUN. They come apart exactly when the record would be
most interesting, which is what a session that silently ran xhigh under a
board line saying high already showed.

Reading it makes all four fields knowable from inside the ending session, so
the record no longer waits on anyone. The skill does the reading; route.sh
deliberately does NOT default from the variable, because a calculator that
consults its environment is no longer deterministic from its arguments and
the route->board round trip in selftest section 6 rests on that.

Section 13 also pins the trap this opens: skill frontmatter overrides the
session effort while that skill is active, so an effort: field in route's own
SKILL.md would make the reading report the skill instead of the session --
a measurement quietly measuring itself, with nothing in the output to show
it happened.

Also corrects the neighbouring claim that the model is readable from the
environment. There is no CLAUDE_MODEL; the session takes it from what it
knows itself to be running as.

route-selftest 50 -> 56.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gfa1nvwGXdST2MHvbs6htD
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 17:21:10 +02:00
commit ac62a38b40
5 changed files with 96 additions and 17 deletions

View file

@ -53,7 +53,7 @@ marketplace plugin. Three components, one boundary:
rationale, and prints one block of `key=value` lines: the rubric row, the rule
that fired, the `next-cost` value, a pasteable startup command, the one-row
cheaper fallback, and the STATE.md comment lines. Pinned by
`route-selftest.sh` (50 checks).
`route-selftest.sh` (56 checks).
**It is here because it is the WRITER for the field `board.sh` already reads.**
`next-cost` had a reader and no writer, so it was hand-typed every session and
@ -70,6 +70,18 @@ marketplace plugin. Three components, one boundary:
*grammar* and points here for the *values*. Scoring the traits is judgement
and belongs to the skill; turning scores into a row is a lookup and takes zero
model calls.
**`--last-effort` is MEASURED from `CLAUDE_EFFORT`, and the calculator must
never default it.** Claude Code exports that variable into every tool-use
context as the session's current effort, so the caller reads it and passes it
in; having `route.sh` read it directly would make the output depend on the
environment instead of on its arguments, and the round trip in selftest
section 6 rests on that determinism. The two sources it replaces fail
identically: the previous board line holds what was *prescribed*, and asking
the operator launders that same prescription through someone reading their own
startup command. Corollary pinned by section 13: `skills/route/SKILL.md` must
never declare an `effort:` frontmatter field, because frontmatter overrides the
session effort and the reading would then measure the skill, not the session.
- **Skills (`skills/coord-send/`, `skills/board/`, `skills/route/`):** natural-language front
doors mapping user intent to engine invocations. No mailbox logic lives here
either. `board` additionally owns the *ranking* — which repo wins and why —
@ -119,7 +131,7 @@ obligations in another repo.
- TDD: no behavior change without a failing selftest check first.
`bash scripts/coord-selftest.sh` must exit 0 (156/156),
`bash scripts/board-selftest.sh` must exit 0 (36/36) and
`bash scripts/route-selftest.sh` must exit 0 (50/50).
`bash scripts/route-selftest.sh` must exit 0 (56/56).
- English for all code, docs, and commit messages (public repo). Norwegian
trigger aliases in the skill description are deliberate.
- Conventional Commits: `type(scope): description`.

View file

@ -134,7 +134,7 @@ Note that raising the inbox's priority (Rule 7) deliberately does **not** widen
bash scripts/coord-selftest.sh # 156 checks against a throwaway mailbox
bash scripts/board-selftest.sh # 36 checks against a throwaway repo tree
bash scripts/route-selftest.sh # 50 checks, incl. the route->board round trip
bash scripts/route-selftest.sh # 56 checks, incl. the route->board round trip
npm test # all three selftests via node --test
TDD is the house rule: every behavior change lands with a failing selftest check first.

View file

@ -371,6 +371,44 @@ check "--help shows no versionless model example" "$rc"
if printf '%s' "$HELP" | grep -q 'blocked-on='; then rc=1; else rc=0; fi
check "--help does not restate the board line grammar" "$rc"
# --- 13. Where --last-effort comes from -----------------------------------
# The record exists to make the policy falsifiable, which it only is if the
# effort field is MEASURED. Two earlier sources both measured the wrong thing:
# the previous board line holds what was PRESCRIBED, and asking the operator
# launders that same prescription through a human who is reading it off the
# startup command they typed. $CLAUDE_EFFORT is what the session actually
# resolved - it is documented as the CURRENT effort level and is exported into
# every tool-use context, which is why a Bash call can read it at all.
#
# The trap this section exists to pin: skill frontmatter can set `effort`, and
# frontmatter overrides the session level while that skill is active. An
# `effort:` field in route's own SKILL.md would therefore make the reading
# report the SKILL's effort and not the session's - a measurement quietly
# measuring itself, with nothing in the output to show it happened.
SKILL="$DIR/../skills/route/SKILL.md"
[ -f "$SKILL" ]; check "route SKILL.md is where the selftest expects it" $?
# Frontmatter only: the body must be free to DISCUSS effort at length.
FM="$(awk 'NR==1 && /^---$/ {f=1; next} f && /^---$/ {exit} f {print}' "$SKILL" 2>/dev/null)"
if printf '%s' "$FM" | grep -q '^effort:'; then rc=1; else rc=0; fi
check "route SKILL.md declares no effort: frontmatter field" "$rc"
grep -q 'CLAUDE_EFFORT' "$SKILL"
check "route SKILL.md names CLAUDE_EFFORT as the --last-effort source" $?
grep -q 'board line' "$SKILL"
check "route SKILL.md still warns off the previous board line" $?
# route.sh carried the claim as a documented premise. It was true when written
# and is not any more, so it must not survive as a comment that reads like a
# measured fact three weeks from now.
FLAT="$(tr '\n' ' ' < "$R" | sed 's/#//g' | tr -s ' ')"
if printf '%s' "$FLAT" | grep -q 'is not observable from inside'; then rc=1; else rc=0; fi
check "route.sh no longer claims effort is unobservable from inside" "$rc"
grep -q 'CLAUDE_EFFORT' "$R"
check "route.sh names the source the caller should measure from" $?
echo ""
echo "route-selftest: $PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ] || exit 1

View file

@ -91,11 +91,16 @@
# zero on row 4 means escalation fires too readily.
#
# All four fields are closed sets or numbers, and required together, because the
# NEXT session reads the record back to decide --opus-xhigh-failed. Note what
# the caller must supply rather than infer: the effort a session ran with is not
# observable from inside that session, and reading it back off the previous
# board line would measure what was PRESCRIBED, not what was RUN. Omit the
# record rather than guess - a guessed value reads back as a measurement.
# NEXT session reads the record back to decide --opus-xhigh-failed. Every one of
# them must be MEASURED by the caller: --last-effort comes from CLAUDE_EFFORT,
# which Claude Code exports into every tool-use context as the session's current
# effort level. It is deliberately NOT defaulted from that variable here - a
# calculator that reads its own environment stops being deterministic from its
# arguments, and section 6 of the selftest depends on that determinism. Reading
# the value off the previous board line would measure what was PRESCRIBED rather
# than what was RUN; so, less obviously, does asking the operator, who reads it
# off the startup command they typed. Omit the record rather than guess - a
# guessed value reads back as a measurement.
#
# It is deliberately NOT wired to the Fable rows. Inferring "the model failed"
# from "the session did not finish" would fire on context exhaustion and on

View file

@ -109,15 +109,39 @@ in the session that is ending — never from what STATE.md prescribed:
- `--last-corrections <n>` — how many rounds of rework it took. This is the
cheap proxy for whether the routing was right.
**Effort is not observable from inside a running session.** The model is
readable from the environment; the effort level the operator launched with is
not. So do not infer it, and in particular do not read it back from the previous
board line — that measures what was *prescribed*, not what was *run*, and the
two come apart exactly when the record would be most interesting. Ask the
operator for the effort level when closing the session. If they are not there to
ask, **omit the record entirely** — it is all four fields or none. A guessed
effort is worse than a missing one, because it reads back later as a
measurement.
**Measure the effort, never infer it.** Read what this session actually
resolved:
```bash
echo "$CLAUDE_EFFORT"
```
`CLAUDE_EFFORT` is Claude Code's own *current* effort level, exported into every
tool-use context — which is why a Bash call can read it. Pass it verbatim as
`--last-effort`. The model is not in the environment (there is no
`CLAUDE_MODEL`); take it from what this session knows itself to be running as.
Two sources are **wrong on purpose**, and both fail the same way. The previous
board line holds what was *prescribed*, not what was *run* — the two come apart
exactly when the record would be most interesting. Asking the operator launders
that same prescription through a human, who is reading it off the startup
command they typed rather than off the running process. Confirming a measured
value with them is fine; sourcing it from them is not.
So the record no longer waits on anyone: all four fields are knowable from
inside the session that is ending. Still omit it entirely — all four or none —
if any one of them is genuinely unknown. A guessed value is worse than a missing
one, because it reads back later as a measurement.
**This file must never declare an `effort:` frontmatter field.** Skill
frontmatter overrides the session effort while the skill is active, so the
reading above would report *this skill's* effort instead of the session's — a
measurement measuring itself, with nothing in the output to show it happened.
Pinned by `route-selftest.sh` section 13.
One honest limit: `CLAUDE_EFFORT` is the *current* level, so if the operator
changed it mid-session with `/effort`, "the effort this session ran with" is not
a single value. Record the level the work was actually done at and say so.
Read the previous `route-last` line out of STATE.md before overwriting it.
Pass `--opus-xhigh-failed` **only** when it says an `Opus 5`/`xhigh` session ran