fix(route): close the record's drift surface and the confounded smoke test
Six follow-ups on 0d11838, three of which mattered.
The model-alias gate passed for the wrong reason: after the quoted-alias grep
it fell back to an unanchored grep for the bare word, which matches "opus"
anywhere in `claude --help` and would have reported success even if --model
stopped accepting the alias. A gate that cannot fail is worse than no gate.
Only the quoted form is matched now, and the failing aliases are named.
--last-model and --last-effort were unvalidated free text while the other two
record fields were gated. The next session READS the record back to decide
--opus-xhigh-failed, so a drifted spelling there rebuilds the exact
reader-versus-writer drift this script exists to remove, one field over. Both
are closed sets now: the row table's three model names and the verified effort
levels. That also makes the record's sanitizing dead code, so it is gone.
The skill told future sessions to write the record "every session" while
STATE documented that the effort level is not observable from inside a running
session. A session following both would have fabricated the value, and a
fabricated effort reads back later as a measurement. The skill now says: ask
the operator, and omit the record rather than guess -- explicitly including
that reading it off the previous board line measures what was PRESCRIBED, not
what was RUN.
Also: README said "seven bash scripts" (nine files, six user-facing) and its
skills badge still said 2; selftest counts updated to 50.
Verified, not assumed: the installed plugin cache at 0.9.0 contains only
board and coord-send, so route is not discoverable until a release bumps it --
the smoke test STATE had queued before release would have failed with 127 for
a reason unrelated to the skill. STATE reordered to release-then-test. The
manifest is auto_discover, so no skills array needs an entry.
check-versions.mjs is green (11 OK, 0 ERROR) with the new skill at 0.9.0.
Selftests: coord 136, board 30, route 50, node 7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017peNgsxVt1BR4BTuMwiPoX
This commit is contained in:
parent
0d1183800e
commit
a435a031db
5 changed files with 70 additions and 16 deletions
|
|
@ -45,7 +45,7 @@ marketplace plugin. Three components, one boundary:
|
|||
rationale, and prints one block of `key=value` lines: the rubric row, the rule
|
||||
that fired, the `next-cost` value, a pasteable startup command, the one-row
|
||||
cheaper fallback, and the STATE.md comment lines. Pinned by
|
||||
`route-selftest.sh` (47 checks).
|
||||
`route-selftest.sh` (50 checks).
|
||||
|
||||
**It is here because it is the WRITER for the field `board.sh` already reads.**
|
||||
`next-cost` had a reader and no writer, so it was hand-typed every session and
|
||||
|
|
@ -104,7 +104,7 @@ halves together for exactly that reason.
|
|||
- TDD: no behavior change without a failing selftest check first.
|
||||
`bash scripts/coord-selftest.sh` must exit 0 (136/136),
|
||||
`bash scripts/board-selftest.sh` must exit 0 (30/30) and
|
||||
`bash scripts/route-selftest.sh` must exit 0 (47/47).
|
||||
`bash scripts/route-selftest.sh` must exit 0 (50/50).
|
||||
- English for all code, docs, and commit messages (public repo). Norwegian
|
||||
trigger aliases in the skill description are deliberate.
|
||||
- Conventional Commits: `type(scope): description`.
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
|
@ -83,7 +83,7 @@ It lives here because it is the **writer** for the cost field the board already
|
|||
|
||||
Scoring is judgement and belongs to the skill; turning scores into a row is a lookup and costs no model calls. One deliberate side effect is worth more than the tokens saved: a next step that cannot be scored `known` or `partial`, with no design phase planned, is an **underspecified task description** — the answer is to rewrite the step, not to upgrade the model.
|
||||
|
||||
**CLI.** The engine is seven bash scripts in the plugin's `scripts/` directory; resolve them as `"${CLAUDE_PLUGIN_ROOT:-$HOME/.claude}/scripts/coord-<name>.sh"` (from a terminal, use the plugin's install path):
|
||||
**CLI.** The engine is six user-facing bash scripts in the plugin's `scripts/` directory (plus three selftests); resolve them as `"${CLAUDE_PLUGIN_ROOT:-$HOME/.claude}/scripts/coord-<name>.sh"` (from a terminal, use the plugin's install path):
|
||||
|
||||
coord-send.sh --to <repo> --subject "<subject>" [--message "<text>"] # or body on stdin
|
||||
coord-send.sh --broadcast --subject "<subject>" <<'BODY' ... BODY
|
||||
|
|
@ -133,7 +133,7 @@ Note that raising the inbox's priority (Rule 7) deliberately does **not** widen
|
|||
|
||||
bash scripts/coord-selftest.sh # 136 checks against a throwaway mailbox
|
||||
bash scripts/board-selftest.sh # 30 checks against a throwaway repo tree
|
||||
bash scripts/route-selftest.sh # 47 checks, incl. the route->board round trip
|
||||
bash scripts/route-selftest.sh # 50 checks, incl. the route->board round trip
|
||||
npm test # all three selftests via node --test
|
||||
|
||||
TDD is the house rule: every behavior change lands with a failing selftest check first.
|
||||
|
|
|
|||
|
|
@ -251,6 +251,24 @@ check "no route-last line when the record is omitted" "$rc"
|
|||
--rationale x --last-corrections three >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "non-numeric --last-corrections exits 2" $?
|
||||
|
||||
# The record is read back by the NEXT session to decide --opus-xhigh-failed, so
|
||||
# its model and effort are compared, not just displayed. Leaving them as free
|
||||
# text would rebuild the exact reader-versus-writer drift this script exists to
|
||||
# kill, one field over. Both are closed sets: the row table's three model names
|
||||
# and the verified effort levels.
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-model "opus 5" --last-effort xhigh \
|
||||
--last-completed no --last-corrections 1 >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "--last-model rejects a non-rubric spelling" $?
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-model "Opus 5" --last-effort extreme \
|
||||
--last-completed no --last-corrections 1 >/dev/null 2>&1
|
||||
[ $? -eq 2 ]; check "--last-effort rejects a value outside the verified set" $?
|
||||
"$R" --path known --verification strong --reversibility cheap --scope local \
|
||||
--rationale x --last-model "Fable 5" --last-effort medium \
|
||||
--last-completed yes --last-corrections 0 >/dev/null 2>&1
|
||||
[ $? -eq 0 ]; check "--last-model/-effort accept every legal value" $?
|
||||
|
||||
# The record is telemetry and must NOT silently fire the Fable rows. Only the
|
||||
# explicit flag does, because "an opus/xhigh session did not finish" also
|
||||
# covers context exhaustion, an operator interrupt and a block on another repo
|
||||
|
|
@ -325,13 +343,16 @@ case "|low|medium|high|xhigh|max|" in *"|$efforts|"*) rc=0 ;; *) rc=1 ;; esac
|
|||
check "emitted effort is in the verified effort set" "$rc"
|
||||
|
||||
if command -v claude >/dev/null 2>&1; then
|
||||
# Match ONLY the quoted alias as --model documents it. An unanchored grep for
|
||||
# the bare word would hit "opus" anywhere in the help text and pass even if
|
||||
# --model stopped accepting the alias entirely - a gate that reports success
|
||||
# without testing anything, which is worse than no gate.
|
||||
HELPTXT="$(claude --help 2>&1)"
|
||||
miss=0
|
||||
miss=""
|
||||
for alias in fable opus sonnet; do
|
||||
printf '%s' "$HELPTXT" | grep -q "'$alias'" || \
|
||||
printf '%s' "$HELPTXT" | grep -q "$alias" || miss=$((miss+1))
|
||||
printf '%s' "$HELPTXT" | grep -q "'$alias'" || miss="$miss $alias"
|
||||
done
|
||||
[ "$miss" -eq 0 ]; check "installed claude still accepts fable/opus/sonnet aliases" $?
|
||||
[ -z "$miss" ]; check "installed claude documents the aliases route emits:${miss:- all three}" $?
|
||||
else
|
||||
echo " skip - claude not on PATH, model alias gate not run"
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -73,7 +73,8 @@
|
|||
# route.sh --path <v> --verification <v> --reversibility <v> --scope <v>
|
||||
# --rationale <text> [--opus-xhigh-failed]
|
||||
#
|
||||
# route.sh ... --last-model <name> --last-effort <level>
|
||||
# route.sh ... --last-model <Sonnet 5|Opus 5|Fable 5>
|
||||
# --last-effort <low|medium|high|xhigh|max>
|
||||
# --last-completed <yes|no> --last-corrections <n>
|
||||
#
|
||||
# --opus-xhigh-failed is the ONLY way rows 5 and 6 can fire, and it means one
|
||||
|
|
@ -89,6 +90,13 @@
|
|||
# high counts on row 1 mean the cheap row is too easy to reach, systematically
|
||||
# zero on row 4 means escalation fires too readily.
|
||||
#
|
||||
# All four fields are closed sets or numbers, and required together, because the
|
||||
# NEXT session reads the record back to decide --opus-xhigh-failed. Note what
|
||||
# the caller must supply rather than infer: the effort a session ran with is not
|
||||
# observable from inside that session, and reading it back off the previous
|
||||
# board line would measure what was PRESCRIBED, not what was RUN. Omit the
|
||||
# record rather than guess - a guessed value reads back as a measurement.
|
||||
#
|
||||
# It is deliberately NOT wired to the Fable rows. Inferring "the model failed"
|
||||
# from "the session did not finish" would fire on context exhaustion and on
|
||||
# operator interrupts, which say nothing about the model. The record is
|
||||
|
|
@ -158,6 +166,18 @@ if [ "$L_SET" -eq 1 ]; then
|
|||
[ -n "$L_EFFORT" ] || die "--last-effort is required with a last-session record"
|
||||
[ -n "$L_DONE" ] || die "--last-completed is required with a last-session record"
|
||||
[ -n "$L_CORR" ] || die "--last-corrections is required with a last-session record"
|
||||
# Model and effort are closed sets here, not free text. The next session READS
|
||||
# this record back to decide --opus-xhigh-failed, so a drifted spelling
|
||||
# ("opus 5" for "Opus 5") rebuilds the reader-versus-writer drift this whole
|
||||
# script exists to remove, one field over.
|
||||
case "$L_MODEL" in
|
||||
"Sonnet 5"|"Opus 5"|"Fable 5") ;;
|
||||
*) die "--last-model: '$L_MODEL' is not a row-table model (Sonnet 5|Opus 5|Fable 5)" ;;
|
||||
esac
|
||||
case "$L_EFFORT" in
|
||||
low|medium|high|xhigh|max) ;;
|
||||
*) die "--last-effort: '$L_EFFORT' is not a verified effort level (low|medium|high|xhigh|max)" ;;
|
||||
esac
|
||||
case "$L_DONE" in
|
||||
yes|no) ;;
|
||||
*) die "--last-completed: unknown value '$L_DONE' (yes|no)" ;;
|
||||
|
|
@ -231,9 +251,9 @@ echo "command=$(row_cmd "$ROW")"
|
|||
echo "fallback=$(row_name "$FB")"
|
||||
echo "fallback-command=$(row_cmd "$FB")"
|
||||
echo "route-line=<!-- route: path=$PATH_T; verification=$VERIF; reversibility=$REVERS; scope=$SCOPE; rationale=$RAT_CLEAN -->"
|
||||
# No sanitizing needed on the record: all four fields are validated against
|
||||
# closed sets above, so none of them can carry a newline or a '-->'.
|
||||
if [ "$L_SET" -eq 1 ]; then
|
||||
LM="$(printf '%s' "$L_MODEL" | tr '\r\n' ' ' | tr -d '\000-\037' | sed 's/-->/-- >/g')"
|
||||
LE="$(printf '%s' "$L_EFFORT" | tr '\r\n' ' ' | tr -d '\000-\037' | sed 's/-->/-- >/g')"
|
||||
echo "route-last=<!-- route-last: model=$LM; effort=$LE; completed=$L_DONE; corrections=$L_CORR -->"
|
||||
echo "route-last=<!-- route-last: model=$L_MODEL; effort=$L_EFFORT; completed=$L_DONE; corrections=$L_CORR -->"
|
||||
fi
|
||||
exit 0
|
||||
|
|
|
|||
|
|
@ -97,15 +97,28 @@ scoring `partial` to keep things moving.
|
|||
## The last-session record
|
||||
|
||||
Rows 5 and 6 (Fable) are history-dependent, so without a record they are dead
|
||||
policy. Write it every session, from what actually happened in the session that
|
||||
is ending:
|
||||
policy. Write it whenever all four fields are actually known, from what happened
|
||||
in the session that is ending — never from what STATE.md prescribed:
|
||||
|
||||
- `--last-model` / `--last-effort` — what this session actually ran with.
|
||||
- `--last-model` / `--last-effort` — what this session actually ran with. Both
|
||||
are closed sets (`Sonnet 5|Opus 5|Fable 5`, and the verified effort levels),
|
||||
because the next session compares these values rather than just displaying
|
||||
them.
|
||||
- `--last-completed yes|no` — did this session finish the next step the previous
|
||||
STATE.md set out? Answer about that step, not about the session in general.
|
||||
- `--last-corrections <n>` — how many rounds of rework it took. This is the
|
||||
cheap proxy for whether the routing was right.
|
||||
|
||||
**Effort is not observable from inside a running session.** The model is
|
||||
readable from the environment; the effort level the operator launched with is
|
||||
not. So do not infer it, and in particular do not read it back from the previous
|
||||
board line — that measures what was *prescribed*, not what was *run*, and the
|
||||
two come apart exactly when the record would be most interesting. Ask the
|
||||
operator for the effort level when closing the session. If they are not there to
|
||||
ask, **omit the record entirely** — it is all four fields or none. A guessed
|
||||
effort is worse than a missing one, because it reads back later as a
|
||||
measurement.
|
||||
|
||||
Read the previous `route-last` line out of STATE.md before overwriting it.
|
||||
Pass `--opus-xhigh-failed` **only** when it says an `Opus 5`/`xhigh` session ran
|
||||
at this same next step and did not complete it. A session that ran out of
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue