feat(1b): et tak som fyrer er ikke en krasj — BudgetExceeded får sin egen kanal [skip-docs]
Prosjektets første levende kjøring døde på `rounds limit=12 observed=13`, og den
hostede flaten svarte `500 {error_type, error}` — nøyaktig det samme den sier når
modell-endepunktet faller. Nå: 429 med trippelen som STRUKTUR.
Beslutningen er S3.4-invarianten anvendt på transporten: `budget_stop` ble holdt
utenfor `stop_reason` fordi de to stoppene betyr motsatte ting, og å svare
ressurs-utmattelse på krasj-kanalen gjør «det gikk ikke» uleselig på samme måte.
IKKE 200, og det er dét som skiller den fra `Rejection`: en `Rejection` er en
kjøring som KONKLUDERTE og hører i payloaden, mens et uttømt budsjett produserte
ingen proposal — en 2xx ville latt en automatisk kaller bokføre «analysert» for en
kjøring som analyserte ingenting. 429 fordi betingelsen oppstår av en TILDELING
(`max_rounds`/`max_tokens` er whitelistede request-felt), aldri av en serverfeil.
`kind`/`limit`/`observed` legges ut som felt, aldri `str(exc)` (kø-(y));
`error_type` holdes ute — den nøkkelen tilhører feilkanalen. `budget_exhausted` er
ikke foldet inn i `outcome_type` og kunne ikke vært det: `outbox.outcome_payload`
er den ene kopien av den forgreningen og tar `ValidatedProposal | Rejection`.
Ærlighets-grense: ingen `Retry-After` — å vente endrer ingenting.
Iron Law: begge nye tester RØDE før armen fantes. Fem mutasjoner mot HELE suiten,
alle røde med hver sin signatur, grønn kontroll 867/4: detach armen (2 røde) ·
flat streng i stedet for struktur (1 rød — struktur-testen alene) · ekko `limit`
som `observed` (1 rød) · utvid armen til `Exception` (6 røde) · stemple
`error_type` på budsjett-kroppen (1 rød).
500-armens vitne ble BYTTET, ikke slettet: den eksisterende testen brukte
`BudgetExceeded` som sin 500-prøve, så en ny arm alene ville etterlatt
krasj-kanalen uten vitne. Den bærer nå en ekte ikke-budsjett-`RuntimeError`, og er
dét som holder den nye armen smal.
Kjørt, ikke bare testet: `python main.py` startet, `/readiness` 200, ukjent felt →
400 med navnet, 404, SIGTERM → exit 0.
865 → 867 passed / 4 skipped; ruff + format + mypy rene. DEPLOY.md §6 dokumenterer
429 for mottakeren.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1bsX79aDS7fJ5udWGWAEN
This commit is contained in:
parent
605c02aa62
commit
986fc19350
4 changed files with 143 additions and 9 deletions
|
|
@ -317,8 +317,11 @@ async def test_contract_violation_is_400_and_run_failure_is_500(
|
|||
) -> None:
|
||||
"""ValueError (pydantic contract violations subclass it) is the CALLER's error → 400; any
|
||||
other failure is an honest 500 carrying {error_type, error} (mirrors RunFailure's shape).
|
||||
BudgetExceeded is RuntimeError, so it lands in the 500 arm — with observed != limit so the
|
||||
two can never be conflated by an echo (kø-(y))."""
|
||||
|
||||
The 500 witness is a NON-budget RuntimeError on purpose. It used to be ``BudgetExceeded``,
|
||||
which is what made this test the one that pinned exhaustion to the crash channel; the two
|
||||
now have separate arms, and this half is what keeps the budget arm NARROW — RED if it is
|
||||
widened to catch ``Exception`` and route every failure to 429."""
|
||||
monkeypatch.setattr(
|
||||
hosting, "run_project", _Recorder(error=ValueError("docs_dir does not exist"))
|
||||
)
|
||||
|
|
@ -326,12 +329,61 @@ async def test_contract_violation_is_400_and_run_failure_is_500(
|
|||
assert status == 400
|
||||
assert "docs_dir does not exist" in body["error"]
|
||||
|
||||
monkeypatch.setattr(hosting, "run_project", _Recorder(error=BudgetExceeded("tokens", 100, 173)))
|
||||
monkeypatch.setattr(
|
||||
hosting, "run_project", _Recorder(error=RuntimeError("chat client fell over"))
|
||||
)
|
||||
status, body = await _post(served, "/invocations", _PAYLOAD)
|
||||
assert status == 500
|
||||
assert body["error_type"] == "BudgetExceeded"
|
||||
assert "limit=100" in body["error"]
|
||||
assert "173" in body["error"]
|
||||
assert body["error_type"] == "RuntimeError"
|
||||
assert "chat client fell over" in body["error"]
|
||||
assert "budget_exhausted" not in body
|
||||
|
||||
|
||||
async def test_budget_exhaustion_is_not_the_failure_channel(
|
||||
served: str, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""Budget exhaustion is a DESIGNED terminal state — the cap firing IS the feature working
|
||||
(``Budget``: fail-fast, never an unbounded loop) — so it must not share a channel with a
|
||||
crash. The first live run died exactly here (``rounds limit=12 observed=13``) and the hosted
|
||||
surface answered 500, i.e. the same thing it says when the model endpoint falls over.
|
||||
|
||||
* NOT 500: nothing broke.
|
||||
* NOT 200: unlike a ``Rejection`` — which is a run that CONCLUDED, and therefore belongs in
|
||||
the payload — an exhausted budget produced no proposal at all. A 2xx would let an automated
|
||||
caller record "analysed" for a run that analysed nothing.
|
||||
* 429: the condition arises from an ALLOWANCE (``max_rounds``/``max_tokens`` are whitelisted
|
||||
request fields, and the raise is the caller's own remedy), never from a server fault.
|
||||
* ``error_type`` is ABSENT: that key belongs to the failure channel, and a caller switching
|
||||
on its presence must not find it on a run that did not fail.
|
||||
|
||||
RED when the arm is detached (falls through to 500) or relabelled to any other status."""
|
||||
monkeypatch.setattr(hosting, "run_project", _Recorder(error=BudgetExceeded("rounds", 12, 13)))
|
||||
|
||||
status, body = await _post(served, "/invocations", _PAYLOAD)
|
||||
|
||||
assert status == 429
|
||||
assert "error_type" not in body
|
||||
|
||||
|
||||
async def test_budget_stop_triple_survives_as_structure(
|
||||
served: str, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""kø-(y): ``kind``/``limit``/``observed`` describe ONE ledger and are ONE structured stop
|
||||
event. ``str(exc)`` flattens them into prose the caller has to re-parse to learn WHICH cap
|
||||
bound and how far past it the run got — which is the whole operational question (raise
|
||||
``max_rounds``? raise ``max_tokens``? give up?).
|
||||
|
||||
Built with ``observed != limit`` deliberately: at an exactly-exhausted cap the two coincide,
|
||||
and a test written there cannot tell a faithful implementation from one that echoes the limit
|
||||
back as the observed value. RED when the payload carries only the message string."""
|
||||
monkeypatch.setattr(hosting, "run_project", _Recorder(error=BudgetExceeded("rounds", 12, 13)))
|
||||
|
||||
status, body = await _post(served, "/invocations", _PAYLOAD)
|
||||
|
||||
assert status == 429
|
||||
assert body["budget_exhausted"] == {"kind": "rounds", "limit": 12, "observed": 13}
|
||||
# The human-readable line stays alongside the structure — an operator reading a log needs it.
|
||||
assert body["error"] == "budget exceeded: rounds limit=12 observed=13"
|
||||
|
||||
|
||||
async def test_readiness_answers_while_an_invocation_is_in_flight(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue