fix(tests): blade 2 was testing the wrong defect, and said so out loud

An independent reviewer found it and the claim was verified by measurement
before being accepted, not taken on trust.

test_portfolio_scripted_pass_makes_no_real_client raised from the patched
_default_factory to prove the scripted portfolio pass never builds a production
client. It cannot: the factory is called inside run_project's coroutine, and
run_portfolio gathers with return_exceptions=True, so the AssertionError was
collected into a RunFailure and never escaped. Measured directly -- with BOTH
the client_factory wiring and the rc rule detached, the blade stayed GREEN. It
was going red on rc alone, which means it was testing defect B while claiming
to test defect A. The original seven-mutation sweep did not catch this because
each mutation was applied singly, and dropping the wiring alone still flips rc.

Replaced with a call sentinel: a list appended inside the factory and asserted
in the test body, which the wave handler cannot swallow. Re-measured -- red on
the wiring detach alone, and red on both detaches together.

The docstring now also states what the sweep could not: blades 1 and 8 are
environment-conditional. The local profile points at loopback, so on a machine
running a local model server the wiring detach would make real calls and could
complete the pass. Their red was real on the machine it was measured on and is
not portable; blade 2's is.

Separately, the budget-stop print is marked as defensive and currently
unreachable from main(), because main() never constructs a PortfolioMeter and
every write to budget_stop is gated on one -- the strict=True precedent
directly above says untested future-proofing must be labelled as such. The
README claim that a portfolio pass reports a cap stop is corrected to say the
cap has no CLI flag yet. Noted for whoever wires that door: BudgetRefused is a
RuntimeError and the existing except clause would not catch it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0118noV9rCfrdREH26XqZB5z
This commit is contained in:
Kjell Tore Guttormsen 2026-08-05 11:06:41 +02:00
commit f87d555840
3 changed files with 36 additions and 9 deletions

View file

@ -1352,6 +1352,14 @@ def main(argv: list[str] | None = None) -> int:
# projects, four APIConnectionError, silent success). ``BudgetStop`` is kept apart from
# ``stop_reason`` precisely so a caller can tell exhaustion from success; showing neither
# collapsed the distinction the dataclass was split to preserve.
# The budget-stop arm is DEFENSIVE and currently UNREACHABLE from here — measured, not
# assumed, and said out loud for the same reason ``strict=True`` below is: ``main()`` never
# constructs a ``PortfolioMeter``, and every write to ``budget_stop`` is gated on one, so
# only a LIBRARY caller passing ``portfolio_meter=`` can produce this field today. It is
# printed anyway because the field exists and a CLI door onto the global cap is a natural
# next step; the test that covers it drives a crafted ``PortfolioResult``, and says so.
# TRAP for whoever wires that door: ``BudgetRefused`` is a ``RuntimeError``, so the
# ``except`` above would NOT catch the startup refusal — it needs adding explicitly.
if portfolio_result.budget_stop is not None:
bs = portfolio_result.budget_stop
print(