test(portfolio): gate the wave handler's catch width and the failed-project ledger (v/t/s)

Three items on one seam — what a FAILED project does to the wave loop — plus the
snapshot copy they sit next to.

(v) The catch is BaseException, not Exception, and that width was ungated. The
existing collect-and-continue test raises RuntimeError, so it stays green when
the handler is narrowed: measured, the whole of test_portfolio_concurrent_
loadbearing.py (13 tests) passes under the narrowing. asyncio.CancelledError is
the one realistic vector that separates the two — probed first, gather(
return_exceptions=True) COLLECTS it, while KeyboardInterrupt propagates
regardless and could never be helped by a wider catch. Narrowed, a cancelled
member is cast into runs as a fake RunResult and the pass dies in _aggregate,
pointing away from its cause. RED measured.

(t) sum_token_usage excludes a failed project's spend, and that is the honest
answer, not a bug: a run that died before producing a stamp has no provenance,
and inventing one is the fabrication RunFailure exists to avoid. What needed
gating is that those tokens still reach the ledger the global cap is enforced
against — otherwise a repeatedly-failing project burns budget while the meter
reads clean. Pins meter.spent as the pass's real cost, sum_token_usage as the
completed-run subtotal, and their difference as exactly the failed spend. RED
measured against the likely "fix" (sourcing sum_token_usage from the meter),
which is wrong because a seeded meter also carries EARLIER passes' spend; 21
existing budget/portfolio tests stay green under it.

(s) _wave_snapshot uses dataclasses.replace, so a field added later is carried
without touching the function. Not cosmetic: measured, dropping retriever by
hand-enumerating left all 585 tests green — the Step-2 coverage its docstring
credited no longer existed, so the S3.1 retriever seam could be downgraded
mid-pass in silence. Now gated by a property test derived from
dataclasses.fields (not a field count, the shape rejected earlier). The explicit
verdicts copy is retained and separately gated: replace(store) alone shares the
caller's list and takes the byte-identical determinism test RED.

strict=True on the zip is documented as deliberately untested — measured green
when dropped, since gather is built from exactly snapshots, so a test could only
go red by manufacturing a mismatch and would exercise zip rather than this pass.

The new double is registered in the S2.5 consolidation guard's delegating-
overrides list rather than the guard being weakened; it already delegates via
super()._inner_get_response, which test_delegating_overrides_call_super now
enforces on it.

583 -> 586 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbgTCEZma764i1rHTrzceU
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 16:09:12 +02:00
commit 873f5fa272
4 changed files with 322 additions and 8 deletions

View file

@ -622,13 +622,23 @@ def _wave_snapshot(store: VerdictStore) -> VerdictStore:
dropped it would silently downgrade a caller-owned store's semantic retrieval to the
structural default mid-pass.
**Honesty boundary: this copies exactly two fields because ``VerdictStore`` HAS exactly two.**
A third field added later would be silently dropped here the same defect class as the
``retriever`` omission this function was first written with, which the Step-2 contract test
caught. It is left as a documented hazard rather than a guard: an assertion on the field count
would go red on every benign addition to ``VerdictStore``, which trains people to edit the
guard rather than think about the snapshot a worse outcome than the line you are reading."""
return VerdictStore(verdicts=list(store.verdicts), retriever=store.retriever)
**Field-complete by construction.** ``dataclasses.replace`` carries over every field
``VerdictStore`` declares and overrides only ``verdicts``, so a field added later is copied
without this function being touched. The hand-enumerated version this replaced could silently
drop one the exact defect it was first written with (the ``retriever`` omission), which its
own docstring then recorded as a standing hazard. Deriving the copy from the dataclass removes
the hazard instead of documenting it, and does so without the field-count assertion that idea
was rejected for: there is nothing left to keep in sync.
That docstring credited a Step-2 contract test with catching the omission. MEASURED while this
change was made: no such coverage remained reinstating the hand-enumerated form left the
whole suite green, so the S3.1 retriever seam could be downgraded mid-pass in silence. The gate
is now ``test_wave_snapshot_carries_every_field_except_the_copied_verdicts``.
``verdicts`` is still listed explicitly, and must be: ``replace`` copies field REFERENCES, so
omitting it would hand back a store sharing the caller's list — the very race this snapshot
exists to remove, reintroduced by the call that looks tidiest."""
return replace(store, verdicts=list(store.verdicts))
def _merge_wave(store: VerdictStore, wave: Sequence[tuple[str, VerdictStore]]) -> None:
@ -921,7 +931,20 @@ async def run_portfolio(
# ``gather`` resolves in ARGUMENT order, not completion order, and waves follow
# ``project_ids`` — so ``runs`` stays in caller order however the schedule interleaved, and
# position is a sound key for pairing each result back to the pid that produced it.
#
# ``strict=True`` is FUTURE-PROOFING and is deliberately untested — measured, not assumed:
# dropping it leaves the whole suite green, because ``gather`` is constructed from exactly
# ``snapshots``, so the two lengths cannot diverge today. A test could only go red by
# manufacturing a mismatch, which would exercise ``zip`` rather than this pass. It earns its
# place against a later edit that filters or extends one sequence without the other — then a
# silent truncation would mis-attribute every result after the gap, and this fails instead.
for (pid, _snapshot), outcome in zip(snapshots, wave_results, strict=True):
# ``BaseException``, NOT ``Exception``, and the width is load-bearing: a member raising
# ``asyncio.CancelledError`` (a BaseException since 3.8) is COLLECTED by
# ``return_exceptions=True`` and must land in ``failures``. Narrowed to ``Exception`` it
# would fall to the ``cast`` below and put a live exception object into ``runs``, which
# dies later in ``_aggregate`` — a crash pointing away from its cause. Gated by
# ``tests/test_portfolio_failure_accounting_loadbearing.py``.
if isinstance(outcome, BaseException):
failures.append(
RunFailure(