test(portfolio): gate the wave handler's catch width and the failed-project ledger (v/t/s)
Three items on one seam — what a FAILED project does to the wave loop — plus the snapshot copy they sit next to. (v) The catch is BaseException, not Exception, and that width was ungated. The existing collect-and-continue test raises RuntimeError, so it stays green when the handler is narrowed: measured, the whole of test_portfolio_concurrent_ loadbearing.py (13 tests) passes under the narrowing. asyncio.CancelledError is the one realistic vector that separates the two — probed first, gather( return_exceptions=True) COLLECTS it, while KeyboardInterrupt propagates regardless and could never be helped by a wider catch. Narrowed, a cancelled member is cast into runs as a fake RunResult and the pass dies in _aggregate, pointing away from its cause. RED measured. (t) sum_token_usage excludes a failed project's spend, and that is the honest answer, not a bug: a run that died before producing a stamp has no provenance, and inventing one is the fabrication RunFailure exists to avoid. What needed gating is that those tokens still reach the ledger the global cap is enforced against — otherwise a repeatedly-failing project burns budget while the meter reads clean. Pins meter.spent as the pass's real cost, sum_token_usage as the completed-run subtotal, and their difference as exactly the failed spend. RED measured against the likely "fix" (sourcing sum_token_usage from the meter), which is wrong because a seeded meter also carries EARLIER passes' spend; 21 existing budget/portfolio tests stay green under it. (s) _wave_snapshot uses dataclasses.replace, so a field added later is carried without touching the function. Not cosmetic: measured, dropping retriever by hand-enumerating left all 585 tests green — the Step-2 coverage its docstring credited no longer existed, so the S3.1 retriever seam could be downgraded mid-pass in silence. Now gated by a property test derived from dataclasses.fields (not a field count, the shape rejected earlier). The explicit verdicts copy is retained and separately gated: replace(store) alone shares the caller's list and takes the byte-identical determinism test RED. strict=True on the zip is documented as deliberately untested — measured green when dropped, since gather is built from exactly snapshots, so a test could only go red by manufacturing a mismatch and would exercise zip rather than this pass. The new double is registered in the S2.5 consolidation guard's delegating- overrides list rather than the guard being weakened; it already delegates via super()._inner_get_response, which test_delegating_overrides_call_super now enforces on it. 583 -> 586 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbgTCEZma764i1rHTrzceU
This commit is contained in:
parent
8910a673ea
commit
873f5fa272
4 changed files with 322 additions and 8 deletions
|
|
@ -27,6 +27,7 @@ from __future__ import annotations
|
|||
|
||||
import ast
|
||||
import asyncio
|
||||
import dataclasses
|
||||
from collections.abc import Mapping, Sequence
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
|
@ -43,10 +44,17 @@ from test_portfolio_learning_loadbearing import (
|
|||
)
|
||||
|
||||
from portfolio_optimiser import ledger as ledger_mod
|
||||
from portfolio_optimiser import semretrieval
|
||||
from portfolio_optimiser import verdicts as verdicts_mod
|
||||
from portfolio_optimiser.contracts import GoalConfig, GoalContract
|
||||
from portfolio_optimiser.ledger import SavingsLedger
|
||||
from portfolio_optimiser.run import PortfolioResult, RunFailure, _waves, run_portfolio
|
||||
from portfolio_optimiser.run import (
|
||||
PortfolioResult,
|
||||
RunFailure,
|
||||
_wave_snapshot,
|
||||
_waves,
|
||||
run_portfolio,
|
||||
)
|
||||
from portfolio_optimiser.verdicts import VerdictStore
|
||||
|
||||
# The shipped 3-project fixture, in submission order (mirrors tests/test_portfolio.py:57).
|
||||
|
|
@ -223,6 +231,43 @@ def _aggregate_fields(result: PortfolioResult) -> tuple[Any, ...]:
|
|||
)
|
||||
|
||||
|
||||
def test_wave_snapshot_carries_every_field_except_the_copied_verdicts() -> None:
|
||||
"""The snapshot copies ``verdicts`` and carries EVERY other ``VerdictStore`` field over.
|
||||
|
||||
**Detach point: hand-enumerate the fields (``VerdictStore(verdicts=list(store.verdicts))``)
|
||||
-> this test goes RED.** MEASURED, and the measurement is why this test exists: before it was
|
||||
written, dropping ``retriever`` in exactly that way left the whole suite GREEN (585 passed). No
|
||||
test set ``retriever`` at all, so the S3.1 opt-in seam could be silently downgraded to the
|
||||
structural default mid-pass — a caller-owned semantic retriever would stop being used from the
|
||||
first wave onward, and nothing would say so. ``_wave_snapshot``'s docstring had claimed this
|
||||
was caught by a Step-2 contract test; that coverage no longer existed.
|
||||
|
||||
This asserts the PROPERTY (every declared field survives), not a field count — the shape that
|
||||
was rejected for going red on every benign addition to ``VerdictStore``. Deriving the expected
|
||||
set from ``dataclasses.fields`` means a field added later is covered without editing this test,
|
||||
and ``replace`` in the implementation means it is carried without editing the snapshot either.
|
||||
"""
|
||||
# A real ``Retriever``, not a bare sentinel: the field is typed, and the assertion below is
|
||||
# about identity surviving the copy, which a genuine instance demonstrates without pretence.
|
||||
marker = semretrieval.StructuralRetriever(verdicts_mod.similarity)
|
||||
store = VerdictStore(verdicts=[], retriever=marker)
|
||||
|
||||
snapshot = _wave_snapshot(store)
|
||||
|
||||
carried = [f.name for f in dataclasses.fields(VerdictStore) if f.name != "verdicts"]
|
||||
assert carried, "VerdictStore declares no field beyond ``verdicts`` — this test guards nothing"
|
||||
for name in carried:
|
||||
assert getattr(snapshot, name) is getattr(store, name), (
|
||||
f"``{name}`` was not carried into the wave snapshot — a project in the wave would run "
|
||||
f"against a store whose {name} silently differs from the one the caller configured"
|
||||
)
|
||||
|
||||
# The half that must NOT be carried by reference: the verdict list is copied, so a project
|
||||
# appending to its snapshot cannot touch the shared store. Guarded here too, because the
|
||||
# tidiest wrong fix for the above (``replace(store)``) breaks exactly this.
|
||||
assert snapshot.verdicts is not store.verdicts
|
||||
|
||||
|
||||
async def test_concurrent_pass_is_byte_identical_to_sequential() -> None:
|
||||
"""S3.3's core contract: ``concurrency=3`` produces the SAME store-verdict id SEQUENCE as
|
||||
``concurrency=1``, under a completion order that genuinely differs from submission order.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue