open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 1 Activity

feat(s33): collect-and-continue error policy via RunFailure slots

Browse source 
One project's exception no longer cancels its siblings: the wave's gather
runs with return_exceptions=True and each raised project becomes a frozen
RunFailure(project_id, error, error_type) in the defaulted
PortfolioResult.failures, while every completed project keeps its full
RunResult. RunFailure is a distinct type rather than an error field on
RunResult (six required non-defaulted fields -> dummies would be fabricated
provenance); the deviation from the spec's wording is stated in both
docstrings. TaskGroup is rejected: it cancels siblings on first exception.

snapshots reaches _merge_wave unfiltered, and results are paired back to pids
by POSITION (gather resolves in argument order) so a mid-wave failure cannot
disturb store order or misattribute the failure.

Detach points measured, not asserted:
  drop return_exceptions=True      -> RED (both new tests)
  reorder snapshots before merge   -> RED (+ Session 1's determinism test)
  pair results by sorted(), not position -> RED (failure misattributed)
  filter failed members before merge -> GREEN, measured

The last one corrects the plan: its carried-forward requirement implied
filtering before the merge was the hazard. Filtering preserves relative order,
which is all _merge_wave consumes, so the variant is undetectable AND harmless.
The docstring now names the reorder as the detach point and records the
filter asymmetry, rather than claiming a detach point that does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015vYbqW4MppACRvPhEMDpvF
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 17:26:09 +02:00
commit dd15e33556
2 changed files with 257 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

76
src/portfolio_optimiser/run.py
View file

@ -95,6 +95,24 @@ class RunResult:
checker_verdict: str = "absent"
@dataclass(frozen=True)
class RunFailure:
"""One project that RAISED during a portfolio pass (S3.3, SC3 collect-and-continue).
A DISTINCT type from ``RunResult`` rather than an error field on it, and deliberately so: the
session spec's wording was "a ``RunResult`` slot with an error field", but ``RunResult`` is
frozen with six required non-defaulted fields (``:89-94``) — a run that raised before producing
an outcome has no honest value for ``outcome``, ``provenance`` or ``verdict``. Filling them with
dummies would put FABRICATED provenance into the aggregate, which is the failure mode this
repo's provenance rules exist to prevent. The exception is recorded as text (``error``) plus its
class name (``error_type``) rather than the live exception object, so a ``PortfolioResult``
stays a plain frozen value with no traceback frames held alive."""
project_id: str
error: str
error_type: str
@dataclass(frozen=True)
class DryRunReport:
"""S4.2 offline ``--live-dry-run`` outcome (comparison protocol §4 pkt 3): everything a real run
@ -132,8 +150,14 @@ class PortfolioResult:
The remaining fields are a thin aggregate over ``runs``: ``validated_count`` /
``rejected_count`` partition the outcomes; ``sum_claimed_saving_nok`` totals the claimed
saving of the validated proposals only; ``sum_token_usage`` totals every run's
provenance token usage. ``stopped_early`` / ``stop_reason`` record a Step-8 goal-stop: they
default so the frozen aggregate and every existing constructor call are unaffected."""
provenance token usage. ``stopped_early`` / ``stop_reason`` record a Step-8 goal-stop, and
``failures`` records the projects that RAISED (S3.3 collect-and-continue): all three default so
the frozen aggregate and every existing constructor call are unaffected.
``runs`` and ``failures`` PARTITION the projects that were actually submitted — a project
appears in exactly one of them, never both, and the counts do not overlap. ``validated_count`` /
``rejected_count`` therefore total ``len(runs)``, not the portfolio size: a failure is neither a
validation nor a rejection, and folding it into either would misreport the pass."""
runs: tuple[RunResult, ...]
store: VerdictStore
@ -143,6 +167,7 @@ class PortfolioResult:
sum_token_usage: int
stopped_early: bool = False
stop_reason: GoalReached | None = None
failures: tuple[RunFailure, ...] = ()
def _authored_texts(result: Any, name: str) -> list[str]:
@ -656,7 +681,16 @@ async def run_portfolio(
``concurrency`` (S3.3, D-D wave model) caps how many projects run at once. It is validated
BEFORE anything loads — an unusable cap must surface as an error, never as a silently-empty
pass. At the default ``1`` the pass is the sequential one described above."""
pass. At the default ``1`` the pass is the sequential one described above.
Error policy (S3.3, SC3) is COLLECT-AND-CONTINUE: a project that raises does not cancel its
siblings and does not abort the pass. Its exception is recorded as a ``RunFailure`` in
``failures`` while every project that completed keeps its full ``RunResult``, so a portfolio of
independent projects degrades one project at a time rather than all at once. This holds at
every ``k``, including ``k=1``. The policy is carried by ``return_exceptions=True`` on the
wave's ``gather``; ``asyncio.TaskGroup`` would cancel the siblings and is rejected for that
reason. Note the pass still raises for errors that are NOT one project's failure — an unknown
``project_id`` and a non-positive ``concurrency`` are caller mistakes and fail fast."""
if concurrency < 1:
raise ValueError(
f"concurrency must be >= 1, got {concurrency}: a non-positive wave size would run no "
@ -673,6 +707,7 @@ async def run_portfolio(
portfolio_baseline_ore = _to_ore(sum(projects[p].total_cost for p in ids if p in projects))
runs: list[RunResult] = []
failures: list[RunFailure] = []
stopped_early = False
stop_reason: GoalReached | None = None
for wave_ids in _waves(ids, concurrency):
@ -713,6 +748,9 @@ async def run_portfolio(
# run_portfolio only drives full runs (never dry-run), so the return narrows to RunResult;
# the cast keeps the widened run_project signature honest without an @overload duplication.
# ``return_exceptions=True`` is the collect-and-continue seam (SC3): one project's
# exception must not cancel its siblings. ``asyncio.TaskGroup`` is deliberately NOT used —
# it cancels the remaining tasks on first exception, which is this policy's exact negation.
wave_results = await asyncio.gather(
*(
run_project(
@ -733,19 +771,41 @@ async def run_portfolio(
meter=meter_factory() if meter_factory is not None else None,
)
for pid, snapshot in snapshots
)
),
return_exceptions=True,
)
# ``gather`` resolves in ARGUMENT order, not completion order, and waves follow
# ``project_ids`` — so ``runs`` stays in caller order however the schedule interleaved.
runs.extend(cast(RunResult, r) for r in wave_results)
# ``project_ids`` — so ``runs`` stays in caller order however the schedule interleaved, and
# position is a sound key for pairing each result back to the pid that produced it.
for (pid, _snapshot), outcome in zip(snapshots, wave_results, strict=True):
if isinstance(outcome, BaseException):
failures.append(
RunFailure(
project_id=pid, error=str(outcome), error_type=type(outcome).__name__
)
)
else:
runs.append(cast(RunResult, outcome))
# ``snapshots`` is handed over UNFILTERED. ``_merge_wave`` consumes the ORDER this sequence
# arrives in and nothing else, so that order is the contract. Measured honestly: filtering
# the failed members out here would in fact be harmless, because filtering preserves
# relative order — the variant is green. What is NOT harmless is anything that REORDERS
# (or re-pairs) the sequence, and keeping the list whole is what leaves no place for that
# mistake to be written. A failed project's snapshot holds no new verdicts, so it costs
# nothing to pass through.
_merge_wave(store, snapshots)
if stopped_early:
break
base = _aggregate(tuple(runs), store)
if stopped_early or stop_reason is not None:
return replace(base, stopped_early=stopped_early, stop_reason=stop_reason)
if stopped_early or stop_reason is not None or failures:
return replace(
base,
stopped_early=stopped_early,
stop_reason=stop_reason,
failures=tuple(failures),
)
return base