open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 1 Activity

feat(s33): wave-boundary goal-stop semantics + single-loop and one-writer guards

Browse source 
Documents the wave-model reading of the existing Step-8 goal semantics (a
reading, not a redesign — Session 1 already moved the checks to wave assembly
because the executor required it) and pins it with three tests.

Goal checks run at WAVE ASSEMBLY, per member: a HARD per-project goal removes
the pid before the wave starts, so an excluded project is never STARTED and
leaves no verdict behind; a HARD portfolio goal stops the pass with the
assembled wave still crossing its barrier; SOFT flags and continues.

Detach points measured:
  per-project goal keyed per WAVE, not per member -> RED (membership diverges
    at k=3, unaffected at k=1)
  import threading under src/                     -> RED (AST guard)
  promote_verdict called on the run path          -> RED (tripwire)

Corrects one claim the plan and my first docstring both implied: checking the
goal MID-WAVE does NOT make membership completion-order dependent.
_goal_limit_if_reached reads only the ledger, contract and baseline, all
invariant during a pass, so a later check reaches the same decision.
Determinism of membership is the ONE-WRITER rule's (C3, now a tripwire test),
not the check's placement; placement buys the never-started property. The
docstrings say this rather than claiming a detach point that does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015vYbqW4MppACRvPhEMDpvF
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 17:33:17 +02:00
commit 796f8d3af0
2 changed files with 198 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

14
src/portfolio_optimiser/run.py
View file

@ -690,7 +690,19 @@ async def run_portfolio(
every ``k``, including ``k=1``. The policy is carried by ``return_exceptions=True`` on the
wave's ``gather``; ``asyncio.TaskGroup`` would cancel the siblings and is rejected for that
reason. Note the pass still raises for errors that are NOT one project's failure — an unknown
``project_id`` and a non-positive ``concurrency`` are caller mistakes and fail fast."""
``project_id`` and a non-positive ``concurrency`` are caller mistakes and fail fast.
Goal-stop under waves (S3.3 reading of the Step-8 semantics above — a reading, NOT a redesign):
the checks run at WAVE ASSEMBLY, per member, before the wave starts. A HARD per-project goal
removes that pid from its wave, so an excluded project is never STARTED — it costs no model
round-trip and leaves no verdict behind, which a post-hoc filter over completed runs could not
achieve. A HARD portfolio goal stops the pass, and the wave already assembled still crosses its
merge barrier before the loop exits, so a stop never strands verdicts outside the store. A SOFT
goal flags ``stop_reason`` and continues. Membership is identical at every ``concurrency`` — but
the reason is that the ledger is STATIC during a pass (C3: no realization happens on the run
path, which ``test_concurrent_pass_does_not_write_on_the_run_path`` pins), so every check reads
the same accumulated sum regardless of when it runs. Wave-assembly placement buys the
never-started property, not determinism; determinism is the one-writer rule's."""
if concurrency < 1:
raise ValueError(
f"concurrency must be >= 1, got {concurrency}: a non-positive wave size would run no "

185
tests/test_portfolio_concurrent_loadbearing.py
View file

@ -15,14 +15,22 @@ fail-fast; the determinism contract and its probe arrive in Step 2.
from __future__ import annotations
import ast
import asyncio
from collections.abc import Mapping, Sequence
from pathlib import Path
from typing import Any
from unittest import mock
import pytest
from conftest import _PORTFOLIO_DEFAULT_REPLY, _ProjectAwareUsageChatClient
from test_ledger import _prefilled
from test_portfolio import REPLIES
from portfolio_optimiser import ledger as ledger_mod
from portfolio_optimiser import verdicts as verdicts_mod
from portfolio_optimiser.contracts import GoalConfig, GoalContract
from portfolio_optimiser.ledger import SavingsLedger
from portfolio_optimiser.run import PortfolioResult, RunFailure, _waves, run_portfolio
from portfolio_optimiser.verdicts import VerdictStore
@ -500,3 +508,180 @@ async def test_store_order_survives_a_mid_wave_failure() -> None:
f"the two survivors did not mint distinct verdict ids ({store_ids}), so there is no "
"ordering left for this test to pin"
)
# --------------------------------------------------------------------------------------------
# Step 5 — goal-stop at wave boundaries, and the two structural guards (single loop, one writer).
# --------------------------------------------------------------------------------------------
async def _goal_pass(
k: int, *, ledger: SavingsLedger, goals: GoalConfig, recorder: _Recorder
) -> PortfolioResult:
"""A pass at concurrency ``k`` under a goal contract, on a FRESH store."""
def factory(_role: str) -> Any:
return _OrderProbeClient(REPLIES, default_reply=_PORTFOLIO_DEFAULT_REPLY, recorder=recorder)
return await run_portfolio(
_PORTFOLIO_IDS,
"local",
store=VerdictStore(verdicts=[]),
client_factory=factory,
ledger=ledger,
goals=goals,
concurrency=k,
)
def _ran(result: PortfolioResult) -> list[str]:
"""The project ids that actually RAN, in ``runs`` order."""
return [r.outcome.proposal.project_id for r in result.runs]
async def test_goal_stop_membership_is_identical_across_concurrency() -> None:
"""WHICH projects run under a goal contract must not depend on ``concurrency`` (SC6 under D-D).
The wave-model reading of the existing ``break``/``continue`` semantics, now pinned rather than
only described: goal checks run at WAVE ASSEMBLY, never mid-wave. A HARD per-project goal
removes that pid from its wave before the wave starts; a HARD portfolio goal stops the pass, and
the in-flight wave still crosses its merge barrier before the loop exits. Checking mid-wave
instead would make membership depend on which project's round-trips finished first — the exact
nondeterminism S3.3 exists to avoid.
**Detach point: check the goal at WAVE granularity instead of per member -> this test goes RED
(measured by keying the per-project lookup on ``wave_ids[0]``, which leaves the skipped pid in
the wave at k=3 while k=1 is unaffected, since at k=1 the wave IS the member).**
**What is NOT the detach point, corrected against measurement rather than assumed.** The
intuitive claim — "check mid-wave and membership becomes completion-order dependent" — is
false here, and stating it would have been an over-read. ``_goal_limit_if_reached`` reads only
the ledger, the goal contract and the baseline, and all three are INVARIANT for the duration of
a pass (C3, pinned by the one-writer test below). A check moved later in the wave therefore
reads exactly the same values and reaches exactly the same decision. Determinism of membership
comes from the ledger being static, NOT from where the check sits; what wave-assembly placement
actually buys is that a skipped project is never STARTED, so no work and no store write happen
for a project the goal excluded.
**The two halves are not equally strong, and this test does not pretend otherwise.** The
per-project skip is the informative half: two projects still run, concurrently, and the skipped
pid must be absent at both ``k``. The portfolio hard stop is the weak half — because the ledger
is static during a pass (C3, and Step 5's one-writer guard below turns that from prose into a
test), a reached portfolio goal is observed on the very first check, so BOTH ``k`` yield an
empty ``runs`` and the equality is nearly free. It is asserted anyway because a regression that
let a hard stop leak one wave's worth of runs would show up here first."""
# Strong half — a per-project HARD goal skips exactly that pid, at every k.
per_project = GoalConfig(per_project={"RV13-RAS-TP": GoalContract(absolute_ore=1000)})
probe = _Recorder()
concurrent = await _goal_pass(
3, ledger=_prefilled("RV13-RAS-TP", 1000), goals=per_project, recorder=probe
)
sequential = await _goal_pass(
1, ledger=_prefilled("RV13-RAS-TP", 1000), goals=per_project, recorder=_Recorder()
)
assert probe.max_in_flight > 1, (
f"max in-flight was {probe.max_in_flight}: the surviving two projects never overlapped, so "
"membership under concurrency is untested here"
)
assert _ran(concurrent) == _ran(sequential) == ["FV42-GSV-E1", "BRU-LAKS-REHAB"], (
f"membership diverged with k: k=3 ran {_ran(concurrent)}, k=1 ran {_ran(sequential)}"
)
assert concurrent.stopped_early is False and sequential.stopped_early is False, (
"a per-project skip is not a pass-stop"
)
# Weak half — a HARD portfolio goal already reached stops the pass identically at every k.
portfolio = GoalConfig(portfolio=GoalContract(absolute_ore=100))
stopped_concurrent = await _goal_pass(
3, ledger=_prefilled("FV42-GSV-E1", 100), goals=portfolio, recorder=_Recorder()
)
stopped_sequential = await _goal_pass(
1, ledger=_prefilled("FV42-GSV-E1", 100), goals=portfolio, recorder=_Recorder()
)
assert _ran(stopped_concurrent) == _ran(stopped_sequential) == []
assert stopped_concurrent.stopped_early is stopped_sequential.stopped_early is True
assert stopped_concurrent.stop_reason == stopped_sequential.stop_reason, (
"the goal-stop DECISION itself must not depend on k — frozen-dataclass equality"
)
def test_no_thread_or_process_path_exists_under_src() -> None:
"""NG1, structural: S3.3's concurrency is ONE asyncio event loop and nothing else.
A ratchet, green today, in the style of ``test_okf.py::test_okf_is_maf_free`` and AST-based for
the same reason — a raw substring scan would trip on prose that merely NAMES the forbidden
mechanism, which this very docstring does.
Why it is worth a test rather than a convention: MAF's thread-safety is undocumented, and Spike
B (``a2dff21``) measured workflow state bleeding across reused instances. A ``run_in_executor``
or ``asyncio.to_thread`` hop introduced later would move project runs onto threads where none
of S3.3's reasoning holds — the wave snapshot argument is about coroutines interleaving at
``await`` points, not about parallel memory access — and it would do so without any existing
test noticing. ``asyncio.to_thread`` is included deliberately: it is the modern, innocuous-
looking spelling of exactly the hop this guard forbids."""
src_dir = Path(__file__).resolve().parents[1] / "src" / "portfolio_optimiser"
forbidden_modules = {"threading", "_thread", "multiprocessing", "concurrent"}
forbidden_calls = {"run_in_executor", "to_thread", "ThreadPoolExecutor", "ProcessPoolExecutor"}
offences: list[str] = []
for path in sorted(src_dir.rglob("*.py")):
tree = ast.parse(path.read_text(encoding="utf-8"))
for node in ast.walk(tree):
if isinstance(node, ast.Import):
offences += [
f"{path.name}: import {a.name}"
for a in node.names
if a.name.split(".")[0] in forbidden_modules
]
elif isinstance(node, ast.ImportFrom):
if (node.module or "").split(".")[0] in forbidden_modules:
offences.append(f"{path.name}: from {node.module} import ...")
elif isinstance(node, ast.Call):
name = getattr(node.func, "attr", None) or getattr(node.func, "id", None)
if name in forbidden_calls:
offences.append(f"{path.name}: {name}(...)")
assert offences == [], (
f"a thread/process execution path exists under src/ ({offences}) — S3.3's determinism "
"argument covers coroutines on one loop only (NG1)"
)
async def test_concurrent_pass_does_not_write_on_the_run_path() -> None:
"""The one-writer rule, turned from a prose invariant into a measurement (C3, assumption 2).
The whole goal-stop design rests on the ledger being STATIC during a pass: ``_goal_limit_if_
reached`` reads an ACCUMULATED sum that earlier, out-of-band HITL realizations produced, and it
reads it while projects are running. If a run realized savings mid-pass, the reads would race
the writes and goal membership would stop being deterministic — so assumption 2 is load-bearing
for the test above, not merely tidy.
Likewise ``promote_verdict``: the wiki is the CURATED layer behind the fail-closed Step-8 gate.
A concurrent pass writing to it would push raw agent output into next run's navigation context
without any human/persona approval — self-contamination, at k-times the rate.
Both are asserted by TRIPWIRE rather than by grep: the module attribute is replaced, so any call
reaching it through any path fails the test, including one added later in a module this file
never mentions. ``write_verdict`` is deliberately NOT covered here — it is guarded by the Steg-7
role-split tests, and duplicating it would imply this test knows something about the outbox
that it does not."""
calls: list[str] = []
def _tripwire(name: str) -> Any:
def _fail(*_args: Any, **_kwargs: Any) -> Any:
calls.append(name)
raise AssertionError(f"{name} was called on the run path during a concurrent pass")
return _fail
with (
mock.patch.object(verdicts_mod, "promote_verdict", _tripwire("promote_verdict")),
mock.patch.object(ledger_mod, "realize", _tripwire("realize")),
):
_, result = await _pass(3, _Recorder())
assert calls == [], f"one-writer rule violated during a concurrent pass: {calls}"
assert len(result.runs) == 3, (
"the pass must still have completed — an empty pass proves nothing"
)