Fase 1b, funn 1b. Den første levende kjøringen brant tolv runder på svar som
ikke lot seg parse til IR-formen; e371890 gjorde teksten synlig, dette fjerner
årsaken. generate_via_llm sender nå
options={"response_format": proposal_response_format()} på hvert
genererings-kall.
Formen er MÅLT, ikke valgt. ChatOptions.response_format tar
type[BaseModel] | Mapping, og begge profiler ærer den: LOCAL sender en Mapping
ordrett til Chat Completions, AZURE (FoundryChatClient -> RawFoundryChatClient
-> RawOpenAIChatClient) konverterer samme envelope til Responses-APIets
text.format. Klassen — det korteste svaret — er avvist på bevis: gitt en klasse
konverterer klienten med type_to_response_format_param, som emitterer
minimum/exclusiveMinimum/minItems/prefixItems og et assumptions-node hvis
additionalProperties er et skjema. Azures publiserte subset utelukker alle fire.
assumptions kan ikke bare droppes, og det er også en måling: validator
._monte_carlo faller tilbake på item.unit_cost for hver kode uten bånd, så uten
bånd er alle 512 samples identiske og P10 == P50 == P90. Den stokastiske
falsifisereren ville gått inert mens den fortsatt rapporterte persentiler.
Wire-en bærer derfor et array av navngitte entries som _parse_ir folder tilbake
til IR-ens map — additivt, aldri erstatning. Skjemaet deriveres fra
SavingsProposal; sanitiseren er fail-closed (StructuredOutputUnsupported).
Load-bearing målt mot hele suiten, seks mutasjoner alle røde, grønn kontroll
864/4: detach wiringen (1) · detach sanitiseren (3) · dropp assumptions fra
skjemaet (1) · fail-closed -> stille reparasjon (1) · detach normaliseringen
(3) · erstatning i stedet for tillegg (2, inkl. golden-transkriptet).
T3 ble skrevet vakuøs først og felt av sin egen mutasjon: den påsto å bli rød
når assumptions forsvant fra skjemaet, men den scriptede klienten ignorerer
skjemaet. Testen fikk en direkte assert på skjemaet.
Ærlighets-grense: ingen betalt kjøring gjort. Testene beviser konformitet med
det dokumenterte subsettet, ikke aksept fra det levende endepunktet.
859 -> 864 passed / 4 skipped. ruff + format + mypy rene.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EQNU4tfAhsBvdefT1jUhk
133 lines
7 KiB
Python
133 lines
7 KiB
Python
"""S2.5 (Step 9) consolidation guard (T-2.5e): the four scripted ``_inner_get_response`` bodies
|
|
collapsed to ONE canonical client (``simulation.ScriptedChatClient``); conftest's three test doubles
|
|
now SUBCLASS it. These grep-guards lock that in — they go RED if a divergent ``_inner_get_response``
|
|
is re-added, if a ``src``→``tests`` import creeps in, or if a double stops subclassing the canonical.
|
|
|
|
The guard is a regression lock over an already-verified consolidation: before the collapse there were
|
|
five ``def _inner_get_response`` sites (four scripted + test_step5's own-lineage double); after, two.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import re
|
|
from pathlib import Path
|
|
|
|
_ROOT = Path(__file__).resolve().parents[1]
|
|
|
|
|
|
def _py_files(base: str) -> list[Path]:
|
|
return sorted((_ROOT / base).rglob("*.py"))
|
|
|
|
|
|
# Files permitted to define ``_inner_get_response``. The invariant this guard protects is that the
|
|
# scripted BODY is not duplicated — not that the def-site count is frozen. Adding a file here is a
|
|
# deliberate act: a new entry must either be the canonical, or a thin override that DELEGATES to it
|
|
# (which ``test_delegating_overrides_call_super`` below then enforces mechanically).
|
|
_CANONICAL_SITE = "src/portfolio_optimiser/simulation.py"
|
|
|
|
# Overrides in the SCRIPTED lineage — they subclass ``ScriptedChatClient``, so a body of their own
|
|
# would be a copy of the canonical. They must delegate.
|
|
_DELEGATING_OVERRIDES = [
|
|
# S3.3 ordering probe: yields to the event loop N times, then delegates. It cannot live in the
|
|
# reply-selector seam, which the canonical calls synchronously and so can never await.
|
|
"tests/test_portfolio_concurrent_loadbearing.py",
|
|
# S3.3 failure-accounting probe: RAISES for one project (after N completed calls), otherwise
|
|
# delegates. Like the ordering probe it cannot live in the reply-selector seam — that seam
|
|
# returns a reply string, and this double's whole subject is the absence of one.
|
|
"tests/test_portfolio_failure_accounting_loadbearing.py",
|
|
# B4 tool-call probe: its first response is a function CALL rather than text, then it delegates.
|
|
# It cannot live in the reply-selector seam either — that seam returns a reply STRING, and a
|
|
# response that is not text is precisely this double's subject.
|
|
"tests/test_b4_mcp_call_trace_loadbearing.py",
|
|
# Fase 1b structured-output probe: records the ``options`` mapping of every generation call,
|
|
# then delegates. It cannot live in the reply-selector seam either — that seam is handed
|
|
# ``(prompt_blob, role)`` and returns a reply string, and ``options`` (this double's whole
|
|
# subject) never reaches it.
|
|
"tests/test_structured_output_loadbearing.py",
|
|
]
|
|
|
|
# Doubles in a DIFFERENT lineage (``spikes._harness.FakeChatClient``). There is no canonical
|
|
# scripted body above them to delegate to, so the delegation rule does not apply — but the
|
|
# separation is asserted rather than assumed, so a file cannot be parked here to dodge the rule.
|
|
_FOREIGN_LINEAGE = ["tests/test_step5_refine_loadbearing.py"]
|
|
|
|
|
|
def test_inner_get_response_collapsed_to_two_sites() -> None:
|
|
"""The four scripted clients collapse to ONE canonical ``_inner_get_response``
|
|
(``simulation.py``). Every other def-site must be a registered, DELEGATING override — never a
|
|
fourth copy of the body.
|
|
|
|
The guard originally pinned a literal count of 2. That made it fail on any new legitimate
|
|
subclass while still passing if someone pasted a duplicated body into an already-listed file —
|
|
a count is the wrong shape for the invariant. The list below plus
|
|
``test_delegating_overrides_call_super`` pin the property itself."""
|
|
sites = [
|
|
p.relative_to(_ROOT).as_posix()
|
|
for base in ("src", "tests")
|
|
for p in _py_files(base)
|
|
if p.name != Path(__file__).name # this guard file references the pattern in prose
|
|
and "def _inner_get_response" in p.read_text(encoding="utf-8")
|
|
]
|
|
expected = sorted([_CANONICAL_SITE, *_DELEGATING_OVERRIDES, *_FOREIGN_LINEAGE])
|
|
assert sorted(sites) == expected, (
|
|
f"unregistered ``_inner_get_response`` def-site — the scripted body must not be copied. "
|
|
f"Expected {expected}, got: {sites}"
|
|
)
|
|
|
|
|
|
def test_foreign_lineage_doubles_are_genuinely_foreign() -> None:
|
|
"""A file listed as foreign lineage must NOT subclass the scripted canonical.
|
|
|
|
Without this, ``_FOREIGN_LINEAGE`` would be an escape hatch: any scripted-lineage subclass
|
|
could be moved into that list to skip the delegation rule below."""
|
|
for site in _FOREIGN_LINEAGE:
|
|
text = (_ROOT / site).read_text(encoding="utf-8")
|
|
assert "ScriptedChatClient" not in text, (
|
|
f"{site} is registered as foreign lineage but references ``ScriptedChatClient`` — if it "
|
|
"is in the scripted lineage it belongs in _DELEGATING_OVERRIDES and must delegate"
|
|
)
|
|
|
|
|
|
def test_delegating_overrides_call_super() -> None:
|
|
"""Every scripted-lineage override actually DELEGATES to the canonical rather than
|
|
reimplementing it.
|
|
|
|
This is the strength the literal count never had: without it, a file already on the list could
|
|
grow a full copy of the scripted body and the consolidation would be cosmetic again."""
|
|
for site in _DELEGATING_OVERRIDES:
|
|
text = (_ROOT / site).read_text(encoding="utf-8")
|
|
# Match the delegation ITSELF — ``super()._inner_get_response`` or the explicit
|
|
# ``super(Cls, self)._inner_get_response`` form a nested function needs. Searching for
|
|
# "super(" and "_inner_get_response" independently would pass on any file that merely
|
|
# calls ``super().__init__`` near a def, which is accidental-green, not a guard.
|
|
assert re.search(r"super\([^)]*\)\._inner_get_response", text), (
|
|
f"{site} defines ``_inner_get_response`` but never delegates to the canonical via "
|
|
"``super()._inner_get_response`` — that is a duplicated body, which is exactly what "
|
|
"this guard exists to prevent"
|
|
)
|
|
|
|
|
|
def test_no_src_imports_tests() -> None:
|
|
"""No ``src`` module imports from ``tests`` — the canonical lives in ``src/simulation.py`` so
|
|
``conftest`` imports ``src``, never the reverse (the forbidden src→tests direction)."""
|
|
offenders = [
|
|
p.relative_to(_ROOT).as_posix()
|
|
for p in _py_files("src")
|
|
if ("from tests" in (text := p.read_text(encoding="utf-8")) or "import tests" in text)
|
|
]
|
|
assert offenders == [], f"src must not import tests: {offenders}"
|
|
|
|
|
|
def test_conftest_doubles_subclass_canonical() -> None:
|
|
"""conftest's three test doubles genuinely SUBCLASS the canonical ``ScriptedChatClient``
|
|
(delegating the shared body) — so the consolidation is real, not cosmetic."""
|
|
from conftest import (
|
|
ScriptedChatClient,
|
|
SyntheticUsageChatClient,
|
|
_ProjectAwareUsageChatClient,
|
|
_RecordingChatClient,
|
|
)
|
|
|
|
assert issubclass(SyntheticUsageChatClient, ScriptedChatClient)
|
|
assert issubclass(_ProjectAwareUsageChatClient, ScriptedChatClient)
|
|
assert issubclass(_RecordingChatClient, ScriptedChatClient)
|