open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 1 Activity

fix(s42): close review WARN — hermetic dry-run env + scoped azure hint

Browse source 
Post-hoc /trekreview of S4.2 surfaced two confirmed findings; both closed via TDD.

S42-001 (MAJOR): the new --live-dry-run CLI tests read PORTFOLIO_MODEL_MAP /
PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT via resolve_model/AzureFoundryBackend but did
not isolate them, so both arms inverted their rc in a Foundry-configured env.
Add an autouse fixture that delenvs both, mirroring test_backends/test_preflight.

S42-002 (MINOR): the --live-dry-run except ValueError attached the azure-preflight
remediation to every offline-path ValueError (unknown project_id, empty docs_dir,
bundle mismatch). Scope the hint to args.profile == "azure"; structured refusal +
rc 1 preserved for all. New test proves a LOCAL unknown-project refusal carries no
azure hint.

Gate: pytest 358 passed / 4 skipped, ruff check + format clean, mypy 24 files.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-15 18:30:30 +02:00
commit ce5b1151c8
2 changed files with 46 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

35
tests/test_live_dry_run.py
View file

@ -10,11 +10,25 @@ from __future__ import annotations
from pathlib import Path
import pytest
from portfolio_optimiser import run
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
@pytest.fixture(autouse=True)
def _isolate_model_env(monkeypatch: pytest.MonkeyPatch) -> None:
"""Hermetic env: clear the S4.1 out-of-tree overrides so these CLI assertions read the BUNDLED
map/config, not the operator's Foundry-configured environment. Without this, ``PORTFOLIO_MODEL_MAP``
(an azure-only override wins over the bundled placeholder, ``backends.py:48-56``) and
``PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT`` (``backends.py:103``) invert both arms' outcomes — the
LOCAL success arm can't resolve, the azure refusal arm resolves a real id. Mirrors the sibling
isolation in ``test_backends.py``/``test_preflight.py`` (both ``delenv PORTFOLIO_MODEL_MAP``)."""
monkeypatch.delenv("PORTFOLIO_MODEL_MAP", raising=False)
monkeypatch.delenv("PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT", raising=False)
def test_cli_live_dry_run_ok(capsys) -> None:
"""Success arm: LOCAL profile (offline client construction) + ``--live-dry-run`` → rc 0 and a
summary naming the profile + a no-model-call note. No socket (stops before ``debate.run``)."""
@ -55,3 +69,24 @@ def test_cli_live_dry_run_azure_placeholder_refuses(capsys) -> None:
err = capsys.readouterr().err
assert "refused" in err.lower()
assert "preflight" in err.lower()
def test_cli_live_dry_run_local_unknown_project_no_azure_hint(capsys) -> None:
"""A LOCAL-profile refusal from a non-config ``ValueError`` (here: an unknown project_id) must
still be a STRUCTURED refusal + ``rc 1``, but must NOT carry the azure-preflight remediation —
that hint is only meaningful for the AZURE config gate (S4.1). Guards against the over-broad
``except`` re-attaching an azure-specific hint to every offline-path ``ValueError``."""
rc = run.main(
[
"NOPE-UNKNOWN-PROJ",
"--docs-dir",
str(BUNDLE_DIR),
"--bundle-dir",
str(BUNDLE_DIR),
"--live-dry-run",
]
)
assert rc == 1
err = capsys.readouterr().err
assert "refused" in err.lower()
assert "preflight" not in err.lower()