open/portfolio-optimiser-claude
1
0
Fork 0
Code Issues Releases 1 Activity

feat(portfolio): K7 — SDK/API preflight, offline pre-spend boundary (parity row 20) [skip-docs]

Browse source 
Everything that CAN be validated WITHOUT a model call is validated BEFORE the
operator pays for one (S4.1-analog, SDK-native — Foundry-auth is MAF-specific,
not mirrored). The preflight IS the boundary: it never calls query(), never
validates a credential VALUE online, never touches the network. It returns a
list of structured, actionable Refusals; the CLI exits non-zero on any, so a
broken config stops cheaply instead of on the first billed call.

- preflight.py: run_preflight + `python -m …preflight --profile anthropic`.
  Four offline checks:
  * credential — ANTHROPIC_API_KEY set + not a placeholder form (the value is
    NEVER checked online, only presence, §1); the bundled CLI's own
    CLAUDE_CODE_OAUTH_TOKEN also satisfies it (run_s10 relies on it — refusing
    would be a false alarm).
  * model_map — the requested profile exists and every id it resolves to,
    THROUGH resolve_model (so the default fall-through is covered), is real,
    not a placeholder left in config.
  * sdk — claude_agent_sdk imports (a missing install is a structured refusal
    naming `uv sync`, never an ImportError out of the preflight), the run-path
    symbols exist, and the bundled Claude Code CLI is present on disk — located
    OFFLINE via the SDK package's own files, mirroring the SDK's _find_cli
    order (bundled first, then a claude on PATH). Verified against installed
    0.2.120 (bundled binary present; the STATE 0.2.110 note was stale).
  * termination — the §8 stop contract constructs with positive caps and the
    per-call USD belt is positive.
- tests/test_preflight.py: credential contract, placeholder detector, model_map
  incl. unknown-profile-without-raising, SDK + bundled-CLI offline probe (with a
  monkeypatched missing-SDK refusal), termination, run_preflight aggregation,
  CLI both paths, and the offline guards (no network import, no query() call —
  AST-based so prose stays green). THREE seams detach-proven RED: credential
  branch, placeholder model-id guard, no-network grep-guard.

478→514 green, golden byte-exact, full gate clean (ruff+format+mypy strict,
24 src files), run_s10.py/runs/ byte-untouched. README test-count sync ×2 +
preflight.py module note + load-bearing mention. IKKE-scope (held): the actual
API call (ALDRI — the preflight IS the boundary) and Foundry/Azure auth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RiTwaKLesgcwXx2mDviqpt
This commit is contained in:
Kjell Tore Guttormsen 2026-07-24 01:34:18 +02:00
commit c08d92a358
3 changed files with 588 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

17
README.md
View file

@ -13,7 +13,7 @@ human-in-the-loop, and the system learns from the verdicts.
> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
> seam, each proven by load-bearing tests (478 tests, all running offline without an API
> seam, each proven by load-bearing tests (514 tests, all running offline without an API
> key). The programme's single budgeted **live model run has been executed and validated** —
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
@ -118,6 +118,15 @@ description, never from its code)
with no price fails fast — there is no hardcoded rate anywhere (a grep-guard proves it), and
the figure is marked `ESTIMAT` (the whole cap billed at the rate is an upper bound; real runs
cost less). `uv run python -m portfolio_optimiser_claude.costsim`.
- `preflight.py` — the SDK/API preflight (**offline** — the boundary the operator crosses
*before* any spend): everything that can be validated without a model call is checked here,
so a broken config stops cheaply instead of on the first billed call. Four checks — a
credential is present and not a placeholder form (`ANTHROPIC_API_KEY`, or the bundled CLI's
own `CLAUDE_CODE_OAUTH_TOKEN`; the value itself is *never* validated online), the model_map
profile exists and every id it resolves to is real, `claude_agent_sdk` imports and its
bundled CLI is present on disk, and the §8 stop/budget contract is set. It never calls the
API — a green preflight implies no more than that (§1). Each deficiency is a structured,
actionable refusal. `uv run python -m portfolio_optimiser_claude.preflight`.
### Load-bearing tests (§11)
@ -133,7 +142,9 @@ verdicts; the promoted signal stays out of the read-context),
`test_portfolio_learning_loadbearing.py` (a verdict available at project k survives into
project k+1's fold via the shared store, with a marker-absent control),
`test_outbox_loadbearing.py` (a completed run's `run_id`-named outbox pair is written on the
entrance path, with a no-outbox control, and the outcome carries the inbox join key), and
entrance path, with a no-outbox control, and the outcome carries the inbox join key),
`test_preflight.py` (a missing credential and a placeholder model id are each refused before
any spend, and the preflight carries no network path of its own), and
`test_sdk_isolation.py` (local config cannot capture the checker).
## The ingest layer — CSV and SQL, in front of the loop
@ -196,7 +207,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
```bash
uv sync # install dependencies
uv run pytest # 478 tests — run without any API key and without network
uv run pytest # 514 tests — run without any API key and without network
uv run ruff check . && uv run ruff format --check .
uv run mypy src # strict
```

314
src/portfolio_optimiser_claude/preflight.py Normal file
View file

@ -0,0 +1,314 @@
"""SDK/API preflight (method-spec §4.1/§10 analog; S4.1; paritetsrad 20; K7).
Everything that CAN be validated WITHOUT a model call is validated HERE, before
the operator pays for one. The preflight is the BOUNDARY: it never calls
``query()``, never validates the credential VALUE online (only that one is SET
and is not an obvious placeholder form), never touches the network. It returns a
list of structured ``Refusal``s — one actionable message per deficiency; an
empty list means "clear to run". The CLI exits non-zero on any refusal so a
broken config stops here, cheaply, instead of on the first billed call.
Four offline checks (for a given backend profile, ``anthropic`` today):
* **credential** — a credential is available (``ANTHROPIC_API_KEY``, or the
bundled CLI's own ``CLAUDE_CODE_OAUTH_TOKEN``) and is not a placeholder form.
The value itself is NEVER checked online — a bad key surfaces on the first
call, never here (§1 honesty: the preflight claims no more than it proves).
* **model_map** — the requested profile exists, and every id it resolves to
(THROUGH ``resolve_model``, so the ``default`` fall-through is covered) is a
real id, not a placeholder left in the config.
* **sdk** — ``claude_agent_sdk`` imports (a missing install is a structured
refusal naming ``uv sync``, never an ImportError), the run-path symbols exist,
and the bundled Claude Code CLI is present on disk — located offline via the
SDK package's own files, mirroring the SDK's ``_find_cli`` order (bundled
first, then a ``claude`` on PATH). Verified against installed 0.2.120.
* **termination** — the §8 stop contract constructs with positive round/token
caps and the per-call USD cap is positive (the budget belt is SET).
Importing the SDK to prove it imports is offline-safe (no key, no socket) — the
whole suite already imports the client. The no-network grep-guard
(``tests/test_preflight.py``) proves this module carries no HTTP path of its own.
Run: uv run python -m portfolio_optimiser_claude.preflight [--profile anthropic]
[--max-rounds N] [--max-tokens N] [--max-budget-usd-per-call F]
"""
from __future__ import annotations
import argparse
import importlib
import os
import platform
import shutil
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
from pydantic import ValidationError
from portfolio_optimiser_claude.contracts import (
ModelMapContract,
TerminationContract,
_bundled_model_map,
resolve_model,
)
# Tokens that never appear in a real Anthropic key or Claude model id, but do in
# the placeholders operators leave behind. Matched case-insensitively as a
# SUBSTRING so ``sk-ant-YOUR-KEY`` and ``<model>`` are both caught. Kept
# conservative to avoid ever refusing a genuine value (§1: no false alarms).
_PLACEHOLDER_TOKENS: tuple[str, ...] = (
"placeholder",
"your-",
"your_",
"yourkey",
"changeme",
"change-me",
"change_me",
"replace",
"example",
"todo",
"fixme",
"dummy",
"xxxx",
"<",
">",
"...",
)
# A probe role guaranteed absent from any mapping, used to exercise the
# resolve_model default fall-through (the id an UNMAPPED role would receive).
_FALLTHROUGH_PROBE = "__preflight_fallthrough_probe__"
# The run-path symbols sdk_client.py depends on; their presence is what
# "the SDK imports" must actually mean (a bare module is not enough).
_REQUIRED_SDK_SYMBOLS: tuple[str, ...] = ("query", "ClaudeAgentOptions")
@dataclass(frozen=True)
class Refusal:
"""One preflight deficiency: which check tripped + an actionable message."""
check: str
detail: str
def _looks_like_placeholder(value: str) -> bool:
"""True for an empty/whitespace value or one carrying a placeholder token.
Shared by the credential and model-id checks so a stub left in either place
is caught by the same conservative rule (never validates a value online)."""
low = value.strip().lower()
if not low:
return True
return any(token in low for token in _PLACEHOLDER_TOKENS)
def _check_credentials(env: Mapping[str, str]) -> list[Refusal]:
"""A usable credential is present (§10) — the VALUE is never checked online.
``ANTHROPIC_API_KEY`` is the named contract; the bundled CLI's own
``CLAUDE_CODE_OAUTH_TOKEN`` also satisfies it (run_s10 relies on the CLI's
credentials when no key is exported — refusing that would be a false alarm).
A credential set to a placeholder form is refused; NEITHER set is refused.
"""
key = env.get("ANTHROPIC_API_KEY", "").strip()
oauth = env.get("CLAUDE_CODE_OAUTH_TOKEN", "").strip()
if key:
if _looks_like_placeholder(key):
return [
Refusal(
"credential",
"ANTHROPIC_API_KEY is set to a placeholder-form value — export a "
"real key (its value is never validated online, only that one is set).",
)
]
return []
if oauth and not _looks_like_placeholder(oauth):
return [] # the bundled CLI authenticates via its OAuth token
return [
Refusal(
"credential",
"no API credential: export ANTHROPIC_API_KEY (or authenticate the bundled "
"CLI, which sets CLAUDE_CODE_OAUTH_TOKEN). The value is never checked online "
"— only that a credential is present.",
)
]
def _check_model_map(profile: str, *, model_map: ModelMapContract | None = None) -> list[Refusal]:
"""The requested profile exists and every id it RESOLVES to is real (§10).
Placeholder refusal runs THROUGH ``resolve_model`` (not a raw dict read), so
the ``default`` fall-through an unmapped role receives is checked too. An
unknown profile — which ``resolve_model`` would raise on — becomes a
structured refusal, never a stack trace before spend.
"""
resolved = ModelMapContract(**_bundled_model_map()) if model_map is None else model_map
mapping = resolved.profiles.get(profile)
if mapping is None:
have = ", ".join(sorted(resolved.profiles)) or "(none)"
return [
Refusal(
"model_map",
f"backend profile '{profile}' is not in model_map (have: {have}).",
)
]
refusals: list[Refusal] = []
# Every configured role PLUS the fall-through the default serves — all via
# resolve_model, deduped so one placeholder id is reported once.
for role in sorted(mapping) + [_FALLTHROUGH_PROBE]:
model_id = resolve_model(resolved, role, profile=profile)
if _looks_like_placeholder(model_id):
refusals.append(
Refusal(
"model_map",
f"model id '{model_id}' (profile '{profile}') is a placeholder form — "
"set a real Claude model id (verified against platform.claude.com/docs).",
)
)
break # a placeholder id is a single config fault; one message suffices
return refusals
def _locate_cli(sdk: Any) -> Path | None:
"""The CLI the SDK would spawn, resolved OFFLINE via the package's files.
Mirrors the SDK's ``_find_cli`` order: the bundled binary shipped in the
wheel first, then a ``claude`` on PATH. No network, no subprocess — just a
file-existence probe, so the preflight can prove there IS a CLI to run.
"""
cli_name = "claude.exe" if platform.system() == "Windows" else "claude"
bundled = Path(sdk.__file__).parent / "_bundled" / cli_name
if bundled.is_file():
return bundled
found = shutil.which("claude")
return Path(found) if found else None
def _check_sdk() -> list[Refusal]:
"""The SDK imports, exposes the run-path symbols, and its CLI is present.
A missing install is a structured refusal naming the fix (never an
ImportError raised out of the preflight itself). Importing is offline-safe.
"""
try:
sdk = importlib.import_module("claude_agent_sdk")
except ImportError as exc:
return [
Refusal(
"sdk",
f"claude-agent-sdk is not importable ({exc}); run 'uv sync' to install the "
"pinned SDK (it bundles the Claude Code CLI — no separate install).",
)
]
missing = [name for name in _REQUIRED_SDK_SYMBOLS if not hasattr(sdk, name)]
if missing:
return [
Refusal(
"sdk",
f"claude-agent-sdk is installed but missing run-path symbols {missing} — "
"the pin may be out of the verified range (see test_sdk_version_guard).",
)
]
if _locate_cli(sdk) is None:
pkg_file = sdk.__file__
bundled = f"{Path(pkg_file).parent / '_bundled'}" if pkg_file else "the SDK package"
return [
Refusal(
"sdk",
f"no Claude Code CLI found (neither the SDK's bundled binary under {bundled} "
"nor a 'claude' on PATH); reinstall claude-agent-sdk or pass "
"ClaudeAgentOptions(cli_path=...).",
)
]
return []
def _check_termination(
max_rounds: int, max_tokens: int, max_budget_usd_per_call: float
) -> list[Refusal]:
"""The §8 stop contract constructs and the per-call USD belt is positive."""
refusals: list[Refusal] = []
try:
TerminationContract(max_rounds=max_rounds, max_tokens=max_tokens)
except ValidationError as exc:
refusals.append(
Refusal(
"termination",
f"stop contract invalid — max_rounds/max_tokens must be positive (§8): {exc}",
)
)
if max_budget_usd_per_call <= 0:
refusals.append(
Refusal(
"termination",
f"max_budget_usd_per_call must be positive, got {max_budget_usd_per_call} "
"(the per-call USD belt must be SET before any spend).",
)
)
return refusals
def run_preflight(
*,
profile: str = "anthropic",
env: Mapping[str, str] | None = None,
max_rounds: int = 12,
max_tokens: int = 150_000,
max_budget_usd_per_call: float = 0.25,
) -> list[Refusal]:
"""Run every offline check; the refusals are the union (empty = clear to run).
No check makes a model call or a network request — this is the boundary the
operator crosses BEFORE spending. ``env`` defaults to ``os.environ``.
"""
environ = os.environ if env is None else env
return [
*_check_credentials(environ),
*_check_model_map(profile),
*_check_sdk(),
*_check_termination(max_rounds, max_tokens, max_budget_usd_per_call),
]
def main(argv: list[str] | None = None) -> int:
"""The thin CLI: run the offline preflight → print refusals → exit code.
Exit 0: clear to run (no refusal). Exit 1: one or more refusals — printed
with their fix, and a line stating no model call was made (the preflight IS
the boundary; a green preflight never implies a validated key or a run).
"""
parser = argparse.ArgumentParser(
description=(
"Validate everything that CAN be checked WITHOUT a model call, BEFORE any "
"spend — credential presence, model_map, SDK + bundled CLI, stop/budget "
"contract (offline; the preflight IS the boundary, it never calls the API)."
)
)
parser.add_argument("--profile", type=str, default="anthropic")
parser.add_argument("--max-rounds", type=int, default=12)
parser.add_argument("--max-tokens", type=int, default=150_000)
parser.add_argument("--max-budget-usd-per-call", type=float, default=0.25)
args = parser.parse_args(argv)
refusals = run_preflight(
profile=args.profile,
max_rounds=args.max_rounds,
max_tokens=args.max_tokens,
max_budget_usd_per_call=args.max_budget_usd_per_call,
)
if not refusals:
print(
f"PREFLIGHT OK — clear to run (profile={args.profile}); no model call was "
"made (the preflight is the boundary, not a validated key or a run)."
)
return 0
print(f"PREFLIGHT FAILED — {len(refusals)} refusal(s), no spend (no model call was made):")
for refusal in refusals:
print(f" [{refusal.check}] {refusal.detail}")
return 1
if __name__ == "__main__":
raise SystemExit(main())

260
tests/test_preflight.py Normal file
View file

@ -0,0 +1,260 @@
"""SDK/API preflight (K7, S4.1-analog; paritetsrad 20) — LOAD-BEARING (§11).
Everything that CAN be validated WITHOUT a model call is validated here, BEFORE
the operator pays for one. Every seam is load-bearing: the RED-when-detached
notes on the three required proofs (credential, placeholder model id, no-network
grep-guard) name the mutation that makes them fail, so a green-but-dead check
can't hide.
Offline invariant: the preflight never calls ``query()``, never validates a key
VALUE online, never touches the network. Importing ``claude_agent_sdk`` (to prove
importability + bundled CLI presence) is offline-safe — the whole suite already
imports the SDK client without a key or a socket.
"""
from __future__ import annotations
import ast
import importlib
from pathlib import Path
import pytest
from portfolio_optimiser_claude.contracts import ModelMapContract
from portfolio_optimiser_claude.preflight import (
Refusal,
_check_credentials,
_check_model_map,
_check_sdk,
_check_termination,
_locate_cli,
_looks_like_placeholder,
main,
run_preflight,
)
SRC_PKG = Path(__file__).resolve().parents[1] / "src" / "portfolio_optimiser_claude"
_REAL_KEY = "sk-ant-api03-abc123def456" # shape only — never validated online
def _kinds(refusals: list[Refusal]) -> set[str]:
return {r.check for r in refusals}
class TestCredentialContract:
"""RED (detach the no-credential branch → green without a key): a run with
NO API credential set is refused BEFORE any spend — the key VALUE, however,
is never checked online, only that one is present and not a placeholder."""
def test_no_credential_is_refused(self) -> None:
refusals = _check_credentials({})
assert any(r.check == "credential" for r in refusals)
def test_a_set_key_clears_the_credential_check(self) -> None:
assert _check_credentials({"ANTHROPIC_API_KEY": _REAL_KEY}) == []
def test_a_placeholder_key_is_refused(self) -> None:
refusals = _check_credentials({"ANTHROPIC_API_KEY": "your-key-here"})
assert any(r.check == "credential" for r in refusals)
assert any("placeholder" in r.detail.lower() for r in refusals)
def test_the_key_value_is_never_asserted_only_its_presence(self) -> None:
# Any non-placeholder string satisfies the contract — the preflight is
# the boundary; a bad key surfaces on the FIRST call, never here (§1).
assert _check_credentials({"ANTHROPIC_API_KEY": "obviously-not-a-real-key-42"}) == []
def test_the_bundled_cli_oauth_token_satisfies_the_credential(self) -> None:
# run_s10 relies on the bundled CLI's own credentials when no key is
# exported; refusing that would be a false alarm (§1 honesty).
assert _check_credentials({"CLAUDE_CODE_OAUTH_TOKEN": "sk-ant-oat01-real"}) == []
class TestPlaceholderModelIdRefused:
"""RED (detach the resolve_model placeholder guard → a placeholder id passes):
a model_map whose resolved id is a placeholder form is refused — and the check
goes THROUGH resolve_model, so the ``default`` fall-through is covered too."""
def test_a_placeholder_default_model_id_is_refused(self) -> None:
model_map = ModelMapContract(profiles={"anthropic": {"default": "REPLACE_ME"}})
refusals = _check_model_map("anthropic", model_map=model_map)
assert any(r.check == "model_map" for r in refusals)
def test_a_placeholder_role_model_id_is_refused(self) -> None:
model_map = ModelMapContract(
profiles={"anthropic": {"default": "claude-haiku-4-5-20251001", "proposer": "<todo>"}}
)
refusals = _check_model_map("anthropic", model_map=model_map)
assert any(r.check == "model_map" for r in refusals)
def test_a_real_model_map_clears(self) -> None:
model_map = ModelMapContract(
profiles={"anthropic": {"default": "claude-haiku-4-5-20251001"}}
)
assert _check_model_map("anthropic", model_map=model_map) == []
def test_an_unknown_profile_is_refused_without_raising(self) -> None:
# resolve_model would raise on an unknown profile; the preflight turns
# that into a structured refusal, never a stack trace before spend.
model_map = ModelMapContract(
profiles={"anthropic": {"default": "claude-haiku-4-5-20251001"}}
)
refusals = _check_model_map("bedrock", model_map=model_map)
assert any(r.check == "model_map" for r in refusals)
assert any("bedrock" in r.detail for r in refusals)
def test_the_bundled_model_map_is_clear(self) -> None:
# Nøkkelantakelse: the shipped model_map configures only real ids.
assert _check_model_map("anthropic") == []
class TestPlaceholderDetector:
"""The shared placeholder detector guards BOTH the key and the model id path."""
@pytest.mark.parametrize(
"value",
["", " ", "your-key-here", "REPLACE_ME", "changeme", "<model>", "TODO", "xxxx-xxxx"],
)
def test_placeholder_forms_are_caught(self, value: str) -> None:
assert _looks_like_placeholder(value)
@pytest.mark.parametrize("value", ["claude-haiku-4-5-20251001", _REAL_KEY, "sk-ant-oat01-real"])
def test_real_values_pass(self, value: str) -> None:
assert not _looks_like_placeholder(value)
class TestSdkAndBundledCli:
"""Nøkkelantakelse (verified against installed claude-agent-sdk 0.2.120): the
SDK imports and its bundled CLI is present on disk — checked OFFLINE via the
package's own files, mirroring the SDK's ``_find_cli`` order. Never a net call."""
def test_the_installed_sdk_clears(self) -> None:
assert _check_sdk() == []
def test_the_bundled_cli_is_locatable_offline(self) -> None:
sdk = importlib.import_module("claude_agent_sdk")
cli = _locate_cli(sdk)
assert cli is not None
assert cli.exists()
def test_a_missing_sdk_is_refused_not_crashed(self, monkeypatch: pytest.MonkeyPatch) -> None:
# Detach-proof for graceful degradation: an uninstallable SDK becomes a
# structured refusal naming the fix, never an ImportError at preflight.
import portfolio_optimiser_claude.preflight as pf
def _boom(name: str) -> object:
raise ImportError(f"no module named {name}")
monkeypatch.setattr(pf.importlib, "import_module", _boom)
refusals = _check_sdk()
assert any(r.check == "sdk" for r in refusals)
assert any("claude-agent-sdk" in r.detail for r in refusals)
class TestTerminationContract:
"""§8: the stop/budget contract must be SET and valid before any spend."""
def test_positive_caps_clear(self) -> None:
assert _check_termination(12, 150_000, 0.25) == []
@pytest.mark.parametrize("rounds,tokens", [(0, 150_000), (12, 0), (-1, 10)])
def test_non_positive_stop_caps_are_refused(self, rounds: int, tokens: int) -> None:
refusals = _check_termination(rounds, tokens, 0.25)
assert any(r.check == "termination" for r in refusals)
@pytest.mark.parametrize("cap", [0.0, -0.5])
def test_non_positive_usd_cap_is_refused(self, cap: float) -> None:
refusals = _check_termination(12, 150_000, cap)
assert any(r.check == "termination" for r in refusals)
class TestRunPreflightAggregates:
"""The whole preflight is the union of its checks — offline, no model call."""
def test_a_good_config_yields_no_refusals(self) -> None:
assert run_preflight(profile="anthropic", env={"ANTHROPIC_API_KEY": _REAL_KEY}) == []
def test_a_bad_config_collects_every_failing_dimension(self) -> None:
refusals = run_preflight(
profile="bedrock", # unknown profile
env={}, # no credential
max_rounds=0, # bad stop contract
max_tokens=150_000,
)
assert {"credential", "model_map", "termination"} <= _kinds(refusals)
class TestCli:
"""The thin CLI: refusals → non-zero exit + actionable lines; clear → 0.
Honesty (§1): the output states no model call was made — the preflight IS
the boundary, so a green preflight never implies a validated key or a run.
"""
def test_clear_run_returns_zero(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
monkeypatch.setenv("ANTHROPIC_API_KEY", _REAL_KEY)
monkeypatch.delenv("CLAUDE_CODE_OAUTH_TOKEN", raising=False)
rc = main(["--profile", "anthropic"])
out = capsys.readouterr().out
assert rc == 0
assert "OK" in out
assert "no model call" in out.lower()
def test_missing_credential_returns_one_with_actionable_line(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("CLAUDE_CODE_OAUTH_TOKEN", raising=False)
rc = main(["--profile", "anthropic"])
out = capsys.readouterr().out
assert rc == 1
assert "ANTHROPIC_API_KEY" in out
assert "no spend" in out.lower() or "no model call" in out.lower()
def _imported_module_names(module_path: Path) -> set[str]:
tree = ast.parse(module_path.read_text(encoding="utf-8"))
names: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
names.update(alias.name.split(".")[0] for alias in node.names)
elif isinstance(node, ast.ImportFrom) and node.level == 0 and node.module:
names.add(node.module.split(".")[0])
return names
def _called_names(module_path: Path) -> set[str]:
"""Every function/method NAME called in the module (AST, ignores prose)."""
tree = ast.parse(module_path.read_text(encoding="utf-8"))
called: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Call):
func = node.func
if isinstance(func, ast.Name):
called.add(func.id)
elif isinstance(func, ast.Attribute):
called.add(func.attr)
return called
class TestPreflightIsOffline:
"""RED (add ``import socket`` / ``import httpx`` to preflight.py): the
grep-guard proves the preflight carries NO network path — it validates the
SDK by IMPORTING it, never by reaching the API."""
def test_no_network_module_is_imported(self) -> None:
# The offline seam (grep-guard, AST form): no socket/httpx path exists —
# importing any of these is the detach that turns this RED.
names = _imported_module_names(SRC_PKG / "preflight.py")
assert not names & {"socket", "urllib", "http", "requests", "httpx", "anthropic"}
def test_no_sdk_completion_is_called(self) -> None:
# The preflight IS the boundary: it may IMPORT the SDK but must never
# invoke query()/ClaudeSDKClient — that call would be the spend it guards.
# AST-based, so a docstring mentioning query() stays green; only a real
# call trips it (add ``query(prompt=...)`` → RED).
called = _called_names(SRC_PKG / "preflight.py")
assert "query" not in called
assert "ClaudeSDKClient" not in called