The guard checked whether the installed SDK satisfied the pin. Nobody had ever checked whether anyone had READ it. Those are different questions, and the gap between them was a whole version range: pinned >=0.2.111,<0.3, premises source-verified through 0.2.110, installed 0.2.120. Every build in between was admissible and unexamined — `uv sync --upgrade` would have kept 806 tests green on an SDK no one had opened. Written red first: a guard handed 0.2.140 returned it without complaint. _VERIFIED_THROUGH is the ratchet. It records the newest build actually read at source, and a newer one fails naming the five premises to re-check. The pin is untouched and was never the defect — measurement dissolved the premise that it needed lifting. It was not too narrow but too wide, and a wider permission is not repaired by widening it further. The premises themselves were prose the failure message recited. Nothing tested them, so one that stopped being true would have surfaced on the one live paid run (S10, D6). They are now a table introspected against the installed package, with the printed prose derived from that same table so a checked attribute cannot go unreported or a reported one unchecked. The premise introspection structurally cannot see — that query() yields an AssistantMessage then a closing ResultMessage — is named apart, and is the honest reason the human reading still has to happen. Value-proved, not merely named: disabling the ratchet reds 1 test, stubbing the inventory to "no gaps" reds 3, re-hardcoding the prose reds 1, and lowering _VERIFIED_THROUGH below the installed build reds the real installed-version test rather than only a monkeypatched one. 0.2.139 read at source (0.2.120 -> 0.2.139, latest on PyPI today; STATE said 0.2.134, measured 08-09 and stale). The public query.py is byte-identical, every premise field keeps its type and default, and the parser changes are additive. One needed a look: 0.2.139 added a skills path defaulting setting_sources to ["user", "project"], which would have undone the S10 isolation fix — it fires only on None, so the explicit [] is out of reach. Prose carrying stale version claims moved with the reading, never ahead of it: each was re-verified at 0.2.139 before being restated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014dKDjVG7qrBh9NkAAxutqN
316 lines
12 KiB
Python
316 lines
12 KiB
Python
"""SDK/API preflight (method-spec §4.1/§10 analog; S4.1; paritetsrad 20; K7).
|
|
|
|
Everything that CAN be validated WITHOUT a model call is validated HERE, before
|
|
the operator pays for one. The preflight is the BOUNDARY: it never calls
|
|
``query()``, never validates the credential VALUE online (only that one is SET
|
|
and is not an obvious placeholder form), never touches the network. It returns a
|
|
list of structured ``Refusal``s — one actionable message per deficiency; an
|
|
empty list means "clear to run". The CLI exits non-zero on any refusal so a
|
|
broken config stops here, cheaply, instead of on the first billed call.
|
|
|
|
Four offline checks (for a given backend profile, ``anthropic`` today):
|
|
|
|
* **credential** — a credential is available (``ANTHROPIC_API_KEY``, or the
|
|
bundled CLI's own ``CLAUDE_CODE_OAUTH_TOKEN``) and is not a placeholder form.
|
|
The value itself is NEVER checked online — a bad key surfaces on the first
|
|
call, never here (§1 honesty: the preflight claims no more than it proves).
|
|
* **model_map** — the requested profile exists, and every id it resolves to
|
|
(THROUGH ``resolve_model``, so the ``default`` fall-through is covered) is a
|
|
real id, not a placeholder left in the config.
|
|
* **sdk** — ``claude_agent_sdk`` imports (a missing install is a structured
|
|
refusal naming ``uv sync``, never an ImportError), the run-path symbols exist,
|
|
and the bundled Claude Code CLI is present on disk — located offline via the
|
|
SDK package's own files, mirroring the SDK's ``_find_cli`` order (bundled
|
|
first, then a ``claude`` on PATH). Verified against installed 0.2.139 by
|
|
reading its ``_find_cli`` (bundled at line 250, PATH at 256) — the order is
|
|
unchanged from 0.2.120.
|
|
* **termination** — the §8 stop contract constructs with positive round/token
|
|
caps and the per-call USD cap is positive (the budget belt is SET).
|
|
|
|
Importing the SDK to prove it imports is offline-safe (no key, no socket) — the
|
|
whole suite already imports the client. The no-network grep-guard
|
|
(``tests/test_preflight.py``) proves this module carries no HTTP path of its own.
|
|
|
|
Run: uv run python -m portfolio_optimiser_claude.preflight [--profile anthropic]
|
|
[--max-rounds N] [--max-tokens N] [--max-budget-usd-per-call F]
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import importlib
|
|
import os
|
|
import platform
|
|
import shutil
|
|
from dataclasses import dataclass
|
|
from pathlib import Path
|
|
from typing import Any, Mapping
|
|
|
|
from pydantic import ValidationError
|
|
|
|
from portfolio_optimiser_claude.contracts import (
|
|
ModelMapContract,
|
|
TerminationContract,
|
|
_bundled_model_map,
|
|
resolve_model,
|
|
)
|
|
|
|
# Tokens that never appear in a real Anthropic key or Claude model id, but do in
|
|
# the placeholders operators leave behind. Matched case-insensitively as a
|
|
# SUBSTRING so ``sk-ant-YOUR-KEY`` and ``<model>`` are both caught. Kept
|
|
# conservative to avoid ever refusing a genuine value (§1: no false alarms).
|
|
_PLACEHOLDER_TOKENS: tuple[str, ...] = (
|
|
"placeholder",
|
|
"your-",
|
|
"your_",
|
|
"yourkey",
|
|
"changeme",
|
|
"change-me",
|
|
"change_me",
|
|
"replace",
|
|
"example",
|
|
"todo",
|
|
"fixme",
|
|
"dummy",
|
|
"xxxx",
|
|
"<",
|
|
">",
|
|
"...",
|
|
)
|
|
|
|
# A probe role guaranteed absent from any mapping, used to exercise the
|
|
# resolve_model default fall-through (the id an UNMAPPED role would receive).
|
|
_FALLTHROUGH_PROBE = "__preflight_fallthrough_probe__"
|
|
|
|
# The run-path symbols sdk_client.py depends on; their presence is what
|
|
# "the SDK imports" must actually mean (a bare module is not enough).
|
|
_REQUIRED_SDK_SYMBOLS: tuple[str, ...] = ("query", "ClaudeAgentOptions")
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class Refusal:
|
|
"""One preflight deficiency: which check tripped + an actionable message."""
|
|
|
|
check: str
|
|
detail: str
|
|
|
|
|
|
def _looks_like_placeholder(value: str) -> bool:
|
|
"""True for an empty/whitespace value or one carrying a placeholder token.
|
|
|
|
Shared by the credential and model-id checks so a stub left in either place
|
|
is caught by the same conservative rule (never validates a value online)."""
|
|
low = value.strip().lower()
|
|
if not low:
|
|
return True
|
|
return any(token in low for token in _PLACEHOLDER_TOKENS)
|
|
|
|
|
|
def _check_credentials(env: Mapping[str, str]) -> list[Refusal]:
|
|
"""A usable credential is present (§10) — the VALUE is never checked online.
|
|
|
|
``ANTHROPIC_API_KEY`` is the named contract; the bundled CLI's own
|
|
``CLAUDE_CODE_OAUTH_TOKEN`` also satisfies it (run_s10 relies on the CLI's
|
|
credentials when no key is exported — refusing that would be a false alarm).
|
|
A credential set to a placeholder form is refused; NEITHER set is refused.
|
|
"""
|
|
key = env.get("ANTHROPIC_API_KEY", "").strip()
|
|
oauth = env.get("CLAUDE_CODE_OAUTH_TOKEN", "").strip()
|
|
if key:
|
|
if _looks_like_placeholder(key):
|
|
return [
|
|
Refusal(
|
|
"credential",
|
|
"ANTHROPIC_API_KEY is set to a placeholder-form value — export a "
|
|
"real key (its value is never validated online, only that one is set).",
|
|
)
|
|
]
|
|
return []
|
|
if oauth and not _looks_like_placeholder(oauth):
|
|
return [] # the bundled CLI authenticates via its OAuth token
|
|
return [
|
|
Refusal(
|
|
"credential",
|
|
"no API credential: export ANTHROPIC_API_KEY (or authenticate the bundled "
|
|
"CLI, which sets CLAUDE_CODE_OAUTH_TOKEN). The value is never checked online "
|
|
"— only that a credential is present.",
|
|
)
|
|
]
|
|
|
|
|
|
def _check_model_map(profile: str, *, model_map: ModelMapContract | None = None) -> list[Refusal]:
|
|
"""The requested profile exists and every id it RESOLVES to is real (§10).
|
|
|
|
Placeholder refusal runs THROUGH ``resolve_model`` (not a raw dict read), so
|
|
the ``default`` fall-through an unmapped role receives is checked too. An
|
|
unknown profile — which ``resolve_model`` would raise on — becomes a
|
|
structured refusal, never a stack trace before spend.
|
|
"""
|
|
resolved = ModelMapContract(**_bundled_model_map()) if model_map is None else model_map
|
|
mapping = resolved.profiles.get(profile)
|
|
if mapping is None:
|
|
have = ", ".join(sorted(resolved.profiles)) or "(none)"
|
|
return [
|
|
Refusal(
|
|
"model_map",
|
|
f"backend profile '{profile}' is not in model_map (have: {have}).",
|
|
)
|
|
]
|
|
refusals: list[Refusal] = []
|
|
# Every configured role PLUS the fall-through the default serves — all via
|
|
# resolve_model, deduped so one placeholder id is reported once.
|
|
for role in sorted(mapping) + [_FALLTHROUGH_PROBE]:
|
|
model_id = resolve_model(resolved, role, profile=profile)
|
|
if _looks_like_placeholder(model_id):
|
|
refusals.append(
|
|
Refusal(
|
|
"model_map",
|
|
f"model id '{model_id}' (profile '{profile}') is a placeholder form — "
|
|
"set a real Claude model id (verified against platform.claude.com/docs).",
|
|
)
|
|
)
|
|
break # a placeholder id is a single config fault; one message suffices
|
|
return refusals
|
|
|
|
|
|
def _locate_cli(sdk: Any) -> Path | None:
|
|
"""The CLI the SDK would spawn, resolved OFFLINE via the package's files.
|
|
|
|
Mirrors the SDK's ``_find_cli`` order: the bundled binary shipped in the
|
|
wheel first, then a ``claude`` on PATH. No network, no subprocess — just a
|
|
file-existence probe, so the preflight can prove there IS a CLI to run.
|
|
"""
|
|
cli_name = "claude.exe" if platform.system() == "Windows" else "claude"
|
|
bundled = Path(sdk.__file__).parent / "_bundled" / cli_name
|
|
if bundled.is_file():
|
|
return bundled
|
|
found = shutil.which("claude")
|
|
return Path(found) if found else None
|
|
|
|
|
|
def _check_sdk() -> list[Refusal]:
|
|
"""The SDK imports, exposes the run-path symbols, and its CLI is present.
|
|
|
|
A missing install is a structured refusal naming the fix (never an
|
|
ImportError raised out of the preflight itself). Importing is offline-safe.
|
|
"""
|
|
try:
|
|
sdk = importlib.import_module("claude_agent_sdk")
|
|
except ImportError as exc:
|
|
return [
|
|
Refusal(
|
|
"sdk",
|
|
f"claude-agent-sdk is not importable ({exc}); run 'uv sync' to install the "
|
|
"pinned SDK (it bundles the Claude Code CLI — no separate install).",
|
|
)
|
|
]
|
|
missing = [name for name in _REQUIRED_SDK_SYMBOLS if not hasattr(sdk, name)]
|
|
if missing:
|
|
return [
|
|
Refusal(
|
|
"sdk",
|
|
f"claude-agent-sdk is installed but missing run-path symbols {missing} — "
|
|
"the pin may be out of the verified range (see test_sdk_version_guard).",
|
|
)
|
|
]
|
|
if _locate_cli(sdk) is None:
|
|
pkg_file = sdk.__file__
|
|
bundled = f"{Path(pkg_file).parent / '_bundled'}" if pkg_file else "the SDK package"
|
|
return [
|
|
Refusal(
|
|
"sdk",
|
|
f"no Claude Code CLI found (neither the SDK's bundled binary under {bundled} "
|
|
"nor a 'claude' on PATH); reinstall claude-agent-sdk or pass "
|
|
"ClaudeAgentOptions(cli_path=...).",
|
|
)
|
|
]
|
|
return []
|
|
|
|
|
|
def _check_termination(
|
|
max_rounds: int, max_tokens: int, max_budget_usd_per_call: float
|
|
) -> list[Refusal]:
|
|
"""The §8 stop contract constructs and the per-call USD belt is positive."""
|
|
refusals: list[Refusal] = []
|
|
try:
|
|
TerminationContract(max_rounds=max_rounds, max_tokens=max_tokens)
|
|
except ValidationError as exc:
|
|
refusals.append(
|
|
Refusal(
|
|
"termination",
|
|
f"stop contract invalid — max_rounds/max_tokens must be positive (§8): {exc}",
|
|
)
|
|
)
|
|
if max_budget_usd_per_call <= 0:
|
|
refusals.append(
|
|
Refusal(
|
|
"termination",
|
|
f"max_budget_usd_per_call must be positive, got {max_budget_usd_per_call} "
|
|
"(the per-call USD belt must be SET before any spend).",
|
|
)
|
|
)
|
|
return refusals
|
|
|
|
|
|
def run_preflight(
|
|
*,
|
|
profile: str = "anthropic",
|
|
env: Mapping[str, str] | None = None,
|
|
max_rounds: int = 12,
|
|
max_tokens: int = 150_000,
|
|
max_budget_usd_per_call: float = 0.25,
|
|
) -> list[Refusal]:
|
|
"""Run every offline check; the refusals are the union (empty = clear to run).
|
|
|
|
No check makes a model call or a network request — this is the boundary the
|
|
operator crosses BEFORE spending. ``env`` defaults to ``os.environ``.
|
|
"""
|
|
environ = os.environ if env is None else env
|
|
return [
|
|
*_check_credentials(environ),
|
|
*_check_model_map(profile),
|
|
*_check_sdk(),
|
|
*_check_termination(max_rounds, max_tokens, max_budget_usd_per_call),
|
|
]
|
|
|
|
|
|
def main(argv: list[str] | None = None) -> int:
|
|
"""The thin CLI: run the offline preflight → print refusals → exit code.
|
|
|
|
Exit 0: clear to run (no refusal). Exit 1: one or more refusals — printed
|
|
with their fix, and a line stating no model call was made (the preflight IS
|
|
the boundary; a green preflight never implies a validated key or a run).
|
|
"""
|
|
parser = argparse.ArgumentParser(
|
|
description=(
|
|
"Validate everything that CAN be checked WITHOUT a model call, BEFORE any "
|
|
"spend — credential presence, model_map, SDK + bundled CLI, stop/budget "
|
|
"contract (offline; the preflight IS the boundary, it never calls the API)."
|
|
)
|
|
)
|
|
parser.add_argument("--profile", type=str, default="anthropic")
|
|
parser.add_argument("--max-rounds", type=int, default=12)
|
|
parser.add_argument("--max-tokens", type=int, default=150_000)
|
|
parser.add_argument("--max-budget-usd-per-call", type=float, default=0.25)
|
|
args = parser.parse_args(argv)
|
|
|
|
refusals = run_preflight(
|
|
profile=args.profile,
|
|
max_rounds=args.max_rounds,
|
|
max_tokens=args.max_tokens,
|
|
max_budget_usd_per_call=args.max_budget_usd_per_call,
|
|
)
|
|
if not refusals:
|
|
print(
|
|
f"PREFLIGHT OK — clear to run (profile={args.profile}); no model call was "
|
|
"made (the preflight is the boundary, not a validated key or a run)."
|
|
)
|
|
return 0
|
|
print(f"PREFLIGHT FAILED — {len(refusals)} refusal(s), no spend (no model call was made):")
|
|
for refusal in refusals:
|
|
print(f" [{refusal.check}] {refusal.detail}")
|
|
return 1
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|