feat(run): stamp the drill's SDK build in the dry-run run-config [skip-docs]
The K8 drill captured a run-config that described the rig it rehearsed — model ids, parameters, caps — without saying which SDK build would drive it. The SDK's reported USD figure is computed against a price table frozen at build time, so a rig record without the build is not traceable, and the drill exists precisely to rig a future live run. build_dry_run_config now takes the client the drill constructed and reads the build from it, the same seam rule the provenance stamp follows: a drill driven by the scripted stand-in stamps null rather than the installed version, because reading the environment would describe a rig that never existed (§1). Load-bearing (§11): the two new tests went RED before the change (no such key), and the detach point is named in the class docstring — read importlib.metadata instead of the client and the scripted drill claims a build it never used. The existing dry-run tests assert individual keys rather than a key set, so the additive field leaves them untouched, and byte-determinism still holds. 624 -> 627 passed, ruff + mypy --strict clean. README states the new field. STATE post 2a, approved by the operator this session. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
parent
fc4a536e09
commit
f300c64b0e
3 changed files with 80 additions and 2 deletions
|
|
@ -116,8 +116,9 @@ description, never from its code)
|
|||
same orchestration with a scripted client; only the CLI's default constructs the SDK
|
||||
client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
|
||||
would (contracts → compose → client construction → preflight) and captures a `run_id`-named
|
||||
`runconfig` + `preflight` pair to the outbox (model-id, parameters, caps — no wall-clock, so
|
||||
the bytes stay deterministic), then **stops before the first model call**. It exits 0 when
|
||||
`runconfig` + `preflight` pair to the outbox (model-id, parameters, caps, and the SDK build
|
||||
read from the client the drill constructed — no wall-clock, so the bytes stay
|
||||
deterministic), then **stops before the first model call**. It exits 0 when
|
||||
the preflight is clear and non-zero when it refused, but captures the artifacts and makes
|
||||
zero model calls either way — a future operator-gated live run is rigged and rehearsed
|
||||
offline, with no spend.
|
||||
|
|
|
|||
|
|
@ -443,6 +443,7 @@ def write_value_report(
|
|||
def build_dry_run_config(
|
||||
contracts: Contracts,
|
||||
*,
|
||||
client: ModelClient,
|
||||
profile: str,
|
||||
bundle_name: str,
|
||||
run_id: str,
|
||||
|
|
@ -461,11 +462,18 @@ def build_dry_run_config(
|
|||
NO wall-clock date: the outbox promises byte-determinism (same input + run_id
|
||||
⇒ identical file), and the run's date is stamped at report time (§4 pt 3),
|
||||
never into the deterministic log.
|
||||
|
||||
``client`` is the client the drill CONSTRUCTED (never called): the rig's SDK
|
||||
build is read from it, the same seam rule the provenance stamp follows. A
|
||||
drill driven by the scripted stand-in used no SDK and stamps null — reading
|
||||
the installed version from the environment would describe a rig that never
|
||||
existed (§1).
|
||||
"""
|
||||
return {
|
||||
"run_id": run_id,
|
||||
"profile": profile,
|
||||
"bundle": bundle_name,
|
||||
"sdk_version": _client_sdk_version(client),
|
||||
"models": {
|
||||
role: resolve_model(contracts.model_map, role, profile=profile)
|
||||
for role in (_PROPOSER_ROLE, _CHECKER_ROLE)
|
||||
|
|
@ -793,6 +801,7 @@ def main(
|
|||
)
|
||||
run_config = build_dry_run_config(
|
||||
contracts,
|
||||
client=client,
|
||||
profile=_DEFAULT_PROFILE,
|
||||
bundle_name=args.bundle.name,
|
||||
run_id=run_id,
|
||||
|
|
|
|||
|
|
@ -9,6 +9,10 @@ says which build produced it. But a run driven by the scripted stand-in used no
|
|||
SDK at all — stamping the installed version there would attribute a build to a
|
||||
run that never touched it, which is exactly the fabrication §1 forbids.
|
||||
|
||||
The same rule governs the K8 drill's run-config: it records the build of the
|
||||
client the drill CONSTRUCTED (and never called), so the rehearsed rig is
|
||||
traceable without ever claiming a build the drill did not use.
|
||||
|
||||
Detach proof: make the stamp read ``importlib.metadata`` instead of the client
|
||||
→ a scripted run claims an SDK build it never used → red.
|
||||
|
||||
|
|
@ -128,6 +132,70 @@ class TestTheStampStaysOptional:
|
|||
)
|
||||
|
||||
|
||||
class TestTheDrillStampsTheSameWay:
|
||||
"""LOAD-BEARING (§1, §11): the K8 drill's run-config obeys the same seam rule.
|
||||
|
||||
The run-config is the comparison protocol's §4 pt 3 record of what a live
|
||||
run WOULD carry — model ids, parameters, caps. Without the build id it
|
||||
describes a rig whose SDK version is unknown, which is the one thing that
|
||||
makes a reported USD figure traceable. It is read from the CLIENT the drill
|
||||
constructed, exactly like the provenance stamp: the drill builds the client
|
||||
before it stops, so the honest source is right there.
|
||||
|
||||
Detach point: stamp ``importlib.metadata`` (or drop the client argument and
|
||||
hard-code a value) → the scripted drill claims a build it never used → RED.
|
||||
"""
|
||||
|
||||
def _drill_runconfig(self, tmp_path: Path, *, sdk_version: str | None) -> dict[str, object]:
|
||||
created: list[ScriptedClient] = []
|
||||
|
||||
def factory(contracts: Contracts, max_budget_usd_per_call: float) -> ModelClient:
|
||||
# No replies: any model call would both record and raise, so the
|
||||
# drill's zero-call promise still holds while this runs.
|
||||
client = ScriptedClient(replies=[])
|
||||
if sdk_version is not None:
|
||||
client.sdk_version = sdk_version # type: ignore[attr-defined]
|
||||
created.append(client)
|
||||
return client
|
||||
|
||||
outbox = tmp_path / "outbox"
|
||||
main(
|
||||
[
|
||||
"--bundle",
|
||||
str(BUNDLE),
|
||||
"--outbox",
|
||||
str(outbox),
|
||||
"--run-id",
|
||||
"drill-sdk",
|
||||
"--live-dry-run",
|
||||
],
|
||||
client_factory=factory,
|
||||
)
|
||||
(client,) = created
|
||||
assert client.calls == [] # the drill still made no model call
|
||||
payload: dict[str, object] = json.loads(
|
||||
(outbox / "drill-sdk-runconfig.json").read_text(encoding="utf-8")
|
||||
)
|
||||
return payload
|
||||
|
||||
def test_a_drill_not_produced_by_the_sdk_stamps_no_build(self, tmp_path: Path) -> None:
|
||||
payload = self._drill_runconfig(tmp_path, sdk_version=None)
|
||||
assert payload["sdk_version"] is None
|
||||
assert payload["sdk_version"] != importlib.metadata.version("claude-agent-sdk")
|
||||
|
||||
def test_the_drills_client_has_its_build_stamped_verbatim(self, tmp_path: Path) -> None:
|
||||
payload = self._drill_runconfig(tmp_path, sdk_version=FIXTURE_BUILD)
|
||||
assert payload["sdk_version"] == FIXTURE_BUILD
|
||||
|
||||
def test_the_stamp_does_not_disturb_the_rest_of_the_run_config(self, tmp_path: Path) -> None:
|
||||
# The field is additive: §4 pt 3's existing content is unchanged, and the
|
||||
# log still carries no wall-clock date (determinism).
|
||||
payload = self._drill_runconfig(tmp_path, sdk_version=FIXTURE_BUILD)
|
||||
assert payload["run_id"] == "drill-sdk"
|
||||
assert payload["profile"] == "anthropic"
|
||||
assert "date" not in payload
|
||||
|
||||
|
||||
class TestTheFasitIsNotRetroStamped:
|
||||
"""LOAD-BEARING (§1): the historical record is never back-filled."""
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue