feat(run): stamp the drill's SDK build in the dry-run run-config [skip-docs]

The K8 drill captured a run-config that described the rig it rehearsed —
model ids, parameters, caps — without saying which SDK build would drive it.
The SDK's reported USD figure is computed against a price table frozen at build
time, so a rig record without the build is not traceable, and the drill exists
precisely to rig a future live run.

build_dry_run_config now takes the client the drill constructed and reads the
build from it, the same seam rule the provenance stamp follows: a drill driven
by the scripted stand-in stamps null rather than the installed version, because
reading the environment would describe a rig that never existed (§1).

Load-bearing (§11): the two new tests went RED before the change (no such key),
and the detach point is named in the class docstring — read importlib.metadata
instead of the client and the scripted drill claims a build it never used. The
existing dry-run tests assert individual keys rather than a key set, so the
additive field leaves them untouched, and byte-determinism still holds.

624 -> 627 passed, ruff + mypy --strict clean. README states the new field.
STATE post 2a, approved by the operator this session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 15:35:16 +02:00
commit f300c64b0e
3 changed files with 80 additions and 2 deletions

View file

@ -116,8 +116,9 @@ description, never from its code)
same orchestration with a scripted client; only the CLI's default constructs the SDK
client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
would (contracts → compose → client construction → preflight) and captures a `run_id`-named
`runconfig` + `preflight` pair to the outbox (model-id, parameters, caps — no wall-clock, so
the bytes stay deterministic), then **stops before the first model call**. It exits 0 when
`runconfig` + `preflight` pair to the outbox (model-id, parameters, caps, and the SDK build
read from the client the drill constructed — no wall-clock, so the bytes stay
deterministic), then **stops before the first model call**. It exits 0 when
the preflight is clear and non-zero when it refused, but captures the artifacts and makes
zero model calls either way — a future operator-gated live run is rigged and rehearsed
offline, with no spend.

View file

@ -443,6 +443,7 @@ def write_value_report(
def build_dry_run_config(
contracts: Contracts,
*,
client: ModelClient,
profile: str,
bundle_name: str,
run_id: str,
@ -461,11 +462,18 @@ def build_dry_run_config(
NO wall-clock date: the outbox promises byte-determinism (same input + run_id
identical file), and the run's date is stamped at report time (§4 pt 3),
never into the deterministic log.
``client`` is the client the drill CONSTRUCTED (never called): the rig's SDK
build is read from it, the same seam rule the provenance stamp follows. A
drill driven by the scripted stand-in used no SDK and stamps null reading
the installed version from the environment would describe a rig that never
existed (§1).
"""
return {
"run_id": run_id,
"profile": profile,
"bundle": bundle_name,
"sdk_version": _client_sdk_version(client),
"models": {
role: resolve_model(contracts.model_map, role, profile=profile)
for role in (_PROPOSER_ROLE, _CHECKER_ROLE)
@ -793,6 +801,7 @@ def main(
)
run_config = build_dry_run_config(
contracts,
client=client,
profile=_DEFAULT_PROFILE,
bundle_name=args.bundle.name,
run_id=run_id,

View file

@ -9,6 +9,10 @@ says which build produced it. But a run driven by the scripted stand-in used no
SDK at all stamping the installed version there would attribute a build to a
run that never touched it, which is exactly the fabrication §1 forbids.
The same rule governs the K8 drill's run-config: it records the build of the
client the drill CONSTRUCTED (and never called), so the rehearsed rig is
traceable without ever claiming a build the drill did not use.
Detach proof: make the stamp read ``importlib.metadata`` instead of the client
a scripted run claims an SDK build it never used red.
@ -128,6 +132,70 @@ class TestTheStampStaysOptional:
)
class TestTheDrillStampsTheSameWay:
"""LOAD-BEARING (§1, §11): the K8 drill's run-config obeys the same seam rule.
The run-config is the comparison protocol's §4 pt 3 record of what a live
run WOULD carry model ids, parameters, caps. Without the build id it
describes a rig whose SDK version is unknown, which is the one thing that
makes a reported USD figure traceable. It is read from the CLIENT the drill
constructed, exactly like the provenance stamp: the drill builds the client
before it stops, so the honest source is right there.
Detach point: stamp ``importlib.metadata`` (or drop the client argument and
hard-code a value) the scripted drill claims a build it never used RED.
"""
def _drill_runconfig(self, tmp_path: Path, *, sdk_version: str | None) -> dict[str, object]:
created: list[ScriptedClient] = []
def factory(contracts: Contracts, max_budget_usd_per_call: float) -> ModelClient:
# No replies: any model call would both record and raise, so the
# drill's zero-call promise still holds while this runs.
client = ScriptedClient(replies=[])
if sdk_version is not None:
client.sdk_version = sdk_version # type: ignore[attr-defined]
created.append(client)
return client
outbox = tmp_path / "outbox"
main(
[
"--bundle",
str(BUNDLE),
"--outbox",
str(outbox),
"--run-id",
"drill-sdk",
"--live-dry-run",
],
client_factory=factory,
)
(client,) = created
assert client.calls == [] # the drill still made no model call
payload: dict[str, object] = json.loads(
(outbox / "drill-sdk-runconfig.json").read_text(encoding="utf-8")
)
return payload
def test_a_drill_not_produced_by_the_sdk_stamps_no_build(self, tmp_path: Path) -> None:
payload = self._drill_runconfig(tmp_path, sdk_version=None)
assert payload["sdk_version"] is None
assert payload["sdk_version"] != importlib.metadata.version("claude-agent-sdk")
def test_the_drills_client_has_its_build_stamped_verbatim(self, tmp_path: Path) -> None:
payload = self._drill_runconfig(tmp_path, sdk_version=FIXTURE_BUILD)
assert payload["sdk_version"] == FIXTURE_BUILD
def test_the_stamp_does_not_disturb_the_rest_of_the_run_config(self, tmp_path: Path) -> None:
# The field is additive: §4 pt 3's existing content is unchanged, and the
# log still carries no wall-clock date (determinism).
payload = self._drill_runconfig(tmp_path, sdk_version=FIXTURE_BUILD)
assert payload["run_id"] == "drill-sdk"
assert payload["profile"] == "anthropic"
assert "date" not in payload
class TestTheFasitIsNotRetroStamped:
"""LOAD-BEARING (§1): the historical record is never back-filled."""