feat(run): stamp the drill's SDK build in the dry-run run-config [skip-docs]

The K8 drill captured a run-config that described the rig it rehearsed —
model ids, parameters, caps — without saying which SDK build would drive it.
The SDK's reported USD figure is computed against a price table frozen at build
time, so a rig record without the build is not traceable, and the drill exists
precisely to rig a future live run.

build_dry_run_config now takes the client the drill constructed and reads the
build from it, the same seam rule the provenance stamp follows: a drill driven
by the scripted stand-in stamps null rather than the installed version, because
reading the environment would describe a rig that never existed (§1).

Load-bearing (§11): the two new tests went RED before the change (no such key),
and the detach point is named in the class docstring — read importlib.metadata
instead of the client and the scripted drill claims a build it never used. The
existing dry-run tests assert individual keys rather than a key set, so the
additive field leaves them untouched, and byte-determinism still holds.

624 -> 627 passed, ruff + mypy --strict clean. README states the new field.
STATE post 2a, approved by the operator this session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 15:35:16 +02:00
commit f300c64b0e
3 changed files with 80 additions and 2 deletions

View file

@ -116,8 +116,9 @@ description, never from its code)
same orchestration with a scripted client; only the CLI's default constructs the SDK same orchestration with a scripted client; only the CLI's default constructs the SDK
client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
would (contracts → compose → client construction → preflight) and captures a `run_id`-named would (contracts → compose → client construction → preflight) and captures a `run_id`-named
`runconfig` + `preflight` pair to the outbox (model-id, parameters, caps — no wall-clock, so `runconfig` + `preflight` pair to the outbox (model-id, parameters, caps, and the SDK build
the bytes stay deterministic), then **stops before the first model call**. It exits 0 when read from the client the drill constructed — no wall-clock, so the bytes stay
deterministic), then **stops before the first model call**. It exits 0 when
the preflight is clear and non-zero when it refused, but captures the artifacts and makes the preflight is clear and non-zero when it refused, but captures the artifacts and makes
zero model calls either way — a future operator-gated live run is rigged and rehearsed zero model calls either way — a future operator-gated live run is rigged and rehearsed
offline, with no spend. offline, with no spend.

View file

@ -443,6 +443,7 @@ def write_value_report(
def build_dry_run_config( def build_dry_run_config(
contracts: Contracts, contracts: Contracts,
*, *,
client: ModelClient,
profile: str, profile: str,
bundle_name: str, bundle_name: str,
run_id: str, run_id: str,
@ -461,11 +462,18 @@ def build_dry_run_config(
NO wall-clock date: the outbox promises byte-determinism (same input + run_id NO wall-clock date: the outbox promises byte-determinism (same input + run_id
identical file), and the run's date is stamped at report time (§4 pt 3), identical file), and the run's date is stamped at report time (§4 pt 3),
never into the deterministic log. never into the deterministic log.
``client`` is the client the drill CONSTRUCTED (never called): the rig's SDK
build is read from it, the same seam rule the provenance stamp follows. A
drill driven by the scripted stand-in used no SDK and stamps null reading
the installed version from the environment would describe a rig that never
existed (§1).
""" """
return { return {
"run_id": run_id, "run_id": run_id,
"profile": profile, "profile": profile,
"bundle": bundle_name, "bundle": bundle_name,
"sdk_version": _client_sdk_version(client),
"models": { "models": {
role: resolve_model(contracts.model_map, role, profile=profile) role: resolve_model(contracts.model_map, role, profile=profile)
for role in (_PROPOSER_ROLE, _CHECKER_ROLE) for role in (_PROPOSER_ROLE, _CHECKER_ROLE)
@ -793,6 +801,7 @@ def main(
) )
run_config = build_dry_run_config( run_config = build_dry_run_config(
contracts, contracts,
client=client,
profile=_DEFAULT_PROFILE, profile=_DEFAULT_PROFILE,
bundle_name=args.bundle.name, bundle_name=args.bundle.name,
run_id=run_id, run_id=run_id,

View file

@ -9,6 +9,10 @@ says which build produced it. But a run driven by the scripted stand-in used no
SDK at all stamping the installed version there would attribute a build to a SDK at all stamping the installed version there would attribute a build to a
run that never touched it, which is exactly the fabrication §1 forbids. run that never touched it, which is exactly the fabrication §1 forbids.
The same rule governs the K8 drill's run-config: it records the build of the
client the drill CONSTRUCTED (and never called), so the rehearsed rig is
traceable without ever claiming a build the drill did not use.
Detach proof: make the stamp read ``importlib.metadata`` instead of the client Detach proof: make the stamp read ``importlib.metadata`` instead of the client
a scripted run claims an SDK build it never used red. a scripted run claims an SDK build it never used red.
@ -128,6 +132,70 @@ class TestTheStampStaysOptional:
) )
class TestTheDrillStampsTheSameWay:
"""LOAD-BEARING (§1, §11): the K8 drill's run-config obeys the same seam rule.
The run-config is the comparison protocol's §4 pt 3 record of what a live
run WOULD carry model ids, parameters, caps. Without the build id it
describes a rig whose SDK version is unknown, which is the one thing that
makes a reported USD figure traceable. It is read from the CLIENT the drill
constructed, exactly like the provenance stamp: the drill builds the client
before it stops, so the honest source is right there.
Detach point: stamp ``importlib.metadata`` (or drop the client argument and
hard-code a value) the scripted drill claims a build it never used RED.
"""
def _drill_runconfig(self, tmp_path: Path, *, sdk_version: str | None) -> dict[str, object]:
created: list[ScriptedClient] = []
def factory(contracts: Contracts, max_budget_usd_per_call: float) -> ModelClient:
# No replies: any model call would both record and raise, so the
# drill's zero-call promise still holds while this runs.
client = ScriptedClient(replies=[])
if sdk_version is not None:
client.sdk_version = sdk_version # type: ignore[attr-defined]
created.append(client)
return client
outbox = tmp_path / "outbox"
main(
[
"--bundle",
str(BUNDLE),
"--outbox",
str(outbox),
"--run-id",
"drill-sdk",
"--live-dry-run",
],
client_factory=factory,
)
(client,) = created
assert client.calls == [] # the drill still made no model call
payload: dict[str, object] = json.loads(
(outbox / "drill-sdk-runconfig.json").read_text(encoding="utf-8")
)
return payload
def test_a_drill_not_produced_by_the_sdk_stamps_no_build(self, tmp_path: Path) -> None:
payload = self._drill_runconfig(tmp_path, sdk_version=None)
assert payload["sdk_version"] is None
assert payload["sdk_version"] != importlib.metadata.version("claude-agent-sdk")
def test_the_drills_client_has_its_build_stamped_verbatim(self, tmp_path: Path) -> None:
payload = self._drill_runconfig(tmp_path, sdk_version=FIXTURE_BUILD)
assert payload["sdk_version"] == FIXTURE_BUILD
def test_the_stamp_does_not_disturb_the_rest_of_the_run_config(self, tmp_path: Path) -> None:
# The field is additive: §4 pt 3's existing content is unchanged, and the
# log still carries no wall-clock date (determinism).
payload = self._drill_runconfig(tmp_path, sdk_version=FIXTURE_BUILD)
assert payload["run_id"] == "drill-sdk"
assert payload["profile"] == "anthropic"
assert "date" not in payload
class TestTheFasitIsNotRetroStamped: class TestTheFasitIsNotRetroStamped:
"""LOAD-BEARING (§1): the historical record is never back-filled.""" """LOAD-BEARING (§1): the historical record is never back-filled."""