feat(portfolio): stamp the producing SDK build in provenance (wiki-advisory F1) [skip-docs]

The advisory finding: provenance.py/artifacts.py stamped no SDK version, while
the SDK's total_cost_usd is a client-side ESTIMATE computed against a price
table frozen when the SDK was built. An untraceable estimate is a figure nobody
can check later, so the run now records which build produced it.

Provenance gains sdk_version: str | None. The value comes from the PRODUCING
CLIENT — getattr(client, "sdk_version", None) — exactly as model and cost_usd
already do, never from importlib.metadata at stamp time. That distinction is
the seam: a run driven by the scripted stand-in used no SDK at all, and
stamping the installed version there would attribute a build to a run that
never touched it (§1). SdkModelClient reads the installed build once from
package metadata (offline: no key, no network); every other client reports
null. A blank string is refused by the schema — null is the one way to say
"not produced by the SDK".

Scope note: this traceability covers OUR run cost only. The savings the
framework recommends are settled by the deterministic validator against the
golden suite, and no SDK estimate touches them.

Two seams, both detach-proven RED:
- make the stamp read importlib.metadata instead of the client → a scripted run
  claims a build it never used → red
- back-fill runs/s10/provenance.json → red

That second guard is the point of the change as much as the first. runs/s10/ is
the byte-frozen record of the ONE live run (2026-07-03), executed before this
field existed; the suite reads it nowhere else, so nothing would have caught a
retro-stamp. Adding a build id to it now would be a guess presented as
provenance. It stays without one, and the README says why.

run_s10.py is deliberately untouched (byte-frozen fasit script), and the field
defaults to None, so every existing caller and artifact shape is unchanged.

603 passed · ruff clean · mypy strict clean · runs/s10/ byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 06:57:30 +02:00
commit bf87776bb3
6 changed files with 193 additions and 3 deletions

View file

@ -6,6 +6,16 @@ At least one citation into the source documents; the producing ``model`` and
``validator_decision``, which mirrors the DETERMINISTIC VALIDATOR only stamped
from the validator's outcome BEFORE any checker override, so a checker-gated
proposal whose numbers passed is never mislabelled as validator-rejected (§9).
``sdk_version`` records WHICH SDK build produced the run, and comes from the
producing client never from the environment. The SDK's reported cost is a
client-side estimate against a price table frozen at SDK build time, so the
figure is only traceable if the run says which build computed it. A run driven
by anything other than the SDK client (the scripted stand-in, an injected
client) reports ``None``: attributing an installed build to a run that never
used it would be a fabricated claim (§1). Note that this reasoning covers OUR
run cost only the savings the framework recommends are settled by the
deterministic validator against the golden suite, never by an SDK estimate.
"""
from __future__ import annotations
@ -33,6 +43,9 @@ class Provenance(BaseModel):
role: str = Field(min_length=1)
validator_decision: Literal["validated", "rejected"]
tokens_used: int = Field(ge=0)
# None = not produced by the SDK client. A blank string is refused: it would
# read as "no SDK" while occupying the field, and null is the one way to say it.
sdk_version: str | None = Field(default=None, min_length=1)
def stamp_validator_decision(

View file

@ -159,6 +159,15 @@ def _client_cost_usd(client: ModelClient) -> float | None:
return None if cost is None else round(float(cost), 6)
def _client_sdk_version(client: ModelClient) -> str | None:
# The build is read from the PRODUCING CLIENT, never from the environment:
# a run driven by the scripted stand-in used no SDK at all, and stamping
# the installed version there would attribute a build to a run that never
# touched it (§1). Same seam rule as the cost and the model id above.
version = getattr(client, "sdk_version", None)
return None if version is None else str(version)
def execute_run(
client: ModelClient,
composed: ComposedRunContext,
@ -225,6 +234,7 @@ def execute_run(
role=_PROPOSER_ROLE,
validator_decision=result.validator_decision,
tokens_used=meter.tokens_used,
sdk_version=_client_sdk_version(client),
)
paths = persist_run_artifacts(
out_dir,

View file

@ -21,6 +21,7 @@ blocks + real model id) and a closing ``ResultMessage`` (provider-reported
from __future__ import annotations
import asyncio
import importlib.metadata
from typing import Any
from claude_agent_sdk import (
@ -79,6 +80,13 @@ class SdkModelClient:
``total_cost_usd`` accumulates the provider-reported cost across calls so
the run can log it (D6); ``last_model`` carries the REAL model id from the
latest reply for the §9 provenance stamp.
``sdk_version`` is the installed build, read once from package metadata
(offline: no key, no network). It exists so a run's provenance can say which
build produced it the reported cost is an estimate against a price table
frozen at that build, and an untraceable estimate is a figure nobody can
check later. Only THIS client carries the attribute, so a run driven by any
other client honestly reports no build at all (§1).
"""
def __init__(
@ -97,6 +105,7 @@ class SdkModelClient:
self._max_budget_usd_per_call = max_budget_usd_per_call
self.total_cost_usd = 0.0
self.last_model: str | None = None
self.sdk_version: str = importlib.metadata.version("claude-agent-sdk")
def complete(self, prompt: str, *, role: str) -> ModelReply:
model_id = resolve_model(self._model_map, role, profile=self._profile)