feat(portfolio): stamp the producing SDK build in provenance (wiki-advisory F1) [skip-docs]

The advisory finding: provenance.py/artifacts.py stamped no SDK version, while
the SDK's total_cost_usd is a client-side ESTIMATE computed against a price
table frozen when the SDK was built. An untraceable estimate is a figure nobody
can check later, so the run now records which build produced it.

Provenance gains sdk_version: str | None. The value comes from the PRODUCING
CLIENT — getattr(client, "sdk_version", None) — exactly as model and cost_usd
already do, never from importlib.metadata at stamp time. That distinction is
the seam: a run driven by the scripted stand-in used no SDK at all, and
stamping the installed version there would attribute a build to a run that
never touched it (§1). SdkModelClient reads the installed build once from
package metadata (offline: no key, no network); every other client reports
null. A blank string is refused by the schema — null is the one way to say
"not produced by the SDK".

Scope note: this traceability covers OUR run cost only. The savings the
framework recommends are settled by the deterministic validator against the
golden suite, and no SDK estimate touches them.

Two seams, both detach-proven RED:
- make the stamp read importlib.metadata instead of the client → a scripted run
  claims a build it never used → red
- back-fill runs/s10/provenance.json → red

That second guard is the point of the change as much as the first. runs/s10/ is
the byte-frozen record of the ONE live run (2026-07-03), executed before this
field existed; the suite reads it nowhere else, so nothing would have caught a
retro-stamp. Adding a build id to it now would be a guess presented as
provenance. It stays without one, and the README says why.

run_s10.py is deliberately untouched (byte-frozen fasit script), and the field
defaults to None, so every existing caller and artifact shape is unchanged.

603 passed · ruff clean · mypy strict clean · runs/s10/ byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 06:57:30 +02:00
commit bf87776bb3
6 changed files with 193 additions and 3 deletions

View file

@ -13,7 +13,7 @@ human-in-the-loop, and the system learns from the verdicts.
> **Status:** the D7 build (S5S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, the value layer, and the ingest connectors
> are wired seam by seam, each proven by load-bearing tests (597 at the time of writing, all
> are wired seam by seam, each proven by load-bearing tests (603 at the time of writing, all
> running offline without an API key — `uv run pytest` is the source of truth). The
> programme's single budgeted **live model run has been executed and validated** — its
> artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
@ -44,7 +44,12 @@ offline. Module by module:
- `validator.py` — the deterministic validator; blocking, and frozen by the shared golden
suite (§7.2), which is the only fasit it answers to.
- `provenance.py` — the first-class provenance stamp (§9); authoritative data, not
after-the-fact logging.
after-the-fact logging. It also records **which SDK build produced the run**, taken from
the producing client and never from the environment: the SDK's reported cost is a
client-side estimate against a price table frozen at that build, so the figure is only
checkable later if the run says which build computed it. A run driven by anything else
(the scripted stand-in, an injected client) reports `null` rather than borrowing the
installed version it never used.
- `contracts.py` — fail-fast startup contracts (§10): stop criteria and budget caps are
required at startup, and the model map (`data/model_map.json`, role → Claude model id
per backend profile) is validated before anything runs.
@ -273,6 +278,9 @@ the run entrance produces the report on both outcomes while leaving the run's ve
before a single model call — red the moment the goal check is unwired — the portfolio config's
projects genuinely run through the CLI, and this README's documented flags are checked against
the actual `--help` output),
`test_provenance_sdk_version_loadbearing.py` (a run not produced by the SDK stamps no build
rather than borrowing the installed one — red the moment the stamp reads the environment
instead of the producing client — and the committed S10 record stays un-back-filled),
and `test_sdk_isolation.py` (local config cannot capture the checker).
## The ingest layer — CSV and SQL, in front of the loop
@ -321,6 +329,10 @@ whole programme), executed 2026-07-03 against the micro bundle
bundle's p10p90 band of 68.5k121k — and validates.
- All four artifacts are committed as fixed reference output in [`runs/s10/`](runs/s10/):
`proposal.json`, `provenance.json` (with §9 citations), `run_result.json`, `usage.json`.
They are the record of that run as it happened and are never edited afterwards — the
provenance stamp's later `sdk_version` field is absent there because the run predates it,
and back-filling a build id would be a guess presented as provenance. A load-bearing test
keeps that record frozen.
Honesty rule (§1): everything else in the repo is deterministic and offline; nothing here
claims more live behaviour than that one documented run.