Kjell Tore Guttormsen
bf87776bb3
The advisory finding: provenance.py/artifacts.py stamped no SDK version, while the SDK's total_cost_usd is a client-side ESTIMATE computed against a price table frozen when the SDK was built. An untraceable estimate is a figure nobody can check later, so the run now records which build produced it. Provenance gains sdk_version: str | None. The value comes from the PRODUCING CLIENT — getattr(client, "sdk_version", None) — exactly as model and cost_usd already do, never from importlib.metadata at stamp time. That distinction is the seam: a run driven by the scripted stand-in used no SDK at all, and stamping the installed version there would attribute a build to a run that never touched it (§1). SdkModelClient reads the installed build once from package metadata (offline: no key, no network); every other client reports null. A blank string is refused by the schema — null is the one way to say "not produced by the SDK". Scope note: this traceability covers OUR run cost only. The savings the framework recommends are settled by the deterministic validator against the golden suite, and no SDK estimate touches them. Two seams, both detach-proven RED: - make the stamp read importlib.metadata instead of the client → a scripted run claims a build it never used → red - back-fill runs/s10/provenance.json → red That second guard is the point of the change as much as the first. runs/s10/ is the byte-frozen record of the ONE live run (2026-07-03), executed before this field existed; the suite reads it nowhere else, so nothing would have caught a retro-stamp. Adding a build id to it now would be a guess presented as provenance. It stays without one, and the README says why. run_s10.py is deliberately untouched (byte-frozen fasit script), and the field defaults to None, so every existing caller and artifact shape is unchanged. 603 passed · ruff clean · mypy strict clean · runs/s10/ byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
58 lines
3.7 KiB
Markdown
58 lines
3.7 KiB
Markdown
# Changelog
|
||
|
||
All notable changes to this project will be documented in this file.
|
||
|
||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||
|
||
## [Unreleased]
|
||
|
||
### Added
|
||
|
||
**The method, implemented (D7).** Sibling implementation of the portfolio-optimiser method
|
||
on the Claude Agent SDK, built from the shared frozen spec + golden suite alone — never by
|
||
reverse-engineering the MAF sibling.
|
||
|
||
- **Deterministic backbone** — the typed cost-IR, the mandatory blocking validator (frozen by
|
||
the shared golden suite, its only oracle), first-class provenance, and fail-fast startup
|
||
contracts including the role → model map.
|
||
- **Agentic loop** — bounded generation, maker–checker debate, the validator gate, and
|
||
informed refinement; the budget meter admits no unbounded loop anywhere, and carries an
|
||
optional pre-call run-total USD belt on top of the post-charge token/round caps.
|
||
- **Learning loop** — the OKF context seam (navigation, never chunk-stuffing), the ExpeL-style
|
||
experience fold, the async expert-verdict inbox, and the fail-closed promotion gate.
|
||
- **Ingest layer** — deterministic CSV (`file`) and SQLite (`sql`) connectors in front of the
|
||
loop, materializing OKF bundles the unchanged loop consumes; frozen by byte-identical golden
|
||
extractions. `http`/MCP is an extension point this repo does **not** build, and a manifest
|
||
naming it is rejected fail-fast.
|
||
- **Value layer** — the fail-closed savings ledger (dimension-free sum, no double-counting),
|
||
the hard/soft goal contract, the outbox output layer, the HITL pending/routing view, the
|
||
pre-run cost simulation over schema-validated pricing config, the SDK/API preflight, opt-in
|
||
notification sinks (webhook egress only behind an explicit per-run flag), and the per-run
|
||
value report (modelled → expert-corrected → realized, goal progress, quantified learning
|
||
effect, cost against value).
|
||
- **Operator CLI** — one collecting entrance (`run.py`) for a single project (`--bundle`) or a
|
||
portfolio (`--portfolio`, `--verdict-dir`), with `--goals` + `--ledger` stopping a run before
|
||
any model call when the target is already met, and `--value-report` projecting what the run
|
||
delivered. Standalone entrances for `valuereport`, `hitl`, `costsim` and `preflight`. Every
|
||
flag the README documents is checked against the actual `--help` output by a load-bearing
|
||
test.
|
||
- **Knowledge-base recipe** — the documented team process for building the OKF bundles the
|
||
framework reads (`docs/oppskrift-kunnskapsbase.md`), with an honest 1–2 week expectation.
|
||
- **Traceable run cost** — the provenance stamp records which SDK build produced the run,
|
||
read from the producing client rather than the environment, so the SDK's cost estimate can
|
||
be traced to the price table that computed it. A run not produced by the SDK reports `null`
|
||
instead of borrowing the installed version.
|
||
- **The programme's one live model run** (S10) — executed and validated at a documented
|
||
$0.127514, its four artifacts committed as fixed reference output under `runs/s10/`. That
|
||
record is never edited after the fact: it predates the `sdk_version` field and is left
|
||
without one rather than back-filled with a guess.
|
||
- **Load-bearing tests** — every seam is proven by a test that goes red when the seam is
|
||
detached; the whole suite runs offline, with no API key and no network.
|
||
|
||
### Notes
|
||
|
||
- Honesty rule (method spec §1): no artifact in this repo claims more than the implementation
|
||
does. Scripted stand-ins are labelled as such, unbuilt extension points are named as
|
||
unbuilt, and a figure the data does not carry is reported unmarked rather than back-filled.
|
||
- Licensed under the MIT License (see `LICENSE`).
|