docs(release): describe what v0.1.0 actually is, not what it was 53 commits ago
STATE said the release text was already written and only needed dating. Measured:
CHANGELOG.md was last touched at 9e149c6, 53 commits back, so [Unreleased]
described the repo as of S3.1 — and claimed 512 passing tests where the suite
measures 755.
For a FIRST tag there is no predecessor, so the section is not "what changed" but
"what this version is". A description that stops at S3.1 does not under-report a
delta; it misrepresents the artefact being tagged, on a public mirror.
Audited the existing bullets for claims that had gone FALSE rather than merely
stale — the class a test-count catch is only a sample of. Four checked, all still
true at HEAD: numpy confined to semretrieval.py; value_report deliberately not
wired into costsim; the simulation module and knowledge-base recipe present; the
two run modes still two.
Added, grouped by seam rather than by commit: S4.0 cost-baseline anchoring, S2.7,
S3.2 per-candidate verdict keying, S3.3 wave concurrency, S3.4 global token cap,
S2.2/S2.4 ingest transports, --scripted-replies, --mandate, --mcp-config, the
per-approach outbox, and external-call provenance. New Fixed section for the
seven correctness fixes, with a lead-in saying plainly that none of them ever
shipped. Notes gained the deployer-owns-DPIA scope boundary and the pull-only
subtree rule.
The test-suite bullet now records TWO cases where "every seam is load-bearing"
did not hold until measured, not one: the S3.1 CLI-level gap, and four of five
BudgetExceeded raise sites free to report any `observed`.
No code change. Suite measured green at 755 passed / 4 skipped after the edit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVekyKcvT4ah6jnCT8R8fe
This commit is contained in:
parent
991131be3f
commit
520e74128e
1 changed files with 125 additions and 6 deletions
131
CHANGELOG.md
131
CHANGELOG.md
|
|
@ -5,7 +5,10 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
## [0.1.0] - 2026-08-06
|
||||
|
||||
First tagged release. There is no prior release, so the entries below describe what this version
|
||||
**is**, not what changed since a predecessor.
|
||||
|
||||
### Added
|
||||
- Deterministic backbone: mandatory blocking validator (solver + Monte Carlo against the shared
|
||||
|
|
@ -59,6 +62,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
mismatch; missing loads as `None`) — an authoring primitive with no caller in `src/`, offered to
|
||||
extenders like `write_verdict` and `promote_verdict`. numpy is confined to `semretrieval.py` and
|
||||
never enters `okf.py`, `retrieval.py` or `shared/`.
|
||||
- Cost-baseline anchoring of the deterministic gate (S4.0): `validate_proposal(..., baseline=...)`
|
||||
reconciles every `affected_item` against the project's real `CostBaseline` in a **stage 0, before
|
||||
the solver** — two independent rejections (unknown cost code; a real code whose
|
||||
`quantity`/`unit_cost` falls outside a configurable tolerance, default 5 %, of the baseline value).
|
||||
Before this, every stage reasoned only about numbers the proposal itself supplied, so an internally
|
||||
consistent hallucination cleared the whole gate. Validation, never repair: the proposal is
|
||||
rejected, never silently corrected to the baseline. The argument is **optional** (`None` reproduces
|
||||
the earlier behaviour) but both run paths set it — the road path always, the bundle path only when
|
||||
the bundle ships a `cost-baseline.json`, since a pre-amendment bundle is legitimately unanchored.
|
||||
A baseline that exists but is malformed raises on both loaders rather than reading as "no
|
||||
baseline". Method caps are looked up in an injectable `METHOD_CAPS` registry, so a second measure
|
||||
type is data rather than an edit to the validator.
|
||||
- Deterministic stage-2 bound and band enclosure in the validator (S2.7).
|
||||
- Per-candidate verdict keying (S3.2): `seed_store_from_bundle` reads each `type: verdict` file's own
|
||||
structural frontmatter (`affected_codes` / `measure_type` / `claimed_saving_nok`) instead of
|
||||
collapsing a multi-candidate bundle onto the single IR projection, where a verdict about candidate
|
||||
B scored a perfect structural match against candidate A's query. All three fields or none — a
|
||||
partial declaration raises rather than being merged with the bundle candidate, which would mint a
|
||||
key belonging to neither. Bundles that declare none fall back to the previous keying, so every
|
||||
pre-S3.2 seed is unchanged.
|
||||
- Portfolio concurrency (S3.3): wave-partitioned execution with a per-project snapshot and a
|
||||
deterministic merge barrier, pinned by a concurrent-equals-sequential contract test; `k < 1` is
|
||||
fail-fast. Failures are collected and the pass continues, surfaced as `RunFailure` slots rather
|
||||
than aborting the portfolio. Goal-stop is evaluated at wave boundaries, with the intra-wave
|
||||
semantics documented and pinned.
|
||||
- Global portfolio token cap (S3.4/F10): `PortfolioBudget` + `PortfolioMeter` form one ledger across
|
||||
a whole portfolio pass — and, seeded from a spend file, across passes — while the per-run
|
||||
`Budget`/`TokenMeter` is unchanged. Three enforcement points: a **startup refusal** when the
|
||||
remainder cannot fund a single run; **wave admission**, where an unfundable project is never
|
||||
started (a project that is merely aborted has already cost calls) and the pass stops structurally
|
||||
with the completed runs preserved; and a **pre-call guard** in `BudgetMiddleware`, so a call the
|
||||
remainder cannot pay for is refused rather than made. Because a whole wave is checked against the
|
||||
same pre-wave remainder, admission reserves each member's requirement. Resource exhaustion is
|
||||
reported in its own `budget_stop` field, never merged into `stop_reason` — a goal stop is success.
|
||||
The spend file is our own accounting state, so corrupt content raises instead of reading as zero.
|
||||
- Ingest transports (S2.2, S2.4): the MCP connector as a transport inside the http family, and a
|
||||
time-bounded default http transport in front of the pinned ingest library. A server on the ingest
|
||||
path must expose a **zero-argument tool** (the URL carries both coordinates), which is a
|
||||
deliberately separate seam from the in-run data-source demo server.
|
||||
- Azure/Foundry offline preflight config gate (`preflight.py`, S4.1).
|
||||
- Offline live-dry-run drill (`--live-dry-run`, S4.2): walks the whole path up to the eager
|
||||
client build and stops before the first model call — zero chat calls.
|
||||
|
|
@ -67,13 +109,90 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
`FileNotifier` (byte-deterministic JSONL), and `WebhookNotifier`, plus a fail-fast
|
||||
`build_notifier`; the webhook is the only egress point, fail-closed behind an explicit per-run
|
||||
`allow_egress` opt-in. Not auto-wired into `run.py`.
|
||||
- Offline whole-loop CLI door (`--scripted-replies`): drives the entire method over the caller's own
|
||||
data with no model budget, in **both** run modes. Resolved before the portfolio dispatch, because
|
||||
placing it after made `--portfolio --scripted-replies` silently drop the flag and attempt four real
|
||||
model calls (measured). Mutually exclusive with `--live-dry-run`: both are offline and they
|
||||
contradict, so the combination is refused rather than letting one quietly win.
|
||||
- Run mandate (`mandate.py`, `--mandate`): a domain expert commissions **which approaches** a run
|
||||
evaluates, the run announces what it will do before doing it, and it settles for that — every
|
||||
commissioned approach is evaluated and every one appears in a coverage report. A mandate is not a
|
||||
dimension (`dimension.admits` filters what may pass the scoping gate; a mandate directs what the
|
||||
run spends its attempts on) and not a goal (the numeric target keeps its single home in
|
||||
`GoalContract`/`--goals` and is merely restated in the announcement). Two construction-time
|
||||
refusals: an empty commission, and a duplicate approach id — including one claiming the reserved
|
||||
`own-proposal` — since the id keys each coverage row and two rows under one key collapse silently.
|
||||
Pure module: pydantic + stdlib only, D7-portable, guarded MAF-free.
|
||||
- MCP servers as in-run tools (`mcp_tools.py`, `--mcp-config`): concrete external servers become
|
||||
tools the agents can call **during** the debate. Opt-in and additive — with no config there are
|
||||
zero network calls and the tool list is unchanged. Three rules are load-bearing: an **allowlist is
|
||||
required** (an empty one would let the far end decide what the agents may call), **every server and
|
||||
every permitted tool is named in the announcement before the first call** (also without
|
||||
`--mandate` — no undeclared egress), and `--live-dry-run` opens nothing. A malformed config is
|
||||
refused rather than degraded to "no external services", which would make the announcement describe
|
||||
a run nobody configured. This is a separate seam from the ingest-path MCP transport above.
|
||||
- Per-approach outbox artefacts: a run commissioned to evaluate three approaches previously wrote one
|
||||
proposal artefact, so only the selected approach could ever receive a verdict and the others taught
|
||||
the learning loop nothing. Artefacts are now keyed `{run_id}-{approach_id}-*.json` and carry
|
||||
`approach_id` in the payload, with the join key `(run_id, approach_id)` — widening only the
|
||||
filename would not have worked, because the reader joins on the `run_id` field read from file
|
||||
content. Two properties make them genuinely judgeable: `verdict_id` is minted per approach (so one
|
||||
delivered verdict cannot clear all three from the queue), and `provenance.validator_decision`
|
||||
follows its own approach rather than the run's. `verdicts.verdict_key` is the single public
|
||||
verdict-key rule.
|
||||
- External-call provenance: the egress declaration says what a run **may** contact;
|
||||
`ToolCallRecorder` records what it **did**, landing on `ProvenanceStamp.external_calls` read after
|
||||
the debate. It only observes — `call_next` is always awaited, so a trace can never alter the run it
|
||||
traces. Only configured tools are recorded; logging in-process functions would turn the record into
|
||||
a false egress claim, and an empty list is therefore a positive statement that nothing outside the
|
||||
process was contacted. Attribution comes from our own config because MAF reports the bare tool name
|
||||
with no server prefix (measured, not assumed), so a name allowed by two servers is recorded
|
||||
unattributed rather than credited to the first match. Stated honesty limit: this is the call and
|
||||
its source — not evidence that the service's answer reached the proposal.
|
||||
- `docs/knowledge-base-recipe.md` (S5.3, D-H item 1): the documented team process (technical +
|
||||
domain expert) for building a knowledge base, with the honest 1–2 week expectation.
|
||||
- Test suite: 512 passing tests (4 skips are live-provider-only). Every wired seam is covered by a
|
||||
load-bearing test that goes red when the seam is detached — including, since the S3.1 review
|
||||
remediation, the `--semantic-retrieval` wiring at `main()` level. That claim did not hold for
|
||||
S3.1 as first shipped: the flag was covered only below the CLI, so hardcoding it off left the
|
||||
suite green. Each remediated seam now carries a recorded detach point, verified by mutation.
|
||||
- Test suite: 755 passing tests (4 skips are live-provider-only). Every wired seam is covered by a
|
||||
load-bearing test that goes red when the seam is detached, and each carries a recorded detach point
|
||||
verified by mutation. That property is maintained by measurement, not by assumption, and this
|
||||
release contains two recorded cases where it did not hold until it was checked: the S3.1
|
||||
`--semantic-retrieval` flag was covered only below the CLI, so hardcoding it off at `main()` level
|
||||
left the suite green; and four of the five `BudgetExceeded` raise sites could report any value in
|
||||
the `observed` field without a single test noticing. Both are now pinned.
|
||||
|
||||
### Fixed
|
||||
Defects found and closed before this first tag. Nothing here was ever released; they are recorded
|
||||
because each one changed an invariant an adopter depends on.
|
||||
|
||||
- Money quantisation happens in one order, from one source: `ledger.to_ore` is the framework's single
|
||||
NOK→øre conversion, applied **per amount**, after which integers are summed. Previously the goal
|
||||
baseline summed floats and quantised once while the ledger summed per-candidate integers, and the
|
||||
two orders met in exactly one place — the check that decides whether a portfolio pass stops early
|
||||
on a percentage goal. Measured divergence: three lines of `60000.005` NOK are `18000003` øre
|
||||
quantised first but `18000001` summed first, enough to flip a goal. Both call sites were fixed; a
|
||||
fix to only one survived the whole suite (measured).
|
||||
- The MCP stdio ingest transport now runs against a real server, and its error contract is repaired:
|
||||
`stdio_client` and `ClientSession` are each a task group, and anyio wraps everything leaving one in
|
||||
an exception group, so the module's own `IngestError` reached callers as an exception group and
|
||||
never as the type the ingest door switches on. The owned error is unwrapped and re-raised; anything
|
||||
else is re-raised untouched. No canned-tool test could have caught this — the defect only appears
|
||||
once the code is actually run.
|
||||
- The MCP timeout composes with anyio's own cancel scope (`anyio.fail_after` nested inside both task
|
||||
groups) instead of `asyncio.wait_for` from outside, which cancelled the structure anyio owns and
|
||||
produced a `BrokenResourceError` inside an exception group rather than a timeout. The translation
|
||||
is narrowed by `CancelScope.cancelled_caught`, so only the scope that hit its own deadline earns
|
||||
the timeout code — an unconditional `except TimeoutError` would mislabel any `TimeoutError`, since
|
||||
the builtin is also `socket.timeout` and `asyncio.TimeoutError`. `anyio` is now a declared direct
|
||||
dependency rather than a transitive one.
|
||||
- Frontmatter scalars are unquoted by exactly one rule (`okf.unquote_scalar`).
|
||||
- A non-finite embedding is refused rather than scored.
|
||||
- `--embedder-config` is refused when `--semantic-retrieval` is absent, rather than silently dropped.
|
||||
- The portfolio CLI no longer swallows its offline door or its failures.
|
||||
- Constants cited in the live documentation are gated against the code, so a doc and the value it
|
||||
quotes cannot drift apart unnoticed.
|
||||
|
||||
### Notes
|
||||
- Licensed under the MIT License (see `LICENSE`).
|
||||
- Scope boundary: this is a technical framework. The deployer owns DPIA, risk assessment and
|
||||
processing purpose; the framework ships only the technical preconditions (local-only operation,
|
||||
provenance, no silent egress).
|
||||
- The `shared/` directory is a git subtree of `portfolio-optimiser-commons` and is pull-only.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue