docs(release): describe what v0.1.0 actually is, not what it was 53 commits ago
STATE said the release text was already written and only needed dating. Measured:
CHANGELOG.md was last touched at 9e149c6, 53 commits back, so [Unreleased]
described the repo as of S3.1 — and claimed 512 passing tests where the suite
measures 755.
For a FIRST tag there is no predecessor, so the section is not "what changed" but
"what this version is". A description that stops at S3.1 does not under-report a
delta; it misrepresents the artefact being tagged, on a public mirror.
Audited the existing bullets for claims that had gone FALSE rather than merely
stale — the class a test-count catch is only a sample of. Four checked, all still
true at HEAD: numpy confined to semretrieval.py; value_report deliberately not
wired into costsim; the simulation module and knowledge-base recipe present; the
two run modes still two.
Added, grouped by seam rather than by commit: S4.0 cost-baseline anchoring, S2.7,
S3.2 per-candidate verdict keying, S3.3 wave concurrency, S3.4 global token cap,
S2.2/S2.4 ingest transports, --scripted-replies, --mandate, --mcp-config, the
per-approach outbox, and external-call provenance. New Fixed section for the
seven correctness fixes, with a lead-in saying plainly that none of them ever
shipped. Notes gained the deployer-owns-DPIA scope boundary and the pull-only
subtree rule.
The test-suite bullet now records TWO cases where "every seam is load-bearing"
did not hold until measured, not one: the S3.1 CLI-level gap, and four of five
BudgetExceeded raise sites free to report any `observed`.
No code change. Suite measured green at 755 passed / 4 skipped after the edit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVekyKcvT4ah6jnCT8R8fe
This commit is contained in:
parent
991131be3f
commit
520e74128e
1 changed files with 125 additions and 6 deletions
131
CHANGELOG.md
131
CHANGELOG.md
|
|
@ -5,7 +5,10 @@ All notable changes to this project will be documented in this file.
|
||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
## [Unreleased]
|
## [0.1.0] - 2026-08-06
|
||||||
|
|
||||||
|
First tagged release. There is no prior release, so the entries below describe what this version
|
||||||
|
**is**, not what changed since a predecessor.
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- Deterministic backbone: mandatory blocking validator (solver + Monte Carlo against the shared
|
- Deterministic backbone: mandatory blocking validator (solver + Monte Carlo against the shared
|
||||||
|
|
@ -59,6 +62,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||||
mismatch; missing loads as `None`) — an authoring primitive with no caller in `src/`, offered to
|
mismatch; missing loads as `None`) — an authoring primitive with no caller in `src/`, offered to
|
||||||
extenders like `write_verdict` and `promote_verdict`. numpy is confined to `semretrieval.py` and
|
extenders like `write_verdict` and `promote_verdict`. numpy is confined to `semretrieval.py` and
|
||||||
never enters `okf.py`, `retrieval.py` or `shared/`.
|
never enters `okf.py`, `retrieval.py` or `shared/`.
|
||||||
|
- Cost-baseline anchoring of the deterministic gate (S4.0): `validate_proposal(..., baseline=...)`
|
||||||
|
reconciles every `affected_item` against the project's real `CostBaseline` in a **stage 0, before
|
||||||
|
the solver** — two independent rejections (unknown cost code; a real code whose
|
||||||
|
`quantity`/`unit_cost` falls outside a configurable tolerance, default 5 %, of the baseline value).
|
||||||
|
Before this, every stage reasoned only about numbers the proposal itself supplied, so an internally
|
||||||
|
consistent hallucination cleared the whole gate. Validation, never repair: the proposal is
|
||||||
|
rejected, never silently corrected to the baseline. The argument is **optional** (`None` reproduces
|
||||||
|
the earlier behaviour) but both run paths set it — the road path always, the bundle path only when
|
||||||
|
the bundle ships a `cost-baseline.json`, since a pre-amendment bundle is legitimately unanchored.
|
||||||
|
A baseline that exists but is malformed raises on both loaders rather than reading as "no
|
||||||
|
baseline". Method caps are looked up in an injectable `METHOD_CAPS` registry, so a second measure
|
||||||
|
type is data rather than an edit to the validator.
|
||||||
|
- Deterministic stage-2 bound and band enclosure in the validator (S2.7).
|
||||||
|
- Per-candidate verdict keying (S3.2): `seed_store_from_bundle` reads each `type: verdict` file's own
|
||||||
|
structural frontmatter (`affected_codes` / `measure_type` / `claimed_saving_nok`) instead of
|
||||||
|
collapsing a multi-candidate bundle onto the single IR projection, where a verdict about candidate
|
||||||
|
B scored a perfect structural match against candidate A's query. All three fields or none — a
|
||||||
|
partial declaration raises rather than being merged with the bundle candidate, which would mint a
|
||||||
|
key belonging to neither. Bundles that declare none fall back to the previous keying, so every
|
||||||
|
pre-S3.2 seed is unchanged.
|
||||||
|
- Portfolio concurrency (S3.3): wave-partitioned execution with a per-project snapshot and a
|
||||||
|
deterministic merge barrier, pinned by a concurrent-equals-sequential contract test; `k < 1` is
|
||||||
|
fail-fast. Failures are collected and the pass continues, surfaced as `RunFailure` slots rather
|
||||||
|
than aborting the portfolio. Goal-stop is evaluated at wave boundaries, with the intra-wave
|
||||||
|
semantics documented and pinned.
|
||||||
|
- Global portfolio token cap (S3.4/F10): `PortfolioBudget` + `PortfolioMeter` form one ledger across
|
||||||
|
a whole portfolio pass — and, seeded from a spend file, across passes — while the per-run
|
||||||
|
`Budget`/`TokenMeter` is unchanged. Three enforcement points: a **startup refusal** when the
|
||||||
|
remainder cannot fund a single run; **wave admission**, where an unfundable project is never
|
||||||
|
started (a project that is merely aborted has already cost calls) and the pass stops structurally
|
||||||
|
with the completed runs preserved; and a **pre-call guard** in `BudgetMiddleware`, so a call the
|
||||||
|
remainder cannot pay for is refused rather than made. Because a whole wave is checked against the
|
||||||
|
same pre-wave remainder, admission reserves each member's requirement. Resource exhaustion is
|
||||||
|
reported in its own `budget_stop` field, never merged into `stop_reason` — a goal stop is success.
|
||||||
|
The spend file is our own accounting state, so corrupt content raises instead of reading as zero.
|
||||||
|
- Ingest transports (S2.2, S2.4): the MCP connector as a transport inside the http family, and a
|
||||||
|
time-bounded default http transport in front of the pinned ingest library. A server on the ingest
|
||||||
|
path must expose a **zero-argument tool** (the URL carries both coordinates), which is a
|
||||||
|
deliberately separate seam from the in-run data-source demo server.
|
||||||
- Azure/Foundry offline preflight config gate (`preflight.py`, S4.1).
|
- Azure/Foundry offline preflight config gate (`preflight.py`, S4.1).
|
||||||
- Offline live-dry-run drill (`--live-dry-run`, S4.2): walks the whole path up to the eager
|
- Offline live-dry-run drill (`--live-dry-run`, S4.2): walks the whole path up to the eager
|
||||||
client build and stops before the first model call — zero chat calls.
|
client build and stops before the first model call — zero chat calls.
|
||||||
|
|
@ -67,13 +109,90 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||||
`FileNotifier` (byte-deterministic JSONL), and `WebhookNotifier`, plus a fail-fast
|
`FileNotifier` (byte-deterministic JSONL), and `WebhookNotifier`, plus a fail-fast
|
||||||
`build_notifier`; the webhook is the only egress point, fail-closed behind an explicit per-run
|
`build_notifier`; the webhook is the only egress point, fail-closed behind an explicit per-run
|
||||||
`allow_egress` opt-in. Not auto-wired into `run.py`.
|
`allow_egress` opt-in. Not auto-wired into `run.py`.
|
||||||
|
- Offline whole-loop CLI door (`--scripted-replies`): drives the entire method over the caller's own
|
||||||
|
data with no model budget, in **both** run modes. Resolved before the portfolio dispatch, because
|
||||||
|
placing it after made `--portfolio --scripted-replies` silently drop the flag and attempt four real
|
||||||
|
model calls (measured). Mutually exclusive with `--live-dry-run`: both are offline and they
|
||||||
|
contradict, so the combination is refused rather than letting one quietly win.
|
||||||
|
- Run mandate (`mandate.py`, `--mandate`): a domain expert commissions **which approaches** a run
|
||||||
|
evaluates, the run announces what it will do before doing it, and it settles for that — every
|
||||||
|
commissioned approach is evaluated and every one appears in a coverage report. A mandate is not a
|
||||||
|
dimension (`dimension.admits` filters what may pass the scoping gate; a mandate directs what the
|
||||||
|
run spends its attempts on) and not a goal (the numeric target keeps its single home in
|
||||||
|
`GoalContract`/`--goals` and is merely restated in the announcement). Two construction-time
|
||||||
|
refusals: an empty commission, and a duplicate approach id — including one claiming the reserved
|
||||||
|
`own-proposal` — since the id keys each coverage row and two rows under one key collapse silently.
|
||||||
|
Pure module: pydantic + stdlib only, D7-portable, guarded MAF-free.
|
||||||
|
- MCP servers as in-run tools (`mcp_tools.py`, `--mcp-config`): concrete external servers become
|
||||||
|
tools the agents can call **during** the debate. Opt-in and additive — with no config there are
|
||||||
|
zero network calls and the tool list is unchanged. Three rules are load-bearing: an **allowlist is
|
||||||
|
required** (an empty one would let the far end decide what the agents may call), **every server and
|
||||||
|
every permitted tool is named in the announcement before the first call** (also without
|
||||||
|
`--mandate` — no undeclared egress), and `--live-dry-run` opens nothing. A malformed config is
|
||||||
|
refused rather than degraded to "no external services", which would make the announcement describe
|
||||||
|
a run nobody configured. This is a separate seam from the ingest-path MCP transport above.
|
||||||
|
- Per-approach outbox artefacts: a run commissioned to evaluate three approaches previously wrote one
|
||||||
|
proposal artefact, so only the selected approach could ever receive a verdict and the others taught
|
||||||
|
the learning loop nothing. Artefacts are now keyed `{run_id}-{approach_id}-*.json` and carry
|
||||||
|
`approach_id` in the payload, with the join key `(run_id, approach_id)` — widening only the
|
||||||
|
filename would not have worked, because the reader joins on the `run_id` field read from file
|
||||||
|
content. Two properties make them genuinely judgeable: `verdict_id` is minted per approach (so one
|
||||||
|
delivered verdict cannot clear all three from the queue), and `provenance.validator_decision`
|
||||||
|
follows its own approach rather than the run's. `verdicts.verdict_key` is the single public
|
||||||
|
verdict-key rule.
|
||||||
|
- External-call provenance: the egress declaration says what a run **may** contact;
|
||||||
|
`ToolCallRecorder` records what it **did**, landing on `ProvenanceStamp.external_calls` read after
|
||||||
|
the debate. It only observes — `call_next` is always awaited, so a trace can never alter the run it
|
||||||
|
traces. Only configured tools are recorded; logging in-process functions would turn the record into
|
||||||
|
a false egress claim, and an empty list is therefore a positive statement that nothing outside the
|
||||||
|
process was contacted. Attribution comes from our own config because MAF reports the bare tool name
|
||||||
|
with no server prefix (measured, not assumed), so a name allowed by two servers is recorded
|
||||||
|
unattributed rather than credited to the first match. Stated honesty limit: this is the call and
|
||||||
|
its source — not evidence that the service's answer reached the proposal.
|
||||||
- `docs/knowledge-base-recipe.md` (S5.3, D-H item 1): the documented team process (technical +
|
- `docs/knowledge-base-recipe.md` (S5.3, D-H item 1): the documented team process (technical +
|
||||||
domain expert) for building a knowledge base, with the honest 1–2 week expectation.
|
domain expert) for building a knowledge base, with the honest 1–2 week expectation.
|
||||||
- Test suite: 512 passing tests (4 skips are live-provider-only). Every wired seam is covered by a
|
- Test suite: 755 passing tests (4 skips are live-provider-only). Every wired seam is covered by a
|
||||||
load-bearing test that goes red when the seam is detached — including, since the S3.1 review
|
load-bearing test that goes red when the seam is detached, and each carries a recorded detach point
|
||||||
remediation, the `--semantic-retrieval` wiring at `main()` level. That claim did not hold for
|
verified by mutation. That property is maintained by measurement, not by assumption, and this
|
||||||
S3.1 as first shipped: the flag was covered only below the CLI, so hardcoding it off left the
|
release contains two recorded cases where it did not hold until it was checked: the S3.1
|
||||||
suite green. Each remediated seam now carries a recorded detach point, verified by mutation.
|
`--semantic-retrieval` flag was covered only below the CLI, so hardcoding it off at `main()` level
|
||||||
|
left the suite green; and four of the five `BudgetExceeded` raise sites could report any value in
|
||||||
|
the `observed` field without a single test noticing. Both are now pinned.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
Defects found and closed before this first tag. Nothing here was ever released; they are recorded
|
||||||
|
because each one changed an invariant an adopter depends on.
|
||||||
|
|
||||||
|
- Money quantisation happens in one order, from one source: `ledger.to_ore` is the framework's single
|
||||||
|
NOK→øre conversion, applied **per amount**, after which integers are summed. Previously the goal
|
||||||
|
baseline summed floats and quantised once while the ledger summed per-candidate integers, and the
|
||||||
|
two orders met in exactly one place — the check that decides whether a portfolio pass stops early
|
||||||
|
on a percentage goal. Measured divergence: three lines of `60000.005` NOK are `18000003` øre
|
||||||
|
quantised first but `18000001` summed first, enough to flip a goal. Both call sites were fixed; a
|
||||||
|
fix to only one survived the whole suite (measured).
|
||||||
|
- The MCP stdio ingest transport now runs against a real server, and its error contract is repaired:
|
||||||
|
`stdio_client` and `ClientSession` are each a task group, and anyio wraps everything leaving one in
|
||||||
|
an exception group, so the module's own `IngestError` reached callers as an exception group and
|
||||||
|
never as the type the ingest door switches on. The owned error is unwrapped and re-raised; anything
|
||||||
|
else is re-raised untouched. No canned-tool test could have caught this — the defect only appears
|
||||||
|
once the code is actually run.
|
||||||
|
- The MCP timeout composes with anyio's own cancel scope (`anyio.fail_after` nested inside both task
|
||||||
|
groups) instead of `asyncio.wait_for` from outside, which cancelled the structure anyio owns and
|
||||||
|
produced a `BrokenResourceError` inside an exception group rather than a timeout. The translation
|
||||||
|
is narrowed by `CancelScope.cancelled_caught`, so only the scope that hit its own deadline earns
|
||||||
|
the timeout code — an unconditional `except TimeoutError` would mislabel any `TimeoutError`, since
|
||||||
|
the builtin is also `socket.timeout` and `asyncio.TimeoutError`. `anyio` is now a declared direct
|
||||||
|
dependency rather than a transitive one.
|
||||||
|
- Frontmatter scalars are unquoted by exactly one rule (`okf.unquote_scalar`).
|
||||||
|
- A non-finite embedding is refused rather than scored.
|
||||||
|
- `--embedder-config` is refused when `--semantic-retrieval` is absent, rather than silently dropped.
|
||||||
|
- The portfolio CLI no longer swallows its offline door or its failures.
|
||||||
|
- Constants cited in the live documentation are gated against the code, so a doc and the value it
|
||||||
|
quotes cannot drift apart unnoticed.
|
||||||
|
|
||||||
### Notes
|
### Notes
|
||||||
- Licensed under the MIT License (see `LICENSE`).
|
- Licensed under the MIT License (see `LICENSE`).
|
||||||
|
- Scope boundary: this is a technical framework. The deployer owns DPIA, risk assessment and
|
||||||
|
processing purpose; the framework ships only the technical preconditions (local-only operation,
|
||||||
|
provenance, no silent egress).
|
||||||
|
- The `shared/` directory is a git subtree of `portfolio-optimiser-commons` and is pull-only.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue