feat(run): C2.0 — shippable step-7 run entrance + K2.9 seam bindings (closes C-N2, R-10, K2.9)
- run.py: compose_run_context (§5: merge inbox -> seed -> fold, read-only on the inbox) + execute_run (§8 meter, artifacts persisted on BOTH outcomes, structured exit 3 on budget stop) + thin CLI (python -m ..run). The model client is injected; only default_client_factory constructs the SDK client (wired, never executed by the suite). The navigated docs dir comes from the validated startup contract (resolves review OBS-2 on the shippable path; run_s10.py stays byte-frozen fasit -> won't-fix there). - test_run_entrance_loadbearing.py: inbox verdict reaches the composed context (detach-proven: merge dropped -> red), empty/missing-inbox controls, read-only inbox byte-proof, R-10 budget-stop binding via the NEW entrance (detach-proven: stop persistence dropped -> red), happy path through the CLI with the inbox signal surviving the chain, SDK-wiring test. - test_ingest_adoption.py (K2.9): the two library guarantees the consumer relies on, bound through the seam — empty CSV -> typed SourceError with NO partial bundle on disk; non-SELECT SQL -> SourceError 'returned no columns' (behavior verified empirically against pin dae0bd1a before binding). - README: inbox section now points at the shippable entrance; run.py added to the run layer; stale test count 265 -> 395. 386 -> 395 tests, full gate green (pytest, ruff check+format, mypy strict); goldens unchanged; runs/s10 and run_s10.py untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
5732d13369
commit
3587854074
4 changed files with 448 additions and 3 deletions
12
README.md
12
README.md
|
|
@ -13,7 +13,7 @@ human-in-the-loop, and the system learns from the verdicts.
|
||||||
> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
|
> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
|
||||||
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
|
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
|
||||||
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
|
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
|
||||||
> seam, each proven by load-bearing tests (265 tests, all running offline without an API
|
> seam, each proven by load-bearing tests (395 tests, all running offline without an API
|
||||||
> key). The programme's single budgeted **live model run has been executed and validated** —
|
> key). The programme's single budgeted **live model run has been executed and validated** —
|
||||||
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
|
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
|
||||||
|
|
||||||
|
|
@ -65,7 +65,8 @@ offline. Module by module:
|
||||||
**Learning loop** (§3 steps 7–8, §4–§6)
|
**Learning loop** (§3 steps 7–8, §4–§6)
|
||||||
- `inbox.py` — the async verdict-file contract: an expert drops a plain-JSON verdict into
|
- `inbox.py` — the async verdict-file contract: an expert drops a plain-JSON verdict into
|
||||||
an inbox folder after a run; a later run ingests it tolerantly and merges it before the
|
an inbox folder after a run; a later run ingests it tolerantly and merges it before the
|
||||||
fold.
|
fold. The shippable entrance for that later run is `run.py`:
|
||||||
|
`uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir>`.
|
||||||
- `promotion.py` — the promotion gate, **fail-closed**: only an approved verdict is
|
- `promotion.py` — the promotion gate, **fail-closed**: only an approved verdict is
|
||||||
lifted into the OKF context layer; anything else raises and writes nothing.
|
lifted into the OKF context layer; anything else raises and writes nothing.
|
||||||
- `persona.py` — the expert-reviewer persona sourced from the shared artifact in
|
- `persona.py` — the expert-reviewer persona sourced from the shared artifact in
|
||||||
|
|
@ -77,6 +78,11 @@ offline. Module by module:
|
||||||
(`setting_sources=[]`) so no user/project config can leak into a run.
|
(`setting_sources=[]`) so no user/project config can leak into a run.
|
||||||
- `artifacts.py` — §9 citations plus deterministic run-artifact persistence, including on
|
- `artifacts.py` — §9 citations plus deterministic run-artifact persistence, including on
|
||||||
structured stops (a budget stop still leaves artifacts behind).
|
structured stops (a budget stop still leaves artifacts behind).
|
||||||
|
- `run.py` — the generic run entrance: composes merge-inbox → seed → fold (§5) and drives
|
||||||
|
the loop under the budget meter, persisting artifacts on both outcomes — a structured
|
||||||
|
budget stop included. The model client is injected, so the offline suite proves the
|
||||||
|
same orchestration with a scripted client; only the CLI's default constructs the SDK
|
||||||
|
client.
|
||||||
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
|
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
|
||||||
|
|
||||||
### Load-bearing tests (§11)
|
### Load-bearing tests (§11)
|
||||||
|
|
@ -150,7 +156,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv sync # install dependencies
|
uv sync # install dependencies
|
||||||
uv run pytest # 265 tests — run without any API key and without network
|
uv run pytest # 395 tests — run without any API key and without network
|
||||||
uv run ruff check . && uv run ruff format --check .
|
uv run ruff check . && uv run ruff format --check .
|
||||||
uv run mypy src # strict
|
uv run mypy src # strict
|
||||||
```
|
```
|
||||||
|
|
|
||||||
222
src/portfolio_optimiser_claude/run.py
Normal file
222
src/portfolio_optimiser_claude/run.py
Normal file
|
|
@ -0,0 +1,222 @@
|
||||||
|
"""The shippable run entrance (C2.0): merge inbox → seed → fold → run (§3, §5).
|
||||||
|
|
||||||
|
Where ``run_s10.py`` is the byte-frozen fasit of the programme's ONE live run
|
||||||
|
(never imported by the suite), this module is the generic, deliverable
|
||||||
|
entrance the README's inbox claim points at. The composition
|
||||||
|
(``compose_run_context``) is pure config/file logic and offline-testable: it
|
||||||
|
ingests the inbox READ-only (role split §3 Step 7), seeds from the bundle, and
|
||||||
|
folds the retrieved verdicts into the generation context (§3 Step 1). The
|
||||||
|
orchestration (``execute_run``) drives the loop under the §8 meter and
|
||||||
|
persists artifacts on BOTH outcomes — a budget stop is a run outcome, not an
|
||||||
|
absence of one. The model client is injected: the real SDK client is
|
||||||
|
constructed only by ``default_client_factory`` on the CLI path (wired, never
|
||||||
|
executed by the suite — honesty rule §1); the navigated docs dir comes from
|
||||||
|
the validated startup contract, never straight from the raw argument (§10).
|
||||||
|
|
||||||
|
Run: uv run python -m portfolio_optimiser_claude.run --bundle <dir> [--inbox <dir>]
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
from portfolio_optimiser_claude.artifacts import (
|
||||||
|
build_citations,
|
||||||
|
persist_run_artifacts,
|
||||||
|
persist_stop_artifacts,
|
||||||
|
)
|
||||||
|
from portfolio_optimiser_claude.budget import BudgetExceeded, BudgetMeter
|
||||||
|
from portfolio_optimiser_claude.contracts import Contracts, load_contracts
|
||||||
|
from portfolio_optimiser_claude.experience import (
|
||||||
|
CandidateFeatures,
|
||||||
|
VerdictStore,
|
||||||
|
fold_experience,
|
||||||
|
seed_store_from_bundle,
|
||||||
|
)
|
||||||
|
from portfolio_optimiser_claude.inbox import merge_inbox_into_store
|
||||||
|
from portfolio_optimiser_claude.ir import SavingsProposal, load_validator_input
|
||||||
|
from portfolio_optimiser_claude.okf import bundle_context, navigate_bundle
|
||||||
|
from portfolio_optimiser_claude.provenance import Citation, Provenance
|
||||||
|
from portfolio_optimiser_claude.loop import ModelClient, run_project
|
||||||
|
from portfolio_optimiser_claude.validator import Rejection
|
||||||
|
|
||||||
|
_PROPOSER_ROLE = "proposer"
|
||||||
|
|
||||||
|
# The injected client seam of the entrance: (contracts, max_budget_usd_per_call).
|
||||||
|
ClientFactory = Callable[[Contracts, float], ModelClient]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ComposedRunContext:
|
||||||
|
"""The §5 sequence's output: the folded context + what fed it (§9-traceable)."""
|
||||||
|
|
||||||
|
context: str
|
||||||
|
citations: list[Citation]
|
||||||
|
ir_projection: SavingsProposal
|
||||||
|
inbox_merged: int
|
||||||
|
seeded: int
|
||||||
|
|
||||||
|
|
||||||
|
def compose_run_context(
|
||||||
|
bundle_dir: Path, inbox_dir: Path | None = None, *, k: int
|
||||||
|
) -> ComposedRunContext:
|
||||||
|
"""Compose the run context per §5: merge inbox → seed → fold — read-only.
|
||||||
|
|
||||||
|
Citations are built BEFORE anything else so an uncitable context fails
|
||||||
|
fast ahead of any spend (§9). A missing/empty ``inbox_dir`` (or ``None``)
|
||||||
|
leaves the composition identical to the no-inbox base. Nothing is ever
|
||||||
|
written — the system reads the inbox, the expert writes it (§3 Step 7).
|
||||||
|
"""
|
||||||
|
citations = build_citations(navigate_bundle(bundle_dir))
|
||||||
|
ir_projection = load_validator_input(bundle_dir)
|
||||||
|
store = VerdictStore()
|
||||||
|
inbox_merged = merge_inbox_into_store(store, inbox_dir) if inbox_dir is not None else 0
|
||||||
|
seeded = seed_store_from_bundle(store, bundle_dir)
|
||||||
|
context = fold_experience(
|
||||||
|
store,
|
||||||
|
CandidateFeatures.from_proposal(ir_projection),
|
||||||
|
bundle_context(bundle_dir),
|
||||||
|
k,
|
||||||
|
)
|
||||||
|
return ComposedRunContext(
|
||||||
|
context=context,
|
||||||
|
citations=citations,
|
||||||
|
ir_projection=ir_projection,
|
||||||
|
inbox_merged=inbox_merged,
|
||||||
|
seeded=seeded,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def default_client_factory(contracts: Contracts, max_budget_usd_per_call: float) -> ModelClient:
|
||||||
|
"""The CLI's default: the real SDK client (run-path only, §1).
|
||||||
|
|
||||||
|
Imported lazily so composing/executing with an injected client never
|
||||||
|
touches the SDK module — the suite drives the same orchestration with the
|
||||||
|
scripted stand-in.
|
||||||
|
"""
|
||||||
|
from portfolio_optimiser_claude.sdk_client import SdkModelClient
|
||||||
|
|
||||||
|
return SdkModelClient(contracts.model_map, max_budget_usd_per_call=max_budget_usd_per_call)
|
||||||
|
|
||||||
|
|
||||||
|
def _client_cost_usd(client: ModelClient) -> float | None:
|
||||||
|
# Only the SDK client accounts USD; a client without the attribute
|
||||||
|
# persists an honest null (the usage artifact allows it), never a 0.0.
|
||||||
|
cost = getattr(client, "total_cost_usd", None)
|
||||||
|
return None if cost is None else round(float(cost), 6)
|
||||||
|
|
||||||
|
|
||||||
|
def execute_run(
|
||||||
|
client: ModelClient,
|
||||||
|
composed: ComposedRunContext,
|
||||||
|
*,
|
||||||
|
contracts: Contracts,
|
||||||
|
out_dir: Path,
|
||||||
|
max_debate_rounds: int,
|
||||||
|
max_attempts: int,
|
||||||
|
) -> int:
|
||||||
|
"""Drive the loop under the §8 meter; persist artifacts on BOTH outcomes.
|
||||||
|
|
||||||
|
Exit 0: the run completed (validated or typed rejection) and its artifacts
|
||||||
|
are on disk. Exit 3: a structured budget stop (§8) — the stop event and
|
||||||
|
the usage-vs-caps artifact are persisted; a stop is never a silent hang.
|
||||||
|
"""
|
||||||
|
meter = BudgetMeter(contracts.termination)
|
||||||
|
try:
|
||||||
|
result = run_project(
|
||||||
|
client,
|
||||||
|
composed.context,
|
||||||
|
meter=meter,
|
||||||
|
max_debate_rounds=max_debate_rounds,
|
||||||
|
max_attempts=max_attempts,
|
||||||
|
default_project_id=composed.ir_projection.project_id,
|
||||||
|
)
|
||||||
|
except BudgetExceeded as stop:
|
||||||
|
print(f"STOPPED by budget: {stop.kind} observed {stop.observed} > limit {stop.limit}")
|
||||||
|
stop_paths = persist_stop_artifacts(
|
||||||
|
out_dir,
|
||||||
|
stop=stop,
|
||||||
|
termination=contracts.termination,
|
||||||
|
tokens_used=meter.tokens_used,
|
||||||
|
rounds_used=meter.rounds_used,
|
||||||
|
cost_usd=_client_cost_usd(client),
|
||||||
|
)
|
||||||
|
for name, path in sorted(stop_paths.items()):
|
||||||
|
print(f"artifact: {name} -> {path}")
|
||||||
|
return 3
|
||||||
|
|
||||||
|
provenance = Provenance(
|
||||||
|
citations=composed.citations,
|
||||||
|
model=getattr(client, "last_model", None) or "unknown", # §9: real id or neutral
|
||||||
|
role=_PROPOSER_ROLE,
|
||||||
|
validator_decision=result.validator_decision,
|
||||||
|
tokens_used=meter.tokens_used,
|
||||||
|
)
|
||||||
|
paths = persist_run_artifacts(
|
||||||
|
out_dir,
|
||||||
|
run=result,
|
||||||
|
provenance=provenance,
|
||||||
|
termination=contracts.termination,
|
||||||
|
tokens_used=meter.tokens_used,
|
||||||
|
rounds_used=meter.rounds_used,
|
||||||
|
cost_usd=_client_cost_usd(client),
|
||||||
|
)
|
||||||
|
outcome_kind = "rejected" if isinstance(result.outcome, Rejection) else "validated"
|
||||||
|
print(
|
||||||
|
f"result: validator={result.validator_decision} checker={result.checker_decision} "
|
||||||
|
f"attempts={result.attempts} outcome={outcome_kind}"
|
||||||
|
)
|
||||||
|
for name, path in sorted(paths.items()):
|
||||||
|
print(f"artifact: {name} -> {path}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None, *, client_factory: ClientFactory | None = None) -> int:
|
||||||
|
"""The thin CLI: contracts fail-fast (§10) → compose (§5) → execute (§3, §8)."""
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Run one project through the loop (merge inbox → seed → fold → run)."
|
||||||
|
)
|
||||||
|
parser.add_argument("--bundle", type=Path, required=True)
|
||||||
|
parser.add_argument("--inbox", type=Path, default=None)
|
||||||
|
parser.add_argument("--out", type=Path, default=Path("runs") / "run")
|
||||||
|
parser.add_argument("--max-rounds", type=int, default=12)
|
||||||
|
parser.add_argument("--max-tokens", type=int, default=150_000)
|
||||||
|
parser.add_argument("--max-budget-usd-per-call", type=float, default=0.25)
|
||||||
|
parser.add_argument("--max-debate-rounds", type=int, default=3)
|
||||||
|
parser.add_argument("--max-attempts", type=int, default=3)
|
||||||
|
parser.add_argument("--top-k", type=int, default=3)
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
# §10: ALL startup contracts schema-validated BEFORE any model client exists.
|
||||||
|
contracts = load_contracts(
|
||||||
|
data_source={"docs_dir": str(args.bundle), "top_k": args.top_k},
|
||||||
|
termination={"max_rounds": args.max_rounds, "max_tokens": args.max_tokens},
|
||||||
|
feedback={"decision": "approved", "rationale": "startup shape check (§10)"},
|
||||||
|
)
|
||||||
|
# The navigated dir is the CONTRACT's, so the validated config is load-bearing.
|
||||||
|
composed = compose_run_context(
|
||||||
|
Path(contracts.data_source.docs_dir), args.inbox, k=contracts.data_source.top_k
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
f"run: bundle={args.bundle.name} inbox_merged={composed.inbox_merged} "
|
||||||
|
f"seeded={composed.seeded} caps: max_rounds={args.max_rounds} "
|
||||||
|
f"max_tokens={args.max_tokens} "
|
||||||
|
f"max_budget_usd_per_call={args.max_budget_usd_per_call}"
|
||||||
|
)
|
||||||
|
factory = default_client_factory if client_factory is None else client_factory
|
||||||
|
client = factory(contracts, args.max_budget_usd_per_call)
|
||||||
|
return execute_run(
|
||||||
|
client,
|
||||||
|
composed,
|
||||||
|
contracts=contracts,
|
||||||
|
out_dir=args.out,
|
||||||
|
max_debate_rounds=args.max_debate_rounds,
|
||||||
|
max_attempts=args.max_attempts,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
|
@ -99,3 +99,60 @@ class TestErrorContract:
|
||||||
ingest.SourceError,
|
ingest.SourceError,
|
||||||
):
|
):
|
||||||
assert issubclass(exc, ingest.IngestError)
|
assert issubclass(exc, ingest.IngestError)
|
||||||
|
|
||||||
|
|
||||||
|
class TestLibraryGuarantees:
|
||||||
|
"""K2.9 (R-3): library-side guarantees this consumer relies on, bound at the seam.
|
||||||
|
|
||||||
|
Pre-adoption the local connector crashed mid-materialization on an empty
|
||||||
|
CSV and left a PARTIAL bundle on disk (run-proven R-3). The library stages
|
||||||
|
in memory, so the typed ``SourceError`` fires BEFORE the disk phase. Bound
|
||||||
|
THROUGH the consumer entry point so a pin bump can never silently regress
|
||||||
|
either guarantee.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def test_empty_csv_fails_typed_before_any_disk_write(self, tmp_path: Path) -> None:
|
||||||
|
case = tmp_path / "case"
|
||||||
|
fixture = case / "fixture"
|
||||||
|
fixture.mkdir(parents=True)
|
||||||
|
(fixture / "e.csv").write_text("", encoding="utf-8")
|
||||||
|
manifest = {
|
||||||
|
"manifest_version": 1,
|
||||||
|
"source": {"type": "file", "id": "arkiv", "root": "fixture"},
|
||||||
|
"bundle_summary": "s",
|
||||||
|
"extractions": [
|
||||||
|
{"id": "e", "title": "T", "query": "e.csv", "okf_type": "dataset", "max_rows": 5}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
manifest_path = case / "manifest.json"
|
||||||
|
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
|
||||||
|
bundle = tmp_path / "bundle"
|
||||||
|
with pytest.raises(ingest.SourceError):
|
||||||
|
ingest.materialize(manifest_path, bundle, INGESTED_AT)
|
||||||
|
assert not bundle.exists() # never a partial bundle (in-memory staging)
|
||||||
|
|
||||||
|
def test_non_select_sql_fails_typed_with_no_partial_bundle(
|
||||||
|
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
db = tmp_path / "src.sqlite"
|
||||||
|
con = sqlite3.connect(db)
|
||||||
|
con.execute("CREATE TABLE t (a INTEGER)")
|
||||||
|
con.commit()
|
||||||
|
con.close()
|
||||||
|
monkeypatch.setenv("SRC_DSN", str(db))
|
||||||
|
manifest = {
|
||||||
|
"manifest_version": 1,
|
||||||
|
"source": {"type": "sql", "id": "db", "connection_ref": "SRC_DSN"},
|
||||||
|
"bundle_summary": "s",
|
||||||
|
"extractions": [
|
||||||
|
{"id": "e", "title": "T", "query": "BEGIN", "okf_type": "dataset", "max_rows": 5}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
manifest_path = tmp_path / "manifest.json"
|
||||||
|
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
|
||||||
|
bundle = tmp_path / "bundle"
|
||||||
|
with pytest.raises(ingest.SourceError, match="returned no columns"):
|
||||||
|
ingest.materialize(manifest_path, bundle, INGESTED_AT)
|
||||||
|
assert not bundle.exists()
|
||||||
|
|
|
||||||
160
tests/test_run_entrance_loadbearing.py
Normal file
160
tests/test_run_entrance_loadbearing.py
Normal file
|
|
@ -0,0 +1,160 @@
|
||||||
|
"""Run-entrance seams — LOAD-BEARING (C2.0; method-spec §3 Step 7, §5, §8, §11).
|
||||||
|
|
||||||
|
The seam this file keeps alive: the SHIPPABLE entrance (``run.py``) composes
|
||||||
|
the §5 sequence (merge inbox → seed → fold) and drives the same orchestration
|
||||||
|
the fasit run used — so the README's inbox claim is true of a deliverable
|
||||||
|
path. ``run_s10.py`` stays byte-frozen and is still never imported here.
|
||||||
|
|
||||||
|
Detach proofs: drop the merge call from the composition → the inbox verdict
|
||||||
|
never reaches the composed context → red. Drop the budget-stop persistence
|
||||||
|
branch from the entrance → the structured stop leaves no artifacts → red
|
||||||
|
(R-10: the orchestration branch the fasit suite could never bind, bound here
|
||||||
|
on the new path with a scripted client and a low cap).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
from _scripted import ScriptedClient, reply
|
||||||
|
|
||||||
|
from portfolio_optimiser_claude.contracts import Contracts, load_contracts
|
||||||
|
from portfolio_optimiser_claude.experience import CandidateFeatures
|
||||||
|
from portfolio_optimiser_claude.inbox import VerdictDocument, write_verdict
|
||||||
|
from portfolio_optimiser_claude.ir import load_validator_input
|
||||||
|
from portfolio_optimiser_claude.loop import ModelClient
|
||||||
|
from portfolio_optimiser_claude.run import compose_run_context, default_client_factory, main
|
||||||
|
|
||||||
|
BUNDLE = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
|
||||||
|
# Distinct from the bundle seed's own learning marker (realiseringsgrad=0.82),
|
||||||
|
# so its presence proves the INBOX path, not the seeding path.
|
||||||
|
MARKER = "INBOX-MARKER-79"
|
||||||
|
RATIONALE = f"Justert: i drift realiseres ~79% ({MARKER})."
|
||||||
|
|
||||||
|
ClientFactory = Callable[[Contracts, float], ModelClient]
|
||||||
|
|
||||||
|
|
||||||
|
def _inbox_document() -> VerdictDocument:
|
||||||
|
# The expert judges the bundle's own candidate, so retrieval ranks the
|
||||||
|
# verdict at similarity 1.0 and the fold must carry it.
|
||||||
|
features = CandidateFeatures.from_proposal(load_validator_input(BUNDLE))
|
||||||
|
return VerdictDocument.from_candidate(
|
||||||
|
features,
|
||||||
|
decision="approved_with_adjustment",
|
||||||
|
rationale=RATIONALE,
|
||||||
|
description="expert judgement of the bundle candidate",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _scripted_factory(replies: list[object]) -> tuple[ClientFactory, list[ScriptedClient]]:
|
||||||
|
created: list[ScriptedClient] = []
|
||||||
|
|
||||||
|
def factory(contracts: Contracts, max_budget_usd_per_call: float) -> ModelClient:
|
||||||
|
client = ScriptedClient(replies=list(replies)) # type: ignore[arg-type]
|
||||||
|
created.append(client)
|
||||||
|
return client
|
||||||
|
|
||||||
|
return factory, created
|
||||||
|
|
||||||
|
|
||||||
|
class TestComposeRunContext:
|
||||||
|
"""§5 sequence, composed: merge inbox → seed → fold — read-only on the inbox."""
|
||||||
|
|
||||||
|
def test_a_dropped_verdict_reaches_the_composed_context(self, tmp_path: Path) -> None:
|
||||||
|
inbox = tmp_path / "inbox"
|
||||||
|
write_verdict(inbox, _inbox_document())
|
||||||
|
composed = compose_run_context(BUNDLE, inbox, k=3)
|
||||||
|
assert MARKER in composed.context
|
||||||
|
assert composed.inbox_merged == 1
|
||||||
|
assert composed.seeded == 1
|
||||||
|
|
||||||
|
def test_empty_inbox_control_keeps_the_base_composition(self, tmp_path: Path) -> None:
|
||||||
|
inbox = tmp_path / "inbox"
|
||||||
|
inbox.mkdir()
|
||||||
|
with_empty = compose_run_context(BUNDLE, inbox, k=3)
|
||||||
|
without = compose_run_context(BUNDLE, None, k=3)
|
||||||
|
assert with_empty.context == without.context
|
||||||
|
assert MARKER not in without.context
|
||||||
|
assert with_empty.inbox_merged == 0
|
||||||
|
|
||||||
|
def test_missing_inbox_folder_behaves_like_empty(self, tmp_path: Path) -> None:
|
||||||
|
composed = compose_run_context(BUNDLE, tmp_path / "never-created", k=3)
|
||||||
|
assert composed.context == compose_run_context(BUNDLE, None, k=3).context
|
||||||
|
|
||||||
|
def test_composition_never_writes_to_the_inbox(self, tmp_path: Path) -> None:
|
||||||
|
# Role split (§3 Step 7, unwaivable): the entrance READS the inbox.
|
||||||
|
inbox = tmp_path / "inbox"
|
||||||
|
write_verdict(inbox, _inbox_document())
|
||||||
|
before = {p.name: p.read_bytes() for p in inbox.iterdir()}
|
||||||
|
compose_run_context(BUNDLE, inbox, k=3)
|
||||||
|
after = {p.name: p.read_bytes() for p in inbox.iterdir()}
|
||||||
|
assert after == before
|
||||||
|
|
||||||
|
|
||||||
|
class TestEntranceHappyPath:
|
||||||
|
"""The thin CLI wires compose → run → persist; the inbox signal survives the chain."""
|
||||||
|
|
||||||
|
def test_exit_zero_persists_run_artifacts_and_threads_the_inbox(self, tmp_path: Path) -> None:
|
||||||
|
inbox = tmp_path / "inbox"
|
||||||
|
write_verdict(inbox, _inbox_document())
|
||||||
|
out = tmp_path / "out"
|
||||||
|
factory, created = _scripted_factory(
|
||||||
|
[
|
||||||
|
reply("debate reasoning"),
|
||||||
|
reply("VERDICT: APPROVE"),
|
||||||
|
reply(json.dumps(load_validator_input(BUNDLE).model_dump())),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
code = main(
|
||||||
|
["--bundle", str(BUNDLE), "--inbox", str(inbox), "--out", str(out)],
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
assert code == 0
|
||||||
|
(client,) = created
|
||||||
|
assert any(MARKER in prompt for prompt in client.prompts("proposer"))
|
||||||
|
run_result = json.loads((out / "run_result.json").read_text("utf-8"))
|
||||||
|
assert run_result["validator_decision"] == "validated"
|
||||||
|
for artifact in ("proposal.json", "provenance.json", "usage.json"):
|
||||||
|
assert (out / artifact).is_file()
|
||||||
|
|
||||||
|
|
||||||
|
class TestBudgetStopBinding:
|
||||||
|
"""R-10: the budget-stop branch — structured exit + persisted stop artifacts."""
|
||||||
|
|
||||||
|
def test_budget_stop_persists_stop_artifacts_and_exits_structurally(
|
||||||
|
self, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
out = tmp_path / "out"
|
||||||
|
factory, _ = _scripted_factory([reply("debate reasoning", usage_tokens=10)])
|
||||||
|
code = main(
|
||||||
|
["--bundle", str(BUNDLE), "--out", str(out), "--max-tokens", "5"],
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
assert code == 3
|
||||||
|
stop = json.loads((out / "stop.json").read_text("utf-8"))
|
||||||
|
assert stop == {"kind": "tokens", "limit": 5, "observed": 10}
|
||||||
|
usage = json.loads((out / "usage.json").read_text("utf-8"))
|
||||||
|
assert usage["tokens_used"] == 10
|
||||||
|
assert usage["max_tokens"] == 5
|
||||||
|
# A client without cost accounting persists an honest null, never a 0.0.
|
||||||
|
assert usage["cost_usd"] is None
|
||||||
|
assert not (out / "proposal.json").exists()
|
||||||
|
assert not (out / "run_result.json").exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestDefaultClientFactory:
|
||||||
|
"""The CLI's default factory constructs the REAL SDK client — wired, not executed."""
|
||||||
|
|
||||||
|
def test_default_factory_returns_the_sdk_client(self) -> None:
|
||||||
|
# Construction needs no API key (verified premise); no query() is made.
|
||||||
|
from portfolio_optimiser_claude.sdk_client import SdkModelClient
|
||||||
|
|
||||||
|
contracts = load_contracts(
|
||||||
|
data_source={"docs_dir": str(BUNDLE), "top_k": 3},
|
||||||
|
termination={"max_rounds": 1, "max_tokens": 1},
|
||||||
|
feedback={"decision": "approved", "rationale": "startup shape check (§10)"},
|
||||||
|
)
|
||||||
|
client = default_client_factory(contracts, 0.25)
|
||||||
|
assert isinstance(client, SdkModelClient)
|
||||||
Loading…
Add table
Add a link
Reference in a new issue