Fiks-ordre 20260904T070930Z-8335015575-from-.claude. Two scoped defects the
S7b syretest observed and left unfixed (docs/2026-09-04-syretest-s7b-k2.md
§ 3.6):
1. `BudgetExceeded` (a RuntimeError, econ 56) could leave `--explore` as a
raw Python traceback from TWO raise sites: the exploration loop's own
round/token cap (`explore()`/`resume_exploration()`, uncaught in
`main()`'s exploration `try`), and `generate_via_llm`'s retry loop when a
mandate's own-proposal evaluation hits an unparseable reply (the full-run
dispatch's `except` tuple only knew `ValueError`/`FileNotFoundError`/
`ValidationError`). Both now end as `run refused: {exc}` on stderr, rc 1,
same shape as every other loader refusal in run.py.
2. `{run_id}-exploration.json` carried rounds/tool_calls/plan_reviews/
quick_validations but not the approaches the loop actually shaped — those
stood only in the stdout mandate announcement. `explore.trace_payload`
now takes a required `mandate` keyword and renders `mandate.approaches`
under an "approaches" key, so an operator reading the artefact days later
(the whole point of the async U12 door) can recover what the run decided
to evaluate without the terminal.
Fifteen mutations across three detach points, all red against the full
suite: the exploration-loop except clause (2 red), the full-run except
tuple (1 red), and the approaches rendering (1 red) — plus a control per
fix proving the happy path is unaffected. Green control 1295 passed / 5
skipped (supersett of S7b's 1290/5, 0 removed), golden
demo-transcript.stdout byte-unchanged (shasum -a 1 = ea8c534…), ruff +
mypy clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
172 lines
6.8 KiB
Python
172 lines
6.8 KiB
Python
"""Fiks-ordre 20260904T070930Z, punkt 2 — ``{run_id}-exploration.json`` must carry the approaches
|
|
the loop actually SHAPED, not just the mandate announcement on stdout.
|
|
|
|
**Measured** on K2's syretest (``docs/2026-09-04-syretest-s7b-k2.md § 3.6``): the artefact had
|
|
``rounds``/``tool_calls``/``plan_reviews``/``quick_validations``, while which approaches the loop
|
|
minted stood only in ``mandate.announce``'s stdout block — printed by ``run.py``, never written.
|
|
An operator reading the artefact days later (the whole point of the async U12 door) could not
|
|
recover what the run had decided to evaluate without also having kept the terminal's stdout.
|
|
|
|
The fix widens ``explore.trace_payload`` with a required ``mandate`` keyword: it renders
|
|
``mandate.approaches`` under an ``"approaches"`` key, using the SAME fields ``mandate.announce``
|
|
reads off ``Approach`` (id, label). This file proves the artefact and the stdout announcement
|
|
agree — not just that a key exists.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from collections.abc import Callable
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
from agent_framework import BaseChatClient
|
|
|
|
from portfolio_optimiser import explore, run
|
|
from portfolio_optimiser.explore import ExplorationTrace
|
|
from portfolio_optimiser.simulation import ScriptedChatClient
|
|
|
|
_BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
|
|
_PID = "BYGG-KONTOR-NORD"
|
|
|
|
_LABEL = "SENTINEL-APPROACHES-9f2a"
|
|
_RATIONALE = "because the fittings are old (sentinel rationale)"
|
|
|
|
_ENERGY_REPLY = (
|
|
'{"measure":"LED-retrofit av kontorbelysning","affected_items":'
|
|
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
|
|
)
|
|
|
|
_CONTRACT_JSON: dict[str, Any] = {
|
|
"max_rounds": 4,
|
|
"max_tokens": 100_000,
|
|
"max_stall_count": 2,
|
|
"max_reset_count": 1,
|
|
"max_plan_revisions": 0,
|
|
"enable_plan_review": False,
|
|
}
|
|
|
|
|
|
def _ledger_json(*, satisfied: bool, speaker: str = "hypothesiser") -> str:
|
|
return json.dumps(
|
|
{
|
|
"is_request_satisfied": {"reason": "r", "answer": satisfied},
|
|
"is_in_loop": {"reason": "r", "answer": False},
|
|
"is_progress_being_made": {"reason": "r", "answer": True},
|
|
"next_speaker": {"reason": "r", "answer": speaker},
|
|
"instruction_or_question": {"reason": "r", "answer": "Shape one hypothesis."},
|
|
}
|
|
)
|
|
|
|
|
|
def _manager_script(ledgers: list[str]) -> Callable[[str, str], str]:
|
|
def _select(blob: str, _role: str) -> str:
|
|
if "provide the final answer" in blob:
|
|
return "FINAL: exploration done."
|
|
if "pure JSON format" in blob:
|
|
return ledgers.pop(0) if ledgers else _ledger_json(satisfied=True)
|
|
if "went wrong on this last run" in blob:
|
|
return "PLAN-UPDATE: revised plan."
|
|
if "rewrite the following fact sheet" in blob:
|
|
return "FACTS-UPDATE: revised facts."
|
|
if "bullet-point plan" in blob:
|
|
return "PLAN: - ask the hypothesiser"
|
|
if "pre-survey" in blob:
|
|
return "FACTS: the bundle is anchored."
|
|
return "{}"
|
|
|
|
return _select
|
|
|
|
|
|
def _hypothesis_line(label: str, rationale: str) -> str:
|
|
return f"{explore.HYPOTHESIS_MARKER} " + json.dumps({"label": label, "rationale": rationale})
|
|
|
|
|
|
def _factory(*, ledgers: list[str], hypothesiser: list[str]) -> Callable[[str], BaseChatClient]:
|
|
def factory(role: str) -> BaseChatClient:
|
|
if role == explore.MANAGER_ROLE:
|
|
return ScriptedChatClient(reply_selector=_manager_script(ledgers), role=role)
|
|
if role == explore.HYPOTHESISER_ROLE:
|
|
replies = list(hypothesiser)
|
|
|
|
def _hyp(_blob: str, _role: str) -> str:
|
|
return replies.pop(0) if replies else "nothing further."
|
|
|
|
return ScriptedChatClient(reply_selector=_hyp, role=role)
|
|
if role == explore.NAVIGATOR_ROLE:
|
|
return ScriptedChatClient("NAVIGATOR: index read.", role=role)
|
|
return ScriptedChatClient(_ENERGY_REPLY, role=role)
|
|
|
|
return factory
|
|
|
|
|
|
def _config_file(tmp_path: Path) -> str:
|
|
path = tmp_path / "exploration.json"
|
|
path.write_text(json.dumps(_CONTRACT_JSON), encoding="utf-8")
|
|
return str(path)
|
|
|
|
|
|
def _base_argv(tmp_path: Path) -> list[str]:
|
|
return [
|
|
_PID,
|
|
"--docs-dir",
|
|
str(_BUNDLE_DIR),
|
|
"--bundle-dir",
|
|
str(_BUNDLE_DIR),
|
|
"--explore",
|
|
"Find the cheapest saving.",
|
|
"--explore-config",
|
|
_config_file(tmp_path),
|
|
]
|
|
|
|
|
|
def test_the_artefact_names_the_same_approach_the_announcement_did(
|
|
tmp_path, capsys, monkeypatch
|
|
) -> None:
|
|
"""Goal test: the label ``mandate.announce`` printed on stdout is the SAME label the artefact
|
|
carries, under the SAME minted id — read back from disk, not scraped off stdout.
|
|
|
|
Detach point: drop the ``"approaches"`` key from ``trace_payload`` → ``KeyError`` here.
|
|
Detach point: render an empty list regardless of ``mandate`` → the id/label assertions go RED.
|
|
"""
|
|
factory = _factory(
|
|
ledgers=[_ledger_json(satisfied=False), _ledger_json(satisfied=True)],
|
|
hypothesiser=[_hypothesis_line(_LABEL, _RATIONALE)],
|
|
)
|
|
monkeypatch.setattr("portfolio_optimiser.run._default_factory", lambda profile: factory)
|
|
|
|
outbox = tmp_path / "outbox"
|
|
rc = run.main(_base_argv(tmp_path) + ["--outbox-dir", str(outbox), "--run-id", "r3"])
|
|
assert rc == 0, capsys.readouterr().err
|
|
|
|
out = capsys.readouterr().out
|
|
assert f"1. hypothesis-1 — {_LABEL}" in out, "the stdout announcement is the fixture's oracle"
|
|
|
|
payload = json.loads((outbox / "r3-exploration.json").read_text(encoding="utf-8"))
|
|
assert payload["approaches"] == [
|
|
{
|
|
"id": "hypothesis-1",
|
|
"label": _LABEL,
|
|
"description": _RATIONALE,
|
|
"affected_codes": [],
|
|
"claimed_saving_nok": None,
|
|
"bundle_id": "bygg-energi-mikro",
|
|
}
|
|
], (
|
|
"the artefact must name the same approach the terminal announced, or a reader without the terminal learns nothing"
|
|
)
|
|
|
|
|
|
def test_a_mandate_with_no_commissioned_approaches_renders_an_empty_but_present_list() -> None:
|
|
"""Control: ``mandate.approaches == ()`` (pure own-proposals) is a legitimate state and must
|
|
render as ``[]``, not be confused with "no mandate was ever formed" — that absence is what
|
|
``completed=False`` already says (``trace_payload``'s own docstring)."""
|
|
trace = ExplorationTrace()
|
|
from portfolio_optimiser.mandate import Mandate
|
|
|
|
empty_mandate = Mandate(objective="find something", approaches=())
|
|
payload = explore.trace_payload(trace, stop=None, completed=True, mandate=empty_mandate)
|
|
assert payload["approaches"] == []
|
|
|
|
payload_no_mandate = explore.trace_payload(trace, stop=None, completed=False, mandate=None)
|
|
assert payload_no_mandate["approaches"] == []
|