open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(explore): --plan-review gjoer "be om svar, bruke svarene" naabar fra CLI (F4, ORDRE 20260825T133139Z)

Browse source 
F4 fra misjonsreviewen: begge operatorflatene nektet enable_plan_review, og eneste doer var
explore(..., plan_reviewer=...) i bibliotek-APIet. Maalbildets HITL-loop var dermed unaabar for
enhver som ikke importerte pakka. Reviewens tre fil:linje-paastander ble verifisert mot kilden foer
bygging og stemte.

Flate: CLI. `--plan-review` bygger en terminal_plan_reviewer() og gir den til den UENDREDE sloeyfa.
Operatoren vises planen og svarer "approve" eller "revise <hva>"; en revisjon gaar tilbake til
manageren, som replanlegger og spoer IGJEN om den NYE planen.

Gaten er den ANDRE halvdelen av setningen. En doer som printer planen, leser linja og kaster den
bestaar "operatoren ble spurt" og feiler maalbildet -- repoets vakuoes-gate-klasse. T1 er derfor
test_explore_loadbearing sin T15 loeftet til CLI-niva og er ROED mot en alltid-godkjenn-reviewer.
Vitnet er {run_id}-exploration.json (skrevet fra en finally), ikke skrapet stdout.

Fail-closed paa operatorens egen input: alt utenfor vokabularet spoerres paa nytt, og EOF raiser
PlanReviewInputError -- stillhet er aldri en signatur.

Fire nekter ved navn, hvorav to lukket et stille dropp ingen test dekket: report_forbidden (report-
modus returnerer FOER hver utforsknings-nekt) og portefoelje-partisjonen. De to konfig-avhengige
nektene deler tokenet enable_plan_review og har derfor ulik saertekst; den eksisterende testen
asserterte paa det delte tokenet og er rettet (oekt-57-mutasjonen, niende gang).

Hosting nekter fortsatt -- reviewen er synkron og ville blokkert bade HTTP-requesten og event-loekka
som svarer /readiness -- men meldingen navngir na CLI-doeren i stedet for aa paasta at biblioteket er
den eneste.

Mid-loep-spoersmaal er IKKE bygget, og fravaeret er MAALT: _magentic.py har noeyaktig ETT
ctx.request_info (:1044, plan review) i hele modulen. Reviewens "kun plan-review FOER loepet" er
derimot upresist -- samme forespoersel fyrer ogsaa ved re-plan etter en stall.

Load-bearing MAALT (tests/test_plan_review_cli_door_loadbearing.py, 12 tester), elleve mutasjoner
alle roede mot HELE suiten + groenn kontroll 1040/5 og golden demo-transcript.stdout BYTE-UENDRET
(ea8c534773acdbe41ae68f2c55724d69aaf8be4f).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtMDsh2zfp4Bmw8KGJ4aLD
This commit is contained in:
Kjell Tore Guttormsen 2026-08-26 00:27:36 +02:00
commit 84e8de8679
7 changed files with 612 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

77
src/portfolio_optimiser/explore.py
View file

@ -24,10 +24,11 @@ This module imports ``agent_framework.orchestrations`` and therefore may never b
from __future__ import annotations
import json
import sys
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Final, Literal
from typing import Any, Final, Literal, TextIO
from agent_framework import Agent, BaseChatClient, FunctionTool, tool
from agent_framework.orchestrations import (
@ -363,6 +364,80 @@ class PlanReviewDecision:
PlanReviewer = Callable[[PlanReviewRequest], PlanReviewDecision]
class PlanReviewInputError(ExplorationError):
"""A terminal plan review was left without an answer: the input ended mid-review.
A distinct type rather than a distinguishing message, for the reason ``_classify_stop`` reads
counts instead of the termination prose: a caller deciding what happened should never have to
match wording. It is an ``ExplorationError`` (a ``RuntimeError``) because the run FAILED — the
caller's argv was fine and the loop had already started spending; the same channel an
unreadable marked hypothesis leaves by.
"""
#: The closed answer vocabulary of the terminal door. Two words, matched structurally.
_APPROVE_ANSWER: Final = "approve"
_REVISE_ANSWER: Final = "revise"
def terminal_plan_reviewer(
*, stream_in: TextIO | None = None, stream_out: TextIO | None = None
) -> PlanReviewer:
"""A ``PlanReviewer`` that asks the operator at a terminal and reads their typed answer.
This is the door that makes målbilde §3's "still spørsmål, be om svar, bruke svarene" reachable
without importing the package (F4): ``run.py``'s ``--plan-review`` builds one of these and
hands it to ``explore()``. Blocking is not an oversight — ``explore()`` calls the reviewer
synchronously (it is not awaited), so the loop waits on the human exactly as the ``PlanReviewer``
contract says. That is also the reason the hosted surface keeps refusing the review: there,
blocking the reviewer would block the event loop that answers ``/readiness``.
**The streams are resolved at CALL time, not here** (the ``shared_root()`` idiom): a factory
that captured ``sys.stdin`` at construction could not be driven by a caller — or a test — that
replaces the stream afterwards, and the only way left to exercise the door would be a
subprocess.
**Fail-closed on the operator's own input.** ``approve`` signs off; ``revise <what to change>``
sends the words back to the manager. Anything else — a blank line, a typo, a bare ``revise`` —
is asked AGAIN, never taken as a decision. End of input raises ``PlanReviewInputError``:
reading silence as approval would let an autonomous loop run on a plan no human signed, and do
it invisibly. Validation, NEVER repair (the ``write_concept_file`` rule).
"""
def review(request: PlanReviewRequest) -> PlanReviewDecision:
source = sys.stdin if stream_in is None else stream_in
sink = sys.stdout if stream_out is None else stream_out
stalled = " (a RE-PLAN after a stall)" if request.is_stalled else ""
print(f"\nPLAN REVIEW #{request.index + 1}{stalled}", file=sink)
print("--- the plan the exploration would run ---", file=sink)
print(request.plan, file=sink)
if request.current_progress.strip():
print("--- progress so far ---", file=sink)
print(request.current_progress, file=sink)
while True:
print(
f'Answer "{_APPROVE_ANSWER}" to sign it off, '
f'or "{_REVISE_ANSWER} <what to change>": ',
file=sink,
)
sink.flush()
line = source.readline()
if line == "":
raise PlanReviewInputError(
"the plan review reached end of input without an answer. Silence is not a "
"sign-off: the exploration will not run a plan nobody approved"
)
answer = line.strip()
if answer == _APPROVE_ANSWER:
return PlanReviewDecision.approve()
verb, _, feedback = answer.partition(" ")
if verb == _REVISE_ANSWER and feedback.strip():
return PlanReviewDecision.revise(feedback.strip())
print(f"Not an answer: {answer!r}.", file=sink)
return review
# ---------------------------------------------------------------------------------------------
# The tools. Level 1 of the three-guarantee table: real computation, ADVISORY verdicts.
# ---------------------------------------------------------------------------------------------

4
src/portfolio_optimiser/hosting.py
View file

@ -167,7 +167,9 @@ async def _shaped_mandate(consumed: Mapping[str, Any], kwargs: Mapping[str, Any]
if contract.enable_plan_review:
raise InvocationRefused(
"explore_contract sets enable_plan_review, but this surface has no reviewer to answer "
"it: the synchronous plan review would block the request on nobody"
"it: the synchronous plan review would block the request on nobody, and would block "
"the event loop that answers /readiness while doing it. The operator door is the CLI's "
"--plan-review (or explore(..., plan_reviewer=...) in-process)"
)
result = await explore(
str(prompt),

71
src/portfolio_optimiser/run.py
View file

@ -64,6 +64,7 @@ from portfolio_optimiser.explore import (
explore,
exploration_notice,
load_exploration_contract,
terminal_plan_reviewer,
trace_payload,
)
from portfolio_optimiser.generate import ParseFailure, generate_via_llm
@ -1633,8 +1634,18 @@ def main(argv: list[str] | None = None) -> int:
help="the exploration's bounds (JSON, fail-fast, REQUIRES --explore): max_rounds, "
"max_tokens, max_stall_count, max_reset_count, max_plan_revisions, enable_plan_review. "
"Every field is required and none has a default — an omitted bound would fall back to "
"MAF's unbounded loop, not to something conservative. enable_plan_review must be false "
"here: the synchronous review has no reviewer on this surface",
"MAF's unbounded loop, not to something conservative. enable_plan_review requires "
"--plan-review, which is what answers it",
)
parser.add_argument(
"--plan-review",
action="store_true",
help="U13 synchronous HITL door (REQUIRES --explore, and --explore-config must set "
"enable_plan_review): answer the exploration's plan review AT THIS TERMINAL. Before the "
'loop is allowed to run you are shown the plan and answer "approve" or "revise <what to '
'change>"; a revision goes back to the manager, which replans and asks you again. Every '
"round trip is recorded in {run_id}-exploration.json, feedback verbatim. Input that ends "
"without an answer is an error, NEVER a sign-off",
)
parser.add_argument(
"--mcp-config",
@ -1778,6 +1789,10 @@ def main(argv: list[str] | None = None) -> int:
"--scripted-replies": args.scripted_replies is not None,
"--explore": args.explore is not None,
"--explore-config": args.explore_config is not None,
# Report mode returns BELOW, before every exploration refusal, so a flag missing from
# this list is silently dropped rather than refused — which is the whole reason the
# list enumerates every distinguishable flag instead of the ones that would misbehave.
"--plan-review": args.plan_review,
}
if any(report_forbidden.values()):
print(
@ -1823,6 +1838,11 @@ def main(argv: list[str] | None = None) -> int:
# --portfolio --explore has to hear which of the two is wrong.
"--explore": args.explore,
"--explore-config": args.explore_config,
# It answers --explore's review, so it lives on the same side of the partition. By
# NAME for the same reason --explore is: falling through to "--plan-review requires
# --explore" would tell an operator who wrote --portfolio --plan-review to add the one
# flag this mode also refuses.
"--plan-review": args.plan_review,
}
offending = [name for name, value in single_only.items() if value]
if offending:
@ -1922,6 +1942,16 @@ def main(argv: list[str] | None = None) -> int:
# the library API; the refusal names it rather than only forbidding.
# 4. --explore + --live-dry-run contradict: the drill stops before the first model call and an
# exploration IS model calls (the --scripted-replies precedent, same words).
# 5. --plan-review alone answers a review that is never requested (the (1) case, for the U13
# door). Its config-dependent half — the flag against a config that asks for no review, and
# a config that asks for one with no flag — is refused below, once the bounds are loaded.
if args.plan_review and args.explore is None:
print(
"run refused: --plan-review requires --explore (there is no plan to review without an "
"exploration, so the flag would be accepted and then never used)",
file=sys.stderr,
)
return 1
if args.explore_config is not None and args.explore is None:
print(
"run refused: --explore-config requires --explore (the bounds describe an exploration "
@ -1984,16 +2014,31 @@ def main(argv: list[str] | None = None) -> int:
except (FileNotFoundError, ValidationError, ValueError) as exc:
print(f"run refused: {exc}", file=sys.stderr)
return 1
if exploration_contract.enable_plan_review:
# Refused HERE rather than left to ``explore()``, which refuses it too: ExplorationError
# is a RuntimeError and therefore outside this CLI's (ValueError, FileNotFoundError,
# ValidationError) refusal tuple, so it would leave as a traceback instead of the rc 1
# line every other misconfiguration produces. The synchronous review (U13) needs a
# reviewer that blocks the loop, and this surface has none to offer.
# Both halves are refused HERE rather than left to ``explore()``, which refuses them too:
# ExplorationError is a RuntimeError and therefore outside this CLI's (ValueError,
# FileNotFoundError, ValidationError) refusal tuple, so either would leave as a traceback
# instead of the rc 1 line every other misconfiguration produces.
#
# The two messages share the token ``enable_plan_review`` and must NOT share their
# distinguishing wording: a test asserting on the shared substring passes against a
# surface missing one of the branches entirely (measured in økt 57 on --explore).
if exploration_contract.enable_plan_review and not args.plan_review:
# The refusal SURVIVES F4 — a run must never stop at a review nobody can answer — but
# its old wording ("the synchronous door is the library API") stopped being true the
# moment this CLI grew one, so it names the flag instead. A claim a surface makes about
# itself is exactly what Fase 3 measured drifting.
print(
"run refused: --explore-config sets enable_plan_review, but this surface has no "
"reviewer to answer it (the run would stop at a review nobody can answer). The "
"synchronous door is the library API: explore(..., plan_reviewer=...)",
"run refused: --explore-config sets enable_plan_review but no reviewer was "
"offered, so the run would stop at a review nobody can answer. Add --plan-review "
"to answer it at this terminal, or set enable_plan_review to false",
file=sys.stderr,
)
return 1
if args.plan_review and not exploration_contract.enable_plan_review:
print(
"run refused: --plan-review was given but --explore-config sets enable_plan_review "
"false, so no review is ever requested and the reviewer would never be asked "
"anything (refused, never silently ignored)",
file=sys.stderr,
)
return 1
@ -2089,6 +2134,10 @@ def main(argv: list[str] | None = None) -> int:
profile=args.profile,
client_factory=scripted_client_factory,
trace=exploration_trace,
# The F4 door. Built here and never inside ``explore()``: the loop owns the
# seam, the CLI owns which reviewer fills it, and a library that reached for
# stdin on its own would answer for a caller that never offered to.
plan_reviewer=terminal_plan_reviewer() if args.plan_review else None,
)
)
finally: