portfolio-optimiser/src/portfolio_optimiser/workflow.py
Kjell Tore Guttormsen 4ec778c855 feat(fase3): make the maker-checker checker actually gate the reasoning
Closes gap #3 (maalbilde §5): the GroupChat checker critiqued into the void —
output_from=[proposer] surfaced only the proposer, so an explicit checker
rejection was ignored and the deterministic validator was the sole gate. Two
falsifiers now act on the same candidate: the validator gates the NUMBERS
(blocking, unchanged), the checker gates the REASONING (maalbilde §2/§6).

- workflow.py: output_from=agents surfaces both participants; the checker
  instruction ends with a VERDICT: APPROVE / VERDICT: REJECT - <reason> line.
- run.py: _authored_texts() reads author_name through out.messages (MAF 1.9.0
  puts it there, not on the AgentResponse); _debate_text() now selects the
  PROPOSER-authored output (fixes a latent texts[-1] regression that would feed
  the checker's verdict to generation at even round counts); _checker_verdict()
  parses the gate decision. An explicit REJECT overrides an otherwise-validated
  outcome to a checker-sourced Rejection. Opt-in-reject (fail-open on a missing
  marker). RunResult gains checker_verdict; provenance.validator_decision is
  stamped from the validator outcome BEFORE the override, so it never conflates
  the two falsifiers (provenance honesty).

Load-bearing (maalbilde §7): tests/test_checker_gate_loadbearing.py is a PAIR —
an explicit checker REJECT on a VALIDATOR-VALID proposal yields a Rejection whose
reason carries the checker's reason while validator_decision stays "validated";
the causality control (checker APPROVE, same proposer) validates normally. Proven
RED on BOTH detach points (revert output_from, or drop the override).

Suite 134->136 passed, 4 skipped; mypy + ruff check clean. Pre-existing
ruff-format drift (backends/budget/verdicts/test_contracts) left untouched for a
surgical diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
2026-06-30 07:24:30 +02:00

117 lines
5.3 KiB
Python

"""fresh_workflow isolation factory + maker-checker GroupChat (B7 / B4 / Layer-1 HITL).
Two Fase 1 findings are load-bearing here:
* **B7 (state isolation):** a *reused* built workflow accumulates the MAF conversation thread
across ``.run()`` calls, so project N contaminates N+1. The mitigation is a FACTORY that
builds a FRESH ``GroupChatBuilder`` with FRESH chat clients **per project run** — zero
cross-run bleed. ``fresh_workflow`` is that factory.
* **B4 (bounded debate):** a never-converging debate does NOT self-terminate. ``with_max_rounds``
is the hard cap; the budget middleware (Step 4, wired by the orchestrator) is the external
token guard. ``make_termination`` is a higher turn-count safety net, not the sole stop.
**Layer-1 HITL** optionally enables the GA in-run synchronous review gate
(``with_request_info(agents=[checker])``) — no checkpoint (research 01: durable checkpoint
resume is fragile; the durable verdict is captured out-of-band in Step 12).
"""
from __future__ import annotations
from collections.abc import Callable, Sequence
from typing import Any
from agent_framework import Agent, BaseChatClient, Message
from agent_framework.orchestrations import GroupChatBuilder
_MAKER_CHECKER_ROLES = ("proposer", "checker")
_INSTRUCTIONS = {
"proposer": "You propose one concrete cost-saving measure for the project.",
"checker": (
"You critique the proposal's reasoning and flag any constraint violation. "
"End your reply with exactly one verdict line: 'VERDICT: APPROVE' if the reasoning "
"holds, or 'VERDICT: REJECT - <short reason>' if it does not. The validator gates the "
"numbers; your verdict gates the reasoning (Step 3/4 — målbilde §2)."
),
}
def make_termination(n_turns: int) -> Callable[[Sequence[Message]], bool]:
"""A GroupChat ``TerminationCondition`` that stops once ``n_turns`` turns have been taken.
Deterministic and endpoint-free; the hard cap is ``with_max_rounds`` (B4)."""
if n_turns <= 0:
raise ValueError(f"n_turns must be positive, got {n_turns}")
def terminate(conversation: Sequence[Message]) -> bool:
return len(conversation) >= n_turns
return terminate
def maker_checker_agents(
client_factory: Callable[[str], BaseChatClient],
*,
tools: Sequence[Any] | None = None,
middleware: Sequence[Any] | None = None,
) -> list[Agent]:
"""FRESH proposer + checker agents, each backed by a FRESH client (zero cross-run state).
``tools`` (the citation-bearing data-source tool, F7) and ``middleware`` (the budget
``ChatMiddleware``, F2) are attached to EVERY agent so the debate reaches the data source
as a tool and is metered/short-circuited via the middleware. Both are constructed by the
orchestrator (``run_project``) and threaded through ``fresh_workflow``."""
return [
Agent(
client_factory(role),
_INSTRUCTIONS[role],
name=role,
tools=tools,
middleware=middleware,
)
for role in _MAKER_CHECKER_ROLES
]
def fresh_workflow(
client_factory: Callable[[str], BaseChatClient],
*,
max_rounds: int = 3,
enable_layer1_hitl: bool = False,
tools: Sequence[Any] | None = None,
middleware: Sequence[Any] | None = None,
) -> Any:
"""Build a FRESH maker-checker GroupChat with FRESH clients per call (B7). Bounded by
``with_max_rounds`` (B4) plus a higher turn-count termination safety net. ``client_factory``
is called once per role, so each run owns its own clients — no state survives between runs.
``tools`` + ``middleware`` are attached to each agent (F2/F7; constructed by the
orchestrator). ``output_from=agents`` makes ``WorkflowRunResult.get_outputs()`` surface BOTH
participants' converged outputs — the proposer's (fed to generation, F1) AND the checker's
gate verdict (Step 3/4); ``run.py`` separates them by per-message ``author_name``. Without it,
``get_outputs()`` yields only the orchestrator's "reached max rounds" notice (verified against
installed 1.9.0).
"""
agents = maker_checker_agents(client_factory, tools=tools, middleware=middleware)
# Agents are built from _MAKER_CHECKER_ROLES in order with name=role, so the role tuple
# IS the (typed, non-None) name list the selector cycles over.
names: list[str] = list(_MAKER_CHECKER_ROLES)
counter = {"n": 0}
def select(_state: object) -> str:
choice = names[counter["n"] % len(names)]
counter["n"] += 1
return choice
builder = GroupChatBuilder(
participants=agents,
selection_func=select,
# Safety net well above the hard cap; with_max_rounds is the binding bound (B4).
termination_condition=make_termination(max_rounds * len(names) + 1),
# Surface BOTH participants so get_outputs() carries the proposer's converged output
# (fed to generation, F1) AND the checker's gate verdict (Step 3/4); run.py separates
# them by author_name. The default surfaces only the orchestrator's termination notice.
output_from=agents,
).with_max_rounds(max_rounds)
if enable_layer1_hitl:
# Layer-1: in-run synchronous review on the checker (no checkpoint — research 01).
builder = builder.with_request_info(agents=[agents[-1]])
return builder.build()