diff --git a/CLAUDE.md b/CLAUDE.md index a81523b..2c04964 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -46,6 +46,16 @@ Python ≥3.10. MAF (`agent-framework-core` 1.9.0). Pakkehåndtering: `uv`. To b aldri checkeren (de to falsifisererne blandes aldri). Load-bearing: `tests/test_checker_gate_loadbearing.py` blir rød ved BEGGE detach-punkt (revert `output_from`, eller fjern override). Checkeren «må faktisk gate, ELLER vi slutter å kalle det maker-checker». +- **Informert forbedring, bundet (Steg 5, målbilde §5/§7):** `generate_via_llm`s ytre + `max_attempts`-løkke er ikke lenger blind — validatorens *forrige* `Rejection.reason` mates inn i + neste forsøks prompt (`_build_messages(prior_rejection=...)`), så proposeren korrigerer i stedet + for å gjenta. Kun den mest-nylige falsifiseringen (`last`, ikke akkumulert), kun grunnen (aldri + forrige proposal-JSON), under EKSISTERENDE tak (`meter.tick_round` + `max_attempts` — ingen ny + løkke; «forbedre til god nok» uten tak er forbudt). Eneste *per-forsøk*-falsifiserer her er + validatoren; å seede generering med checker-*kritikken* er run-nivå og separat scoped (IKKE bygget + her) — så koden påstår ikke mer enn den gjør. Load-bearing: + `tests/test_step5_refine_loadbearing.py` blir rød når reason-injeksjonen detaches (utfallet + flipper aldri + reason-verbatim-asserten faller); kontrollen beviser at løkka forblir bundet. - **Kostnadsdisiplin:** utvikle primært på lokal profil (gratis); Foundry/Azure (privat tenant finnes) kun til målrettet, minimal verifisering; billigste modeller + små syntetiske data + harde token-tak. Ingen tunge test-kjøringer. - **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet. - Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase. diff --git a/README.md b/README.md index 3e40c70..397f1ad 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,8 @@ The mandatory deterministic backbone (validator + budget meter + provenance) is - **Verdict layer gated out of context.** The `type: verdict` layer is excluded from that context; the candidate's prior expert verdicts reach the hypothesis prompt *only* through the gated ExpeL fold (folded in *before* generation), so a prior verdict provably — and exclusively — influences the next hypothesis. - Two load-bearing tests fail when the seam is detached: the realization signal must reach the prompt via the fold (`tests/test_step1_expel_loadbearing.py`), and must never leak via context (`tests/test_okf.py::test_bundle_context_excludes_verdict_layer`). - **Steps 3/4 — checker gates the reasoning (wired).** Two falsifiers now act on the same candidate: the deterministic validator gates the **numbers** (blocking, as before), and the maker-checker's checker gates the **reasoning** (target picture §2/§6). The checker ends its turn with a `VERDICT: APPROVE` / `VERDICT: REJECT — ` line; an explicit reject blocks an otherwise-validated proposal (`run_project` surfaces both debate participants via `output_from=agents` and overrides the outcome to a checker-sourced `Rejection`). The gate is opt-in-reject (fail-open on a missing marker), and `provenance.validator_decision` stays honest — it reflects the validator only, never the checker. Load-bearing: `tests/test_checker_gate_loadbearing.py` goes red on *either* detach (revert `output_from`, or drop the override). -- **Steps 5–8** (informed refinement, async file feedback, gated wiki promotion) — not yet wired. +- **Step 5 — informed refinement (wired).** The proposer's bounded retry is no longer blind: the validator's *previous* `Rejection.reason` is fed into the next attempt's prompt (`generate.py` `generate_via_llm` → `_build_messages(prior_rejection=...)`), so the model corrects against the falsification instead of re-answering identically (target picture §5/§7). It carries only the most-recent reason (never an accumulated history, never the rejected proposal JSON) and runs under the *existing* `max_attempts` + token-meter cap — no new loop, so "improve until good enough" without a ceiling stays impossible. The only per-attempt falsifier here is the validator; seeding generation with the checker's critique is a run-level, separately-scoped concern and is deliberately not done here. Load-bearing: `tests/test_step5_refine_loadbearing.py` goes red when the reason is detached from the prompt (the outcome never flips *and* the verbatim-reason assertion fails), with a bounded control proving the loop still stops at `max_attempts`. +- **Steps 7–8** (async file feedback, gated wiki promotion) — not yet wired. ## Docs diff --git a/src/portfolio_optimiser/generate.py b/src/portfolio_optimiser/generate.py index 903f31a..1427365 100644 --- a/src/portfolio_optimiser/generate.py +++ b/src/portfolio_optimiser/generate.py @@ -42,7 +42,15 @@ class GenerationError(RuntimeError): """No parseable proposal could be produced within the attempt budget.""" -def _build_messages(project: Project, context: str) -> list[Message]: +def _build_messages( + project: Project, context: str, prior_rejection: Rejection | None = None +) -> list[Message]: + """Build the hypothesis prompt. When ``prior_rejection`` is set (Step 5, målbilde §5/§7), + append a revision block carrying ONLY the falsification *reason* verbatim — never the prior + proposal JSON (minimal honest payload: the model must address the falsification, not parrot + the rejected candidate back). ``None`` -> the byte-identical base prompt, so attempt 1 is + unchanged. The reason carries only the rejected claim/feasible figures, which deliberately + do not collide with other load-bearing prompt markers.""" prompt = ( "Propose ONE concrete cost-saving measure for this project.\n" f"Project: {project.id} - {project.name}\n" @@ -51,6 +59,12 @@ def _build_messages(project: Project, context: str) -> list[Message]: "measure, affected_items (list of {code, quantity, unit_cost}), claimed_saving_nok, " "and optional assumptions." ) + if prior_rejection is not None: + prompt += ( + "\n\nYour previous proposal was REJECTED by the deterministic validator.\n" + f"Reason: {prior_rejection.reason}\n" + "Produce a REVISED SavingsProposal that resolves this." + ) return [Message(role="user", contents=[prompt])] @@ -97,13 +111,23 @@ async def generate_via_llm( *, max_attempts: int = 3, ) -> ValidatedProposal | Rejection: - """Async LLM path: non-streaming chat -> parse -> validate, with bounded retries and the - meter checked in this loop. A malformed/text-leaked reply is retried (never silently - accepted); a validation rejection is retried; returns ``ValidatedProposal | Rejection``; - raises ``BudgetExceeded`` when the meter cap is crossed.""" - messages = _build_messages(project, context) + """Async LLM path: non-streaming chat -> parse -> validate, with TWO bounded retry kinds, + the meter checked in this loop: - async def _fetch_parsed() -> SavingsProposal: + * malformed/text-leaked reply -> BLIND parse-retry (inner loop): re-fetch until the reply + parses; never silently accepted. + * validator rejection -> INFORMED refinement (outer ``max_attempts`` loop; Step 5, + målbilde §5/§7): the previous attempt's ``Rejection.reason`` is fed into the next + attempt's prompt (``_build_messages(prior_rejection=...)``) so the proposer can correct + rather than re-answer blindly. Bounded by ``max_attempts`` + the meter (no new loop; §6). + + The only per-attempt falsifier here is the deterministic validator (the numbers). The + checker is a run-level, one-shot signal (run.py, before generation); seeding generation + with the checker critique is separately scoped and NOT done here. Returns + ``ValidatedProposal | Rejection``; never a malformed proposal; raises ``BudgetExceeded`` + when the meter cap is crossed.""" + + async def _fetch_parsed(messages: list[Message]) -> SavingsProposal: # Parse-robust: a malformed/text-leaked reply is retried; the meter caps total work. while True: meter.tick_round() # between-attempt bound (BudgetExceeded over cap) @@ -116,7 +140,12 @@ async def generate_via_llm( last: Rejection | None = None for _ in range(max_attempts): - candidate = await _fetch_parsed() + # Informed refinement: feed the PREVIOUS attempt's validator rejection into this + # attempt's prompt. ``last`` is None on attempt 1 -> the unchanged base prompt; it is + # overwritten each round -> only the most-recent falsification ("forrige"), never an + # accumulated history (bounded prompt growth). + messages = _build_messages(project, context, prior_rejection=last) + candidate = await _fetch_parsed(messages) result = validate_proposal(candidate) if isinstance(result, ValidatedProposal): return result diff --git a/tests/test_step5_refine_loadbearing.py b/tests/test_step5_refine_loadbearing.py new file mode 100644 index 0000000..b7a0f7c --- /dev/null +++ b/tests/test_step5_refine_loadbearing.py @@ -0,0 +1,138 @@ +"""Step 5 load-bearing seam (målbilde §5/§7): informed refinement MUST feed the previous +attempt's falsification into the next hypothesis prompt. + +The gap (fot-i-bakken, verified in code): ``generate_via_llm``'s outer ``max_attempts`` loop +built the prompt ONCE and re-sent it identically — a BLIND retry. The validator's per-attempt +``Rejection.reason`` was captured in ``last`` but never reached the next prompt, so the model +re-answered the same question with no knowledge of *why* the prior proposal failed. Fase 4 +routes that reason into the next attempt's prompt, under the EXISTING meter/``max_attempts`` +cap (no new loop; §6). + +Scope note: the only PER-ATTEMPT falsifier inside this loop is the validator. The checker is a +run-level, one-shot signal (run.py, before generation); seeding generation with the checker +critique is separately scoped and NOT exercised here. + +These two tests are a load-bearing PAIR (per the Fase-2 green-but-dead trap): +- the positive test keys its proposer's flip on the validator REASON PAYLOAD (the rejected + claim value), not on a wrapper phrase or call-count, and asserts the reason reached the + next prompt VERBATIM. It goes RED if ``_build_messages`` stops injecting the reason (detach): + the flip token never arrives, so the outcome never flips AND the verbatim assertion fails. +- the control proves the loop stays BOUNDED: a proposer that never fixes its claim exhausts + exactly ``max_attempts`` and returns a ``Rejection`` — so the positive test's flip is caused + by the fed-back reason, not by mere retrying. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from typing import Any + +from agent_framework import ChatResponse, ChatResponseUpdate, Message +from spikes._harness import FakeChatClient, message_texts + +from portfolio_optimiser.budget import Budget, TokenMeter +from portfolio_optimiser.generate import generate_via_llm +from portfolio_optimiser.reference_domain import load_reference_projects +from portfolio_optimiser.validator import ( + Rejection, + ValidatedProposal, + proposal_for, + validate_proposal, +) + +# FV42-GSV-E1 cost codes 05.2 + 03.1 -> affected total 1_482_500 -> degenerate Monte Carlo +# P90 = 0.30 x 1_482_500 = 444_750. A claim of 800_000 is Pydantic-parseable (< affected total) +# but > P90 -> the validator REJECTS it (exercising the OUTER max_attempts path). A claim of +# 200_000 (<= P90) validates. Both proposals are built via proposal_for + model_dump_json, so +# the SUT parses EXACTLY what the test built -> the recomputed reason is byte-identical. +_CODES = ["05.2", "03.1"] +_BAD_CLAIM = 800_000 +_CORRECTED_CLAIM = 200_000 + + +def _meter() -> TokenMeter: + # max_rounds well above max_attempts so max_attempts -- not BudgetExceeded -- is the bound. + return TokenMeter(Budget(max_tokens=10**9, max_rounds=20)) + + +class _ReasonAwareChatClient(FakeChatClient): + """A proposer whose reply depends on the prompt: if the prompt carries ``flip_key`` (the + rejected claim value, present only once the validator's reason is fed back), it returns the + CORRECTED proposal; otherwise the BAD one. Records ``received_texts`` per call (inherited) + so the test can assert exactly what reached each attempt's prompt.""" + + def __init__(self, flip_key: str, bad_reply: str, corrected_reply: str) -> None: + super().__init__() + self._flip_key = flip_key + self._bad = bad_reply + self._corrected = corrected_reply + + def _inner_get_response( + self, *, messages: Sequence[Message], stream: bool, options: Any, **kwargs: Any + ) -> Any: + received = message_texts(messages) + self.received_texts.append(received) + self.call_count += 1 + reply = self._corrected if self._flip_key in " ".join(received) else self._bad + + if stream: + + async def _agen() -> Any: + yield ChatResponseUpdate( + role="assistant", contents=[{"type": "text", "text": reply}] + ) + + return self._build_response_stream(_agen()) + + async def _coro() -> ChatResponse: + return ChatResponse( + messages=[Message(role="assistant", contents=[reply])], response_id="fake" + ) + + return _coro() + + +async def test_refinement_feeds_prior_falsification_into_next_prompt() -> None: + """LOAD-BEARING: the validator's rejection from attempt 1 reaches attempt 2's prompt, so the + proposer can correct. Goes RED if ``_build_messages`` stops injecting ``prior_rejection.reason`` + (the flip token never arrives -> no validated outcome AND the verbatim assertion fails).""" + project = load_reference_projects()[0] # FV42-GSV-E1 + bad = proposal_for(project, _CODES, claimed_saving_nok=_BAD_CLAIM) + corrected = proposal_for(project, _CODES, claimed_saving_nok=_CORRECTED_CLAIM) + + # The reason is computed via the SAME validator the SUT uses, on the SAME proposal -> the + # reason string is byte-identical to the one the loop will feed back. flip_key is the claim + # value, guaranteed to appear in that reason. + rej = validate_proposal(bad) + assert isinstance(rej, Rejection), "fixture invariant: the BAD claim must reject" + flip_key = f"{bad.claimed_saving_nok:.0f}" + + client = _ReasonAwareChatClient(flip_key, bad.model_dump_json(), corrected.model_dump_json()) + # context="" so the flip token cannot pre-exist in attempt 1's prompt. + result = await generate_via_llm(client, project, "", _meter(), max_attempts=3) + + assert isinstance(result, ValidatedProposal), ( + "the proposer corrected on attempt 2 but the loop did not validate -- the falsification " + "never reached the next prompt" + ) + # Green-but-dead guard: the validator's reason reached attempt 2's prompt VERBATIM. This is + # what makes the seam load-bearing -- an impl that injects a wrapper but drops the reason + # leaves this RED. + assert rej.reason in " ".join(client.received_texts[1]), ( + "the validator's rejection reason did not reach the next attempt's prompt" + ) + assert client.call_count == 2 + + +async def test_refinement_loop_stays_bounded_when_never_fixed() -> None: + """CAUSALITY + BOUND CONTROL (§6): a proposer that never fixes its claim exhausts exactly + ``max_attempts`` and returns a Rejection -- proving the loop is bounded and that the positive + test's flip is caused by the fed-back reason, not by retrying alone.""" + project = load_reference_projects()[0] + bad = proposal_for(project, _CODES, claimed_saving_nok=_BAD_CLAIM) + client = FakeChatClient(default_reply=bad.model_dump_json()) + + result = await generate_via_llm(client, project, "", _meter(), max_attempts=3) + + assert isinstance(result, Rejection) + assert client.call_count == 3