feat(step5): the falsification that informed the next hypothesis now leaves the loop

generate_via_llm consumed each validator Rejection internally (`last`), fed it into the
next attempt's prompt, and dropped it. So Step 5 was real but unobservable: a caller could
see THAT a proposal validated, never that it validated on attempt 2 after the deterministic
validator falsified attempt 1. It was the one step of the eight with no output to show.

The seam is a typed return value -- GenerationResult(outcome, refinements) -- rather than an
out-parameter or a callback: a returned value cannot be silently lost by a caller that forgets
to pass a collector, and mypy forces every call site to acknowledge it.

refinements carries ONLY rejections that were actually fed back. When the attempt budget runs
out the final rejection IS outcome; counting it here would be double-counting, and the bounded
control test goes red on the collect-everything implementation that gets this wrong.

The loop's bound is untouched: max_attempts and meter.tick_round stand, and `last` still drives
the prompt alone, so prompt growth is unchanged. run.py accumulates across _evaluate calls, so
_evaluate_mandate is untouched; RunResult.refinements defaults (the coverage precedent) and is
concatenated across approaches rather than keyed per approach -- stated as an honesty limit.

The simulation now shows it: the scripted proposer overclaims 250000, which the validator
falsifies against P90 = 90000, and the corrected 30000 validates. Only the overclaim is
scripted -- the rejection is computed. scripted_factory takes a per-role reply selector so this
needs no second scripted client body.

README records the two accuracy changes only (Step 5 is now inspectable; the simulation trace
shows the correction). The level-2 publishing claim stays deferred until after the demo (O4).

Load-bearing MEASURED against the full suite with a control, four mutations all red:
detach the returned history (4 tests) - collect-everything (control only) - detach the run
wiring (2 tests) - revert the simulation's proposer to a constant (the demo-protection test).
Control: 759 passed / 4 skipped; ruff, format and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CcWFcREUi6YPjEpN3ACDP

src/portfolio_optimiser/simulation.py

@@ -51,12 +51,34 @@ def _default_bundle_dir() -> Path:
     return shared_root() / "examples" / "bygg-energi-mikro"
 
 
-# A VALID SavingsProposal for BYGG-KONTOR-NORD: total = 300000 x 1.0, P90 = 0.30 x 300000 = 90000,
-# claimed 30000 <= 90000 -> validates on the first attempt (no `assumptions` -> degenerate MC).
+# Two SavingsProposals for BYGG-KONTOR-NORD: total = 300000 x 1.0, so the degenerate Monte Carlo
+# P90 = 0.30 x 300000 = 90000 (no `assumptions`). The OVERCLAIMED one asks for 250000 — parseable,
+# and internally consistent, but above P90, so the DETERMINISTIC validator falsifies it. The
+# corrected one claims 30000 <= 90000 and validates. Together they drive Step 5 (informed
+# refinement): the proposer is scripted, but the rejection that turns proposal 1 into proposal 2 is
+# genuinely computed by the validator, not scripted.
+_OVERCLAIMED_PROPOSAL = (
+    '{"measure":"LED-retrofit av kontorbelysning","affected_items":'
+    '[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":250000}'
+)
 _VALID_PROPOSAL = (
     '{"measure":"LED-retrofit av kontorbelysning","affected_items":'
     '[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
 )
+# The flip key: the overclaimed figure, which the validator's rejection reason carries and
+# ``generate._build_messages`` appends to the NEXT attempt's prompt. Verified ABSENT from the demo
+# bundle, so it cannot pre-exist in attempt 1's prompt — the correction is caused by the
+# falsification travelling back, never by the proposer simply being asked twice.
+_REJECTED_CLAIM_KEY = "250000"
+
+
+def _proposer_reply(prompt: str, _role: str) -> str:
+    """The scripted proposer, keyed on PROMPT CONTENT (the canonical client's ``reply_selector``
+    seam): it overclaims until the validator's rejection comes back in the prompt, then corrects.
+    Stateless — no per-turn counter — so the debate turns and the generation attempts share it."""
+    return _VALID_PROPOSAL if _REJECTED_CLAIM_KEY in prompt else _OVERCLAIMED_PROPOSAL
+
+
 # The checker's debate turn ends with the gate marker the run parses (run._checker_verdict).
 _CHECKER_APPROVE = "Tallene er innenfor feasibelt område og resonnementet holder. VERDICT: APPROVE"
 
@@ -145,14 +167,24 @@ class ScriptedChatClient(OpenAIChatCompletionClient):
         return _coro()
 
 
-def scripted_factory(replies: dict[str, str], sink: list[str]) -> Callable[[str], BaseChatClient]:
+def scripted_factory(
+    replies: Mapping[str, str | Callable[[str, str], str]], sink: list[str]
+) -> Callable[[str], BaseChatClient]:
     """A role-keyed client factory: ``factory("proposer")`` and ``factory("checker")`` each return a
     fresh ``ScriptedChatClient`` with that role's reply, all sharing ONE ``sink``. MAF stamps the
     proposer/checker identity from the agent name, so role-keyed stateless replies suffice (no
-    per-turn counter); the shared ``sink`` spans the debate turns and the generation call."""
+    per-turn counter); the shared ``sink`` spans the debate turns and the generation call.
+
+    A role's value may be a constant reply OR a ``reply_selector`` over ``(prompt_blob, role)`` —
+    the canonical client's existing seam, passed straight through. That is what lets a role answer
+    DIFFERENTLY on a later attempt (Step 5: the proposer corrects once the validator's rejection
+    comes back in the prompt) without a per-turn counter and without a second scripted body."""
 
     def factory(role: str) -> BaseChatClient:
-        return ScriptedChatClient(replies[role], sink, role=role)
+        reply = replies[role]
+        if callable(reply):
+            return ScriptedChatClient(sink=sink, role=role, reply_selector=reply)
+        return ScriptedChatClient(reply, sink, role=role)
 
     return factory
 
@@ -212,7 +244,10 @@ async def simulate_learning_loop(
     copy = Path(work_dir) / "bundle"
     shutil.copytree(bundle_dir, copy)
     copy_s = str(copy)
-    replies = {"proposer": _VALID_PROPOSAL, "checker": _CHECKER_APPROVE}
+    replies: dict[str, str | Callable[[str, str], str]] = {
+        "proposer": _proposer_reply,
+        "checker": _CHECKER_APPROVE,
+    }
     verdict_input = {"decision": example.decision, "rationale": persona_rationale}
 
     # Run A — empty wiki isolates the persona's NEW knowledge.
@@ -283,6 +318,21 @@ def _outcome_line(result: RunResult) -> str:
     return f"REJECTED   ({o.reason})"
 
 
+def _refinement_lines(result: RunResult) -> list[str]:
+    """Step 5 made visible: every falsification that was fed back into a further hypothesis. Empty
+    when the first candidate validated — printing nothing is the honest output there."""
+    lines = []
+    for n, rejected in enumerate(result.refinements, start=1):
+        lines.append(
+            f"  steg 5 #{n} : REJECTED   (claimed "
+            f"{rejected.proposal.claimed_saving_nok:.0f} NOK) — {rejected.reason}"
+        )
+        lines.append(
+            "             -> grunnen mates tilbake i neste hypotese (bundet av max_attempts)"
+        )
+    return lines
+
+
 def main(argv: list[str] | None = None) -> int:  # pragma: no cover - console trace
     """Run the simulation against the energi bundle in a throwaway temp dir and print an honest,
     readable trace. Invoke: ``uv run python -m portfolio_optimiser.simulation``."""
@@ -299,6 +349,8 @@ def main(argv: list[str] | None = None) -> int:  # pragma: no cover - console tr
     print("=" * 78)
 
     print("\nRUN A  (fresh wiki — no prior verdicts)")
+    for line in _refinement_lines(result.run_a):
+        print(line)
     print(f"  validator : {_outcome_line(result.run_a)}")
     print(f"  checker   : VERDICT={result.run_a.checker_verdict.upper()}")
     print(f"  persona   : {result.run_a.verdict.decision}  ->  {result.run_a.verdict.rationale}")
@@ -310,6 +362,8 @@ def main(argv: list[str] | None = None) -> int:  # pragma: no cover - console tr
     print(f"  wrote     : {result.promoted_path.name}  (linked into index.md, neutral label)")
 
     print("\nRUN B  (re-seeded wiki — reads the promoted verdict)")
+    for line in _refinement_lines(result.run_b):
+        print(line)
     print(f"  validator : {_outcome_line(result.run_b)}")
     print(
         f"  prompt has marker '{result.marker}': {result.marker_in_run_b_prompt}  (expected True)"