feat(step5): the falsification that informed the next hypothesis now leaves the loop

generate_via_llm consumed each validator Rejection internally (`last`), fed it into the
next attempt's prompt, and dropped it. So Step 5 was real but unobservable: a caller could
see THAT a proposal validated, never that it validated on attempt 2 after the deterministic
validator falsified attempt 1. It was the one step of the eight with no output to show.

The seam is a typed return value -- GenerationResult(outcome, refinements) -- rather than an
out-parameter or a callback: a returned value cannot be silently lost by a caller that forgets
to pass a collector, and mypy forces every call site to acknowledge it.

refinements carries ONLY rejections that were actually fed back. When the attempt budget runs
out the final rejection IS outcome; counting it here would be double-counting, and the bounded
control test goes red on the collect-everything implementation that gets this wrong.

The loop's bound is untouched: max_attempts and meter.tick_round stand, and `last` still drives
the prompt alone, so prompt growth is unchanged. run.py accumulates across _evaluate calls, so
_evaluate_mandate is untouched; RunResult.refinements defaults (the coverage precedent) and is
concatenated across approaches rather than keyed per approach -- stated as an honesty limit.

The simulation now shows it: the scripted proposer overclaims 250000, which the validator
falsifies against P90 = 90000, and the corrected 30000 validates. Only the overclaim is
scripted -- the rejection is computed. scripted_factory takes a per-role reply selector so this
needs no second scripted client body.

README records the two accuracy changes only (Step 5 is now inspectable; the simulation trace
shows the correction). The level-2 publishing claim stays deferred until after the demo (O4).

Load-bearing MEASURED against the full suite with a control, four mutations all red:
detach the returned history (4 tests) - collect-everything (control only) - detach the run
wiring (2 tests) - revert the simulation's proposer to a constant (the demo-protection test).
Control: 759 passed / 4 skipped; ruff, format and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CcWFcREUi6YPjEpN3ACDP

CHANGELOG.md

@@ -5,6 +5,21 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+- Step 5 is now observable: `generate_via_llm` returns a `GenerationResult` carrying the validator
+  falsifications that informed a later attempt, surfaced on `RunResult.refinements`. The offline
+  simulation exercises it — the scripted proposer overclaims, the deterministic validator falsifies
+  the number, and the refined proposal validates.
+
+### Changed
+- **Breaking (library API):** `generate_via_llm` returns `GenerationResult` instead of
+  `ValidatedProposal | Rejection`; read `.outcome` for the previous value. The refinement loop's
+  bound is unchanged (`max_attempts` + token meter).
+- `simulation.scripted_factory` accepts a per-role reply *selector* over `(prompt, role)` as well as
+  a constant reply, so a scripted role can answer differently on a later attempt.
+
 ## [0.1.0] - 2026-08-06
 
 First tagged release. There is no prior release, so the entries below describe what this version

CLAUDE.md

@@ -112,6 +112,23 @@ Python ≥3.10. MAF (`agent-framework-core` 1.9.0). Pakkehåndtering: `uv`. To b
   her) — så koden påstår ikke mer enn den gjør. Load-bearing:
   `tests/test_step5_refine_loadbearing.py` blir rød når reason-injeksjonen detaches (utfallet
   flipper aldri + reason-verbatim-asserten faller); kontrollen beviser at løkka forblir bundet.
+- **Falsifiserings-historikken FORLATER generate-løkka som typet returverdi (Steg 5, del 2):**
+  `generate_via_llm` returnerer `GenerationResult(outcome, refinements)` — ikke lenger bare
+  `ValidatedProposal | Rejection`. Før dette forbrukte løkka hver `Rejection` internt (`last`) og
+  DROPPET den, så Steg 5 var det ene av åtte steg uten observerbart utfall. **Returverdi, ikke
+  out-parameter/callback:** en returnert verdi kan ikke bli stille tapt av en kaller som glemmer å
+  sende en samler, og mypy tvinger hvert kallsted til å ta stilling. **`refinements` bærer KUN
+  avvisninger som faktisk ble matet tilbake** i et senere forsøks prompt — ved uttømt budsjett ER
+  den siste avvisningen `outcome`, den informerte ingenting, og å telle den med ville vært
+  dobbeltføring (en «samle alt»-implementasjon består den positive testen og faller på kontrollen).
+  Taket er URØRT: `max_attempts` + `meter.tick_round` står, og `last` driver fortsatt prompten alene
+  (prompt-veksten er uendret). `run.py` akkumulerer på tvers av `_evaluate`-kallene, så
+  `_evaluate_mandate` er urørt; `RunResult.refinements` er defaultet (`coverage`-presedensen), og
+  med mandat er den KONKATENERT på tvers av tiltak, ikke nøklet per tiltak (uttalt ærlighets-grense).
+  `scripted_factory` tar nå `str | reply_selector` per rolle, så simuleringens proposer korrigerer
+  seg innholds-nøklet uten en andre scriptet kropp. Load-bearing MÅLT
+  (`tests/test_step5_history_loadbearing.py`), fire mutasjoner: detach returneringen · samle-alt ·
+  detach run-wiringen · reverter simuleringens proposer til konstant svar.
 - **Lang/async fil-løkke (Steg 7, målbilde §3/§7):** `run_project(verdict_dir=...)` er den lange
   tilbakemeldings-tidsskalaen — en ekspert/persona dropper en verdict-fil (vanlig JSON, RAW-laget
   per §10 R2) i en inbox-mappe ETTER en kjøring, og en separat, senere kjøring `load_verdicts_from_dir`

README.md

@@ -57,8 +57,10 @@ demonstrably informed by the first:
 uv run python -m portfolio_optimiser.simulation
 ```
 
-The trace ends with the approved verdict's marker present in Run B's prompt and absent from Run A's
+Each run shows the refinement step: the proposer's first claim is falsified by the deterministic
-— knowledge crossing runs purely through the file-backed wiki (promote → re-seed → fold).
+validator, and the corrected claim validates. The trace then ends with the approved verdict's marker
+present in Run B's prompt and absent from Run A's — knowledge crossing runs purely through the
+file-backed wiki (promote → re-seed → fold).
 
 **3 — Run the loop over a knowledge base, with answers you supply.** Write the stand-in replies,
 then point the CLI at the bundle:
@@ -262,7 +264,9 @@ One run, one project, eight steps — with the learning loop closing across runs
    is anchored to the project's declared cost baseline, so a proposal cannot invent the cost
    lines it claims to save against.
 5. **Refine** — a rejected attempt retries *informed* by the rejection reason, under hard
-   attempt and token caps. Unbounded loops are forbidden everywhere.
+   attempt and token caps. Unbounded loops are forbidden everywhere. The falsifications that
+   informed a later attempt are surfaced on the result (`RunResult.refinements`), so what the
+   run corrected in response to is inspectable, not just what it ended up with.
 6. **Propose or discard** — a validated proposal with risk percentiles, or a typed rejection.
 7. **Expert feedback** — days later, an expert drops a verdict file in an inbox folder; a
    later run picks it up. Fully resumable; no live session assumed.

docs/plan/2026-08-06-demo-uke-plan.md

@@ -45,7 +45,7 @@ Dette er ikke en unnskyldning som svekker demoen — det er selve grunnregelen r
 | 2 — Hypotese | forslaget med parametere | data finnes, printes ikke | `print` |
 | 3 — Debatt (maker-checker) | begge deltakere + `checker_verdict` | checker printes | `print` |
 | 4 — Valider / falsifiser | validator-linja med P90 | **printes** ✔ | — |
-| 5 — Forbedre, informert og bundet | avvisning → korrigert forslag | **ikke mulig** | **BYGG** |
+| 5 — Forbedre, informert og bundet | avvisning → korrigert forslag | **bygget 7. aug** ✔ | — |
 | 6 — Forkast eller foreslå | typet `outcome` | data finnes, printes ikke | `print` |
 | 7 — Svar på tilbakemelding | persona-dom + fil-innboksen | dommen printes | `print` |
 | 8 — Promoter godkjent kunnskap | promotert fil + index-lenke | **printes** ✔ | — |
@@ -58,12 +58,20 @@ blir avvist og deretter korrigert.
 
 ## 3. Dagsplan
 
-**Fredag 7. august — steg 5-sømmen (den ene kodejobben).**
+**Fredag 7. august — steg 5-sømmen (den ene kodejobben). ✔ GJORT.**
 Slipp avvisnings-historikken ut av `generate_via_llm` uten å endre løkkas tak (`max_attempts` +
 `meter.tick_round` står urørt — «forbedre til god nok» uten tak er forbudt). Nytt skriptet
 avvis-så-korriger-forløp. Ny load-bearing-test: RØD når sømmen kobles fra. Ligger først i uka
 med vilje — det er den eneste jobben som kan overraske, og den har fem dagers slakk bak seg.
 
+> **Utfall:** den åpne beslutningen ble **egen returtype** — `generate_via_llm` returnerer
+> `GenerationResult(outcome, refinements)`, og `RunResult.refinements` bærer den ut av kjøringen.
+> En returverdi kan ikke bli stille tapt slik en out-parameter kan, og mypy tvinger hvert kallsted
+> til å ta stilling. `refinements` bærer KUN avvisninger som faktisk ble matet tilbake (den siste
+> avvisningen ved uttømt budsjett ER `outcome`). Simuleringens proposer overklager 250 000 NOK, som
+> den deterministiske validatoren felt mot P90 = 90 000; det korrigerte forslaget på 30 000
+> validerer. Fire mutasjoner målt røde mot hele suiten, med kontroll.
+
 **Lørdag 8. – søndag 9. august — presentasjonslaget.**
 De fem `print`-tilleggene over, formet som én lesbar gjennomgang med steg-nummer i margen.
 Ingen ny logikk. Målet er at en tilhører kan følge hvert steg uten at du forklarer hva de ser på.

src/portfolio_optimiser/generate.py

@@ -15,14 +15,15 @@ Two entry points, because the LLM call is async while ``validator.self_repair``
   attempts. Used for deterministic candidate sources.
 * ``generate_via_llm`` — the ASYNC LLM path: an async mirror of the same bounded retry that
   awaits the chat call (parse-retry inside the meter budget, then ``validate_proposal``).
-  Returns ``ValidatedProposal | Rejection``; never a malformed proposal; raises
+  Returns a ``GenerationResult`` (the outcome PLUS the falsifications that informed it); never a
-  ``BudgetExceeded`` when the meter cap is crossed.
+  malformed proposal; raises ``BudgetExceeded`` when the meter cap is crossed.
 """
 
 from __future__ import annotations
 
 import json
 from collections.abc import Callable
+from dataclasses import dataclass, field
 
 from agent_framework import BaseChatClient, Message
 from pydantic import ValidationError
@@ -43,6 +44,28 @@ class GenerationError(RuntimeError):
     """No parseable proposal could be produced within the attempt budget."""
 
 
+@dataclass(frozen=True)
+class GenerationResult:
+    """What one ``generate_via_llm`` call produced: the outcome, and the falsification history that
+    informed it (Step 5, målbilde §5/§7).
+
+    A TYPED RETURN VALUE rather than an out-parameter or a callback, deliberately: the informed
+    refinement loop already computed this history internally and then dropped it, so Step 5 was the
+    one step of the eight with no observable output. A returned value cannot be silently lost by a
+    caller that forgets to pass a collector, and it forces every call site to acknowledge the seam.
+
+    ``refinements`` holds ONLY the rejections that were actually fed back into a later attempt's
+    prompt — the honest reading of "informed refinement". When the attempt budget runs out, the
+    final rejection IS ``outcome``: it informed nothing and is not repeated here. So the total
+    number of validator falsifications this call produced is ``len(refinements)`` plus one when
+    ``outcome`` is itself a ``Rejection``. It is empty on the common single-attempt path, which is
+    honest rather than merely convenient: nothing was falsified, so there is nothing to show.
+    """
+
+    outcome: ValidatedProposal | Rejection
+    refinements: tuple[Rejection, ...] = field(default=())
+
+
 def _build_messages(
     project: Project,
     context: str,
@@ -140,7 +163,7 @@ async def generate_via_llm(
     max_attempts: int = 3,
     baseline: CostBaseline | None = None,
     approach: Approach | None = None,
-) -> ValidatedProposal | Rejection:
+) -> GenerationResult:
     """Async LLM path: non-streaming chat -> parse -> validate, with TWO bounded retry kinds,
     the meter checked in this loop:
 
@@ -165,8 +188,12 @@ async def generate_via_llm(
     ``baseline`` (S4.0) is handed straight to ``validate_proposal``, so a fabricated cost line is
     falsified per ATTEMPT like any other rejection — and its reason feeds the next attempt's prompt
     through the SAME informed-refinement path (Step 5), which is why no new loop appears here.
-    Returns
+
-    ``ValidatedProposal | Rejection``; never a malformed proposal; raises ``BudgetExceeded``
+    Returns a ``GenerationResult``: the ``ValidatedProposal | Rejection`` outcome plus every
+    rejection that was fed back into a later attempt's prompt. Surfacing that history changes
+    nothing about the loop's BOUND — ``max_attempts`` and ``meter.tick_round`` are exactly as
+    before ("refine until good enough" without a cap stays forbidden, §6); it only stops the loop
+    from discarding what it already knew. Never a malformed proposal; raises ``BudgetExceeded``
     when the meter cap is crossed."""
 
     async def _fetch_parsed(messages: list[Message]) -> SavingsProposal:
@@ -181,16 +208,25 @@ async def generate_via_llm(
                 continue
 
     last: Rejection | None = None
+    # The falsifications that were FED BACK, in attempt order. ``last`` still drives the PROMPT and
+    # is still overwritten each round -- only the most-recent falsification reaches the model, so
+    # prompt growth is unchanged. This list is a record for the CALLER, appended to only once a
+    # rejection is about to inform a further attempt; it is never read back into a prompt.
+    fed_back: list[Rejection] = []
     for _ in range(max_attempts):
         # Informed refinement: feed the PREVIOUS attempt's validator rejection into this
         # attempt's prompt. ``last`` is None on attempt 1 -> the unchanged base prompt; it is
         # overwritten each round -> only the most-recent falsification ("forrige"), never an
         # accumulated history (bounded prompt growth).
+        if last is not None:
+            fed_back.append(last)
         messages = _build_messages(project, context, prior_rejection=last, approach=approach)
         candidate = await _fetch_parsed(messages)
         result = validate_proposal(candidate, baseline=baseline)
         if isinstance(result, ValidatedProposal):
-            return result
+            return GenerationResult(outcome=result, refinements=tuple(fed_back))
         last = result
     assert last is not None  # max_attempts >= 1, so at least one validation ran
-    return last  # validation never passed within the attempt budget -> typed Rejection
+    # Validation never passed within the attempt budget -> typed Rejection. ``last`` is the outcome
+    # and was never fed back, so it is deliberately absent from ``refinements``.
+    return GenerationResult(outcome=last, refinements=tuple(fed_back))

src/portfolio_optimiser/run.py

@@ -131,6 +131,15 @@ class RunResult:
     #: report is honest there, because nothing was ordered. It defaults so every existing
     #: constructor call and every frozen aggregate over ``RunResult`` is unaffected.
     coverage: tuple[ApproachOutcome, ...] = ()
+    #: Step 5 (målbilde §5/§7): the validator falsifications that informed a LATER generation
+    #: attempt, in attempt order — what ``generate_via_llm`` corrected in response to, rather than
+    #: only what it ended up with. EMPTY on the common path where the first candidate validates:
+    #: nothing was falsified, so there is nothing to show. Honesty limit: with a mandate this is
+    #: the run's refinements CONCATENATED across every commissioned approach, not keyed per
+    #: approach — ``coverage`` is the per-approach report, and hanging proposals off its rows is
+    #: what ``_evaluate_mandate`` deliberately avoids. It defaults, so every existing constructor
+    #: call is unaffected (mirrors ``coverage``).
+    refinements: tuple[Rejection, ...] = ()
 
 
 @dataclass(frozen=True)
@@ -635,10 +644,18 @@ async def run_project(
     # each under the SAME meter — no new unbounded loop; the caps already in force are the bound.
     proposer_client = factory("proposer")
 
+    # Step 5 (målbilde §5/§7): generation now returns its falsification history alongside the
+    # outcome. ``_evaluate`` keeps its ``ValidatedProposal | Rejection`` shape so ``_evaluate_mandate``
+    # is untouched, and the history is accumulated here in call order — one entry per approach that
+    # needed correcting, concatenated (see ``RunResult.refinements`` for that honesty limit).
+    refinements: list[Rejection] = []
+
     async def _evaluate(approach: Approach | None) -> ValidatedProposal | Rejection:
-        return await generate_via_llm(
+        generated = await generate_via_llm(
             proposer_client, project, gen_context, meter, baseline=baseline, approach=approach
         )
+        refinements.extend(generated.refinements)
+        return generated.outcome
 
     coverage: tuple[ApproachOutcome, ...] = ()
     evaluated: tuple[tuple[str, ValidatedProposal | Rejection], ...] = ()
@@ -774,6 +791,7 @@ async def run_project(
         debate_output=debate_output,
         checker_verdict=checker_decision,
         coverage=coverage,
+        refinements=tuple(refinements),
     )
 
 

src/portfolio_optimiser/simulation.py

@@ -51,12 +51,34 @@ def _default_bundle_dir() -> Path:
     return shared_root() / "examples" / "bygg-energi-mikro"
 
 
-# A VALID SavingsProposal for BYGG-KONTOR-NORD: total = 300000 x 1.0, P90 = 0.30 x 300000 = 90000,
+# Two SavingsProposals for BYGG-KONTOR-NORD: total = 300000 x 1.0, so the degenerate Monte Carlo
-# claimed 30000 <= 90000 -> validates on the first attempt (no `assumptions` -> degenerate MC).
+# P90 = 0.30 x 300000 = 90000 (no `assumptions`). The OVERCLAIMED one asks for 250000 — parseable,
+# and internally consistent, but above P90, so the DETERMINISTIC validator falsifies it. The
+# corrected one claims 30000 <= 90000 and validates. Together they drive Step 5 (informed
+# refinement): the proposer is scripted, but the rejection that turns proposal 1 into proposal 2 is
+# genuinely computed by the validator, not scripted.
+_OVERCLAIMED_PROPOSAL = (
+    '{"measure":"LED-retrofit av kontorbelysning","affected_items":'
+    '[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":250000}'
+)
 _VALID_PROPOSAL = (
     '{"measure":"LED-retrofit av kontorbelysning","affected_items":'
     '[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
 )
+# The flip key: the overclaimed figure, which the validator's rejection reason carries and
+# ``generate._build_messages`` appends to the NEXT attempt's prompt. Verified ABSENT from the demo
+# bundle, so it cannot pre-exist in attempt 1's prompt — the correction is caused by the
+# falsification travelling back, never by the proposer simply being asked twice.
+_REJECTED_CLAIM_KEY = "250000"
+
+
+def _proposer_reply(prompt: str, _role: str) -> str:
+    """The scripted proposer, keyed on PROMPT CONTENT (the canonical client's ``reply_selector``
+    seam): it overclaims until the validator's rejection comes back in the prompt, then corrects.
+    Stateless — no per-turn counter — so the debate turns and the generation attempts share it."""
+    return _VALID_PROPOSAL if _REJECTED_CLAIM_KEY in prompt else _OVERCLAIMED_PROPOSAL
+
+
 # The checker's debate turn ends with the gate marker the run parses (run._checker_verdict).
 _CHECKER_APPROVE = "Tallene er innenfor feasibelt område og resonnementet holder. VERDICT: APPROVE"
 
@@ -145,14 +167,24 @@ class ScriptedChatClient(OpenAIChatCompletionClient):
         return _coro()
 
 
-def scripted_factory(replies: dict[str, str], sink: list[str]) -> Callable[[str], BaseChatClient]:
+def scripted_factory(
+    replies: Mapping[str, str | Callable[[str, str], str]], sink: list[str]
+) -> Callable[[str], BaseChatClient]:
     """A role-keyed client factory: ``factory("proposer")`` and ``factory("checker")`` each return a
     fresh ``ScriptedChatClient`` with that role's reply, all sharing ONE ``sink``. MAF stamps the
     proposer/checker identity from the agent name, so role-keyed stateless replies suffice (no
-    per-turn counter); the shared ``sink`` spans the debate turns and the generation call."""
+    per-turn counter); the shared ``sink`` spans the debate turns and the generation call.
+
+    A role's value may be a constant reply OR a ``reply_selector`` over ``(prompt_blob, role)`` —
+    the canonical client's existing seam, passed straight through. That is what lets a role answer
+    DIFFERENTLY on a later attempt (Step 5: the proposer corrects once the validator's rejection
+    comes back in the prompt) without a per-turn counter and without a second scripted body."""
 
     def factory(role: str) -> BaseChatClient:
-        return ScriptedChatClient(replies[role], sink, role=role)
+        reply = replies[role]
+        if callable(reply):
+            return ScriptedChatClient(sink=sink, role=role, reply_selector=reply)
+        return ScriptedChatClient(reply, sink, role=role)
 
     return factory
 
@@ -212,7 +244,10 @@ async def simulate_learning_loop(
     copy = Path(work_dir) / "bundle"
     shutil.copytree(bundle_dir, copy)
     copy_s = str(copy)
-    replies = {"proposer": _VALID_PROPOSAL, "checker": _CHECKER_APPROVE}
+    replies: dict[str, str | Callable[[str, str], str]] = {
+        "proposer": _proposer_reply,
+        "checker": _CHECKER_APPROVE,
+    }
     verdict_input = {"decision": example.decision, "rationale": persona_rationale}
 
     # Run A — empty wiki isolates the persona's NEW knowledge.
@@ -283,6 +318,21 @@ def _outcome_line(result: RunResult) -> str:
     return f"REJECTED   ({o.reason})"
 
 
+def _refinement_lines(result: RunResult) -> list[str]:
+    """Step 5 made visible: every falsification that was fed back into a further hypothesis. Empty
+    when the first candidate validated — printing nothing is the honest output there."""
+    lines = []
+    for n, rejected in enumerate(result.refinements, start=1):
+        lines.append(
+            f"  steg 5 #{n} : REJECTED   (claimed "
+            f"{rejected.proposal.claimed_saving_nok:.0f} NOK) — {rejected.reason}"
+        )
+        lines.append(
+            "             -> grunnen mates tilbake i neste hypotese (bundet av max_attempts)"
+        )
+    return lines
+
+
 def main(argv: list[str] | None = None) -> int:  # pragma: no cover - console trace
     """Run the simulation against the energi bundle in a throwaway temp dir and print an honest,
     readable trace. Invoke: ``uv run python -m portfolio_optimiser.simulation``."""
@@ -299,6 +349,8 @@ def main(argv: list[str] | None = None) -> int:  # pragma: no cover - console tr
     print("=" * 78)
 
     print("\nRUN A  (fresh wiki — no prior verdicts)")
+    for line in _refinement_lines(result.run_a):
+        print(line)
     print(f"  validator : {_outcome_line(result.run_a)}")
     print(f"  checker   : VERDICT={result.run_a.checker_verdict.upper()}")
     print(f"  persona   : {result.run_a.verdict.decision}  ->  {result.run_a.verdict.rationale}")
@@ -310,6 +362,8 @@ def main(argv: list[str] | None = None) -> int:  # pragma: no cover - console tr
     print(f"  wrote     : {result.promoted_path.name}  (linked into index.md, neutral label)")
 
     print("\nRUN B  (re-seeded wiki — reads the promoted verdict)")
+    for line in _refinement_lines(result.run_b):
+        print(line)
     print(f"  validator : {_outcome_line(result.run_b)}")
     print(
         f"  prompt has marker '{result.marker}': {result.marker_in_run_b_prompt}  (expected True)"

tests/test_generate.py

@@ -34,14 +34,14 @@ def _meter() -> TokenMeter:
 async def test_wellformed_reply_yields_validated_proposal(project) -> None:
     client = FakeChatClient(scripted=[_VALID], default_reply=_VALID)
     result = await generate_via_llm(client, project, "", _meter(), max_attempts=3)
-    assert isinstance(result, ValidatedProposal)
+    assert isinstance(result.outcome, ValidatedProposal)
-    assert isinstance(result.proposal, SavingsProposal)
+    assert isinstance(result.outcome.proposal, SavingsProposal)
 
 
 async def test_malformed_reply_is_retried_not_silently_accepted(project) -> None:
     client = FakeChatClient(scripted=["not json at all {{{", _VALID], default_reply=_VALID)
     result = await generate_via_llm(client, project, "", _meter(), max_attempts=3)
-    assert isinstance(result, ValidatedProposal)  # the malformed reply was NOT accepted
+    assert isinstance(result.outcome, ValidatedProposal)  # the malformed reply was NOT accepted
     assert client.call_count >= 2  # it retried past the malformed reply
 
 

tests/test_mandate_generation_loadbearing.py

@@ -85,7 +85,7 @@ async def test_commissioned_approach_gets_no_validator_discount(project) -> None
     result = await generate_via_llm(
         client, project, "", _meter(), max_attempts=1, approach=_APPROACH
     )
-    assert isinstance(result, Rejection)
+    assert isinstance(result.outcome, Rejection)
 
 
 async def test_commissioned_approach_still_validates_when_the_numbers_hold(project) -> None:
@@ -96,5 +96,5 @@ async def test_commissioned_approach_still_validates_when_the_numbers_hold(proje
     result = await generate_via_llm(
         client, project, "", _meter(), max_attempts=1, approach=_APPROACH
     )
-    assert isinstance(result, ValidatedProposal)
+    assert isinstance(result.outcome, ValidatedProposal)
     assert _APPROACH.label in client.received_texts[0][0]  # it went through the commissioned prompt

tests/test_step5_history_loadbearing.py

@@ -0,0 +1,203 @@
+"""Step 5 load-bearing seam, part 2 (målbilde §5/§7): the intermediate falsification history must
+ESCAPE ``generate_via_llm`` and reach the run's result — otherwise Step 5 is real but invisible.
+
+The gap (verified in code before this file existed): the informed-refinement loop captured each
+attempt's ``Rejection`` in a local ``last``, fed it into the next prompt, and then **dropped it**.
+``generate_via_llm`` returned only the final ``ValidatedProposal | Rejection``, so a caller could
+observe THAT a proposal validated but never that it validated *on attempt 2, after the deterministic
+validator falsified attempt 1*. Step 5 was the one step of the eight with no observable output.
+
+The seam is a typed return value (``GenerationResult``), not an out-parameter or a callback: a
+returned value cannot be silently dropped by a caller that forgets to pass a collector, and mypy
+forces every call site to acknowledge it. Nothing about the loop's BOUND changes — ``max_attempts``
+and ``meter.tick_round`` are untouched (§6: "refine until good enough" without a cap is forbidden).
+
+**The honesty line these tests pin** — ``refinements`` holds only the rejections that were actually
+FED BACK into a later attempt's prompt. When the attempt budget runs out, the final rejection IS
+``outcome``; it informed nothing and must not be double-counted as a refinement. An implementation
+that simply collects every rejection it sees passes the positive test and FAILS the bounded control,
+which is exactly why the control is here.
+
+Four tests, load-bearing as a set:
+- the history reaches the caller carrying the SAME rejection the next prompt received (RED if the
+  seam is detached, i.e. the history is computed internally and dropped again);
+- the bounded control pins fed-back-only (RED on a collect-everything implementation);
+- the run-level wiring: ``RunResult.refinements`` carries it out of ``run_project`` (RED if run.py
+  drops what generation returned — the seam would exist but the demo still could not show Step 5);
+- the simulation genuinely exercises it (RED if the scripted proposer reverts to a constant reply,
+  which would leave the seam built but the demo one step shorter, silently).
+
+They drive the CANONICAL ``ScriptedChatClient`` through its ``reply_selector`` seam rather than
+defining another ``_inner_get_response`` body (S2.5 consolidation), so the proposer used here is the
+same content-keyed one the simulation uses.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from portfolio_optimiser.budget import Budget, TokenMeter
+from portfolio_optimiser.generate import generate_via_llm
+from portfolio_optimiser.reference_domain import Project, load_reference_projects
+from portfolio_optimiser.run import RunResult, run_project
+from portfolio_optimiser.simulation import (
+    ScriptedChatClient,
+    scripted_factory,
+    simulate_learning_loop,
+)
+from portfolio_optimiser.validator import (
+    Rejection,
+    ValidatedProposal,
+    proposal_for,
+    validate_proposal,
+)
+
+# Same fixture arithmetic as test_step5_refine_loadbearing: FV42-GSV-E1 codes 05.2 + 03.1 ->
+# affected total 1_482_500 -> degenerate Monte Carlo P90 = 444_750. 800_000 parses (< affected
+# total) but exceeds P90 -> rejected; 200_000 validates. Both proposals are built via proposal_for,
+# so their quantity/unit_cost ARE the project's cost lines and the S4.0 baseline stage passes.
+_CODES = ["05.2", "03.1"]
+_BAD_CLAIM = 800_000
+_CORRECTED_CLAIM = 200_000
+
+_BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
+
+
+def _meter() -> TokenMeter:
+    # max_rounds well above max_attempts so max_attempts -- not BudgetExceeded -- is the bound.
+    return TokenMeter(Budget(max_tokens=10**9, max_rounds=20))
+
+
+def _fixture() -> tuple[Project, str, str, str, Rejection]:
+    """The shared reject-then-correct fixture: the bad proposal's JSON, the corrected one's, the
+    flip key (the rejected claim value, which appears ONLY once the validator's reason is fed back),
+    and the rejection the validator itself produces for the bad proposal — computed with the SAME
+    validator the SUT uses, so the reason is byte-identical to the one the loop feeds back."""
+    project = load_reference_projects()[0]  # FV42-GSV-E1
+    bad = proposal_for(project, _CODES, claimed_saving_nok=_BAD_CLAIM)
+    corrected = proposal_for(project, _CODES, claimed_saving_nok=_CORRECTED_CLAIM)
+    rej = validate_proposal(bad)
+    assert isinstance(rej, Rejection), "fixture invariant: the BAD claim must reject"
+    return (
+        project,
+        bad.model_dump_json(),
+        corrected.model_dump_json(),
+        f"{bad.claimed_saving_nok:.0f}",
+        rej,
+    )
+
+
+async def test_falsification_history_escapes_the_generate_loop() -> None:
+    """LOAD-BEARING: the rejection that informed attempt 2 is RETURNED to the caller, carrying the
+    verbatim reason attempt 2's prompt received. Goes RED when the seam is detached (the history is
+    computed internally and dropped again)."""
+    project, bad_json, corrected_json, flip_key, rej = _fixture()
+
+    sink: list[str] = []
+    client = ScriptedChatClient(
+        sink=sink,
+        reply_selector=lambda prompt, _role: corrected_json if flip_key in prompt else bad_json,
+    )
+    # context="" so the flip token cannot pre-exist in attempt 1's prompt.
+    result = await generate_via_llm(client, project, "", _meter(), max_attempts=3)
+
+    assert isinstance(result.outcome, ValidatedProposal), (
+        "fixture invariant: the proposer corrects on attempt 2 and that proposal validates"
+    )
+    # The seam itself: attempt 1's falsification is observable from OUTSIDE the loop.
+    assert len(result.refinements) == 1, (
+        "the falsification that informed attempt 2 did not escape generate_via_llm"
+    )
+    # Green-but-dead guard: it is the REAL rejection (reason + the rejected proposal), not a
+    # placeholder -- and it is byte-identical to what the next prompt was given.
+    assert result.refinements[0].reason == rej.reason
+    assert result.refinements[0].proposal.claimed_saving_nok == _BAD_CLAIM
+    assert result.refinements[0].reason in sink[1], (
+        "the returned rejection is not the one that was fed into the next attempt's prompt"
+    )
+
+
+async def test_history_holds_only_fed_back_rejections_when_budget_runs_out() -> None:
+    """CONTROL + HONESTY LINE: a proposer that never fixes its claim produces ``max_attempts``
+    rejections, of which only the first ``max_attempts - 1`` were ever fed into a later prompt. The
+    final one IS ``outcome``. RED on a collect-everything implementation, which is the obvious wrong
+    way to build this seam."""
+    project, bad_json, _corrected, _flip, _rej = _fixture()
+
+    client = ScriptedChatClient(bad_json)
+    result = await generate_via_llm(client, project, "", _meter(), max_attempts=3)
+
+    assert isinstance(result.outcome, Rejection)
+    assert client.call_count == 3, "control: the loop stays bounded by max_attempts"
+    assert len(result.refinements) == 2, (
+        "only the rejections that INFORMED a later attempt belong in the history; the final "
+        "rejection is the outcome and informed nothing"
+    )
+    assert all(r.reason == result.outcome.reason for r in result.refinements)
+
+
+async def test_run_result_carries_the_falsification_history(tmp_path: Path) -> None:
+    """RUN-LEVEL WIRING: what generation returns must survive to ``RunResult`` — otherwise the seam
+    exists but Step 5 is still invisible to the demo. RED if run.py drops ``refinements``.
+
+    Drives the ROAD path (a reference project, so the S4.0 baseline is always anchored) with the
+    same content-keyed proposer the simulation uses."""
+    project, bad_json, corrected_json, flip_key, _rej = _fixture()
+
+    docs = tmp_path / "docs"
+    docs.mkdir()
+    (docs / "kilde.md").write_text("Cost saving measure candidates for the project.\n", "utf-8")
+
+    sink: list[str] = []
+    result = await run_project(
+        project.id,
+        "local",
+        docs_dir=str(docs),
+        verdict_input={"decision": "approved", "rationale": "fixture verdict"},
+        client_factory=scripted_factory(
+            {
+                "proposer": lambda prompt, _role: (
+                    corrected_json if flip_key in prompt else bad_json
+                ),
+                "checker": "VERDICT: APPROVE",
+            },
+            sink,
+        ),
+        max_rounds=3,
+    )
+
+    assert isinstance(result, RunResult)
+    assert isinstance(result.outcome, ValidatedProposal), (
+        "fixture invariant: the corrected proposal validates"
+    )
+    assert len(result.refinements) == 1, (
+        "run_project dropped the falsification history returned by generation"
+    )
+    assert result.refinements[0].proposal.claimed_saving_nok == _BAD_CLAIM
+
+
+async def test_simulation_actually_exercises_step_five(tmp_path: Path) -> None:
+    """DEMO PROTECTION: the offline simulation must genuinely go through a falsification before it
+    validates — otherwise Step 5 is buildable but unshown, which is the state this whole seam exists
+    to leave behind. RED the moment the scripted proposer reverts to a constant reply: with nothing
+    to correct, ``refinements`` is empty and the demo silently loses a step.
+
+    The rejection itself is NOT scripted: the proposer only overclaims. That 250000 exceeds the P90
+    of 90000 is computed by the deterministic validator, which is the part worth showing."""
+    result = await simulate_learning_loop(str(_BUNDLE_DIR), str(tmp_path))
+
+    assert isinstance(result.run_a.outcome, ValidatedProposal), (
+        "the corrected proposal must still validate — Step 5 ends in a proposal, not a dead end"
+    )
+    assert len(result.run_a.refinements) == 1, (
+        "the simulation validated on the first attempt — the scripted reject-then-correct sequence "
+        "is gone, so the demo cannot show Step 5"
+    )
+    rejected = result.run_a.refinements[0]
+    assert rejected.proposal.claimed_saving_nok == 250_000
+    assert (
+        result.run_a.outcome.proposal.claimed_saving_nok < rejected.proposal.claimed_saving_nok
+    ), (
+        "the corrected proposal must claim LESS than the falsified one — otherwise the refinement "
+        "did not respond to the falsification"
+    )

tests/test_step5_refine_loadbearing.py

@@ -111,7 +111,7 @@ async def test_refinement_feeds_prior_falsification_into_next_prompt() -> None:
     # context="" so the flip token cannot pre-exist in attempt 1's prompt.
     result = await generate_via_llm(client, project, "", _meter(), max_attempts=3)
 
-    assert isinstance(result, ValidatedProposal), (
+    assert isinstance(result.outcome, ValidatedProposal), (
         "the proposer corrected on attempt 2 but the loop did not validate -- the falsification "
         "never reached the next prompt"
     )
@@ -134,5 +134,5 @@ async def test_refinement_loop_stays_bounded_when_never_fixed() -> None:
 
     result = await generate_via_llm(client, project, "", _meter(), max_attempts=3)
 
-    assert isinstance(result, Rejection)
+    assert isinstance(result.outcome, Rejection)
     assert client.call_count == 3