portfolio-optimiser/tests
Kjell Tore Guttormsen f7c81b45ec feat(fase4): route the validator falsification into the next hypothesis (informed refinement)
Closes gap #5 (maalbilde §5/§7): generate_via_llm's outer max_attempts loop
built the prompt ONCE and re-sent it identically — a BLIND retry. The validator's
per-attempt Rejection.reason was captured in `last` but never reached the next
prompt, so the proposer re-answered the same question with no knowledge of WHY
the prior candidate failed. Step 5 routes that reason into the next attempt.

- generate.py: _build_messages() gains prior_rejection; when set it appends a
  revision block carrying ONLY the falsification reason verbatim (never the
  rejected proposal JSON). None -> the byte-identical base prompt, so attempt 1
  is unchanged. generate_via_llm() rebuilds messages inside the outer loop with
  prior_rejection=`last` (None on attempt 1); _fetch_parsed() takes messages as
  an explicit parameter. `last` is overwritten each round -> only the most-recent
  falsification ("forrige"), never an accumulated history. Bound unchanged:
  max_attempts + meter.tick_round (no new loop; §6 — "improve until good enough"
  without a ceiling stays impossible).
- Scope honesty: the only per-attempt falsifier here is the validator. The
  checker is a run-level, one-shot signal (run.py, before generation); seeding
  generation with the checker critique is separately scoped and NOT done here.
  The boundary is written into the generate_via_llm docstring + README + CLAUDE.

Load-bearing (maalbilde §7): tests/test_step5_refine_loadbearing.py is a PAIR —
the positive test keys the proposer's flip on the validator REASON PAYLOAD (the
rejected claim value, derived from validate_proposal(bad).reason so test and SUT
share one source of truth), and asserts the reason reached attempt 2's prompt
VERBATIM (the green-but-dead guard). It goes RED on detach (build messages once):
the flip token never arrives, so the outcome never flips AND the verbatim
assertion fails — proven double-red. The bounded control proves a never-fixed
proposer exhausts exactly max_attempts and returns a Rejection.

Adversarial Plan agent hardened the design pre-implementation (flip on payload
not wrapper/call-count; derive flip-key from the validator reason; drive through
generate_via_llm directly; docstring honesty). Suite 136->138 passed, 4 skipped;
mypy + ruff check clean. New test ruff-formatted; pre-existing ruff-format drift
(budget/verdicts/test_contracts) left untouched for a surgical diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
2026-06-30 09:08:02 +02:00
..
spikes fix(fase3): relax hardcoded project-count assert in spike test (SC1 4th-project follow-up) 2026-06-26 12:14:34 +02:00
conftest.py feat(fase2): wire Step-1 ExpeL retrieval into the hypothesis prompt 2026-06-29 10:56:48 +02:00
test_backends.py feat(fase2): wire local + Foundry profiles + model-map + env template 2026-06-24 13:38:43 +02:00
test_budget.py test(fase2): assert BudgetMiddleware short-circuits a real chat call + the debate 2026-06-26 00:42:51 +02:00
test_bygg_energi_mikro.py docs(shared): bygg-energi mikro-eksempel — OKF-bundle + golden + load-bearing test 2026-06-29 09:42:13 +02:00
test_checker_gate_loadbearing.py feat(fase3): make the maker-checker checker actually gate the reasoning 2026-06-30 07:24:30 +02:00
test_contracts.py feat(fase2): fail-fast contract loaders 2026-06-24 13:50:28 +02:00
test_datasource.py feat(fase2): expose retriever as citation-bearing data source 2026-06-24 13:34:53 +02:00
test_foundry_profile_live.py test(fase2): gated live local + Foundry profile checks 2026-06-24 14:02:44 +02:00
test_generate.py feat(fase2): LLM->IR generation with validator-as-retry 2026-06-24 13:48:54 +02:00
test_imports_core.py build(fase2): promote orchestrations + pulp to core deps 2026-06-24 13:22:04 +02:00
test_local_profile_live.py test(fase2): gated live local + Foundry profile checks 2026-06-24 14:02:44 +02:00
test_okf.py feat(fase2b): OKF-navigated bundle context replaces chunk-stuffing 2026-06-30 06:42:19 +02:00
test_portfolio.py test(fase3): portfolio rejection arm + provenance-seam assertions (F2 + F1 follow-up) 2026-06-26 15:05:41 +02:00
test_portfolio_live.py test(fase3): SC7 both profiles offline + gated Azure portfolio arm 2026-06-26 12:10:04 +02:00
test_provenance.py feat(fase2): first-class Pydantic provenance stamp 2026-06-24 13:26:55 +02:00
test_reference_domain.py feat(fase3): config-driven docs_dir + verdict_input on Project loader 2026-06-26 11:55:21 +02:00
test_retrieval.py feat(fase2): local-folder retriever core with path-security 2026-06-24 13:26:00 +02:00
test_run_smoke.py feat(fase2): vertical-slice orchestrator + two-layer HITL wiring 2026-06-24 13:54:54 +02:00
test_smoke.py feat: initial scaffold (Python framework on Microsoft Agent Framework) 2026-06-23 22:01:22 +02:00
test_step1_expel_loadbearing.py feat(fase2b): OKF-navigated bundle context replaces chunk-stuffing 2026-06-30 06:42:19 +02:00
test_step5_refine_loadbearing.py feat(fase4): route the validator falsification into the next hypothesis (informed refinement) 2026-06-30 09:08:02 +02:00
test_validator.py feat(fase2): promote blocking validator + IR to src 2026-06-24 13:24:03 +02:00
test_verdicts.py feat(fase2): wire Step-1 ExpeL retrieval into the hypothesis prompt 2026-06-29 10:56:48 +02:00
test_vertical_slice_e2e.py fix(fase3): stamp real proposer model into provenance, kill fake-model leak (F1) 2026-06-26 15:03:45 +02:00
test_workflow.py feat(fase2): thread tools + budget middleware through fresh_workflow onto agents 2026-06-26 00:36:06 +02:00