feat(fase4): route the validator falsification into the next hypothesis (informed refinement)
Closes gap #5 (maalbilde §5/§7): generate_via_llm's outer max_attempts loop built the prompt ONCE and re-sent it identically — a BLIND retry. The validator's per-attempt Rejection.reason was captured in `last` but never reached the next prompt, so the proposer re-answered the same question with no knowledge of WHY the prior candidate failed. Step 5 routes that reason into the next attempt. - generate.py: _build_messages() gains prior_rejection; when set it appends a revision block carrying ONLY the falsification reason verbatim (never the rejected proposal JSON). None -> the byte-identical base prompt, so attempt 1 is unchanged. generate_via_llm() rebuilds messages inside the outer loop with prior_rejection=`last` (None on attempt 1); _fetch_parsed() takes messages as an explicit parameter. `last` is overwritten each round -> only the most-recent falsification ("forrige"), never an accumulated history. Bound unchanged: max_attempts + meter.tick_round (no new loop; §6 — "improve until good enough" without a ceiling stays impossible). - Scope honesty: the only per-attempt falsifier here is the validator. The checker is a run-level, one-shot signal (run.py, before generation); seeding generation with the checker critique is separately scoped and NOT done here. The boundary is written into the generate_via_llm docstring + README + CLAUDE. Load-bearing (maalbilde §7): tests/test_step5_refine_loadbearing.py is a PAIR — the positive test keys the proposer's flip on the validator REASON PAYLOAD (the rejected claim value, derived from validate_proposal(bad).reason so test and SUT share one source of truth), and asserts the reason reached attempt 2's prompt VERBATIM (the green-but-dead guard). It goes RED on detach (build messages once): the flip token never arrives, so the outcome never flips AND the verbatim assertion fails — proven double-red. The bounded control proves a never-fixed proposer exhausts exactly max_attempts and returns a Rejection. Adversarial Plan agent hardened the design pre-implementation (flip on payload not wrapper/call-count; derive flip-key from the validator reason; drive through generate_via_llm directly; docstring honesty). Suite 136->138 passed, 4 skipped; mypy + ruff check clean. New test ruff-formatted; pre-existing ruff-format drift (budget/verdicts/test_contracts) left untouched for a surgical diff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
This commit is contained in:
parent
4ec778c855
commit
f7c81b45ec
4 changed files with 188 additions and 10 deletions
|
|
@ -19,7 +19,8 @@ The mandatory deterministic backbone (validator + budget meter + provenance) is
|
|||
- **Verdict layer gated out of context.** The `type: verdict` layer is excluded from that context; the candidate's prior expert verdicts reach the hypothesis prompt *only* through the gated ExpeL fold (folded in *before* generation), so a prior verdict provably — and exclusively — influences the next hypothesis.
|
||||
- Two load-bearing tests fail when the seam is detached: the realization signal must reach the prompt via the fold (`tests/test_step1_expel_loadbearing.py`), and must never leak via context (`tests/test_okf.py::test_bundle_context_excludes_verdict_layer`).
|
||||
- **Steps 3/4 — checker gates the reasoning (wired).** Two falsifiers now act on the same candidate: the deterministic validator gates the **numbers** (blocking, as before), and the maker-checker's checker gates the **reasoning** (target picture §2/§6). The checker ends its turn with a `VERDICT: APPROVE` / `VERDICT: REJECT — <reason>` line; an explicit reject blocks an otherwise-validated proposal (`run_project` surfaces both debate participants via `output_from=agents` and overrides the outcome to a checker-sourced `Rejection`). The gate is opt-in-reject (fail-open on a missing marker), and `provenance.validator_decision` stays honest — it reflects the validator only, never the checker. Load-bearing: `tests/test_checker_gate_loadbearing.py` goes red on *either* detach (revert `output_from`, or drop the override).
|
||||
- **Steps 5–8** (informed refinement, async file feedback, gated wiki promotion) — not yet wired.
|
||||
- **Step 5 — informed refinement (wired).** The proposer's bounded retry is no longer blind: the validator's *previous* `Rejection.reason` is fed into the next attempt's prompt (`generate.py` `generate_via_llm` → `_build_messages(prior_rejection=...)`), so the model corrects against the falsification instead of re-answering identically (target picture §5/§7). It carries only the most-recent reason (never an accumulated history, never the rejected proposal JSON) and runs under the *existing* `max_attempts` + token-meter cap — no new loop, so "improve until good enough" without a ceiling stays impossible. The only per-attempt falsifier here is the validator; seeding generation with the checker's critique is a run-level, separately-scoped concern and is deliberately not done here. Load-bearing: `tests/test_step5_refine_loadbearing.py` goes red when the reason is detached from the prompt (the outcome never flips *and* the verbatim-reason assertion fails), with a bounded control proving the loop still stops at `max_attempts`.
|
||||
- **Steps 7–8** (async file feedback, gated wiki promotion) — not yet wired.
|
||||
|
||||
## Docs
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue