open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 4 Activity

feat(p20): the requirement that is RIGHT, and a clause number that is not a price

Browse source 
Three seams, one commit: A, B and C touch the same four modules (run.py carries
the debate task, the grounding composition and the announcement; okf.py carries
one reference-number vocabulary read by both A and B), so splitting them into
three commits would have meant hunk-level staging of entangled files. Stated
rather than silently restructured.

A — the declaration answers with the DOCUMENT's own words. Measured: 13
declarations over round 3 and P17b, not one naming a fasit concept, while the
tool answered {"declared": true, ...} by echoing the caller's own arguments. It
now returns the document's title and req_number, read off Bundle.context_files
(so the type: verdict layer can never be named back), plus the sentence saying
what the declaration binds. A path the base carries as no concept answers with
empty strings rather than refusing. The commission's success_criteria now reach
the DEBATE task through mandate.criteria_block, the one renderer, empty when
there are none — which is what keeps every un-commissioned prompt, and the
golden, byte-identical.

B — a clause number is not a price. THE ORDER'S OWN RULE WAS FELLED BY
MEASUREMENT: it asks to refuse a code that IS declared req_number/prosessnr,
and neither of its two known positives is. n500 declares seksjon 10.4.1..10.4.4
but never the bare 10.4; r761 declares 2727 prosessnr and 2753 seksjon, none of
them 1.10.4, which occurs once, as prose ("iht. vegnormal N200 kap. 1.10.4").
The COMPLEMENT fires on both and closes the hole _ground_against_input already
admits in writing -- "it fails OPEN on a coincidental match". Unanchored run +
requirement-shaped code + the base declares a vocabulary + the code is not in
it -> refused, naming the denominator. All five of kontrakt-sorasen's real
process codes ARE declared and pass, which is what keeps the one context set
built on real codes measurable. Replayed over all 24 codes of round 3 + P17b:
exactly the two known positives flip validated -> rejected, 22 unchanged.

C — a parse failure no longer burns the round ledger blind. _fetch_parsed takes
a BUILDER instead of a finished message list, so the retry carries the parse
reason; measured, kontrakt-sorasen-04 spent 11 of 12 rounds re-asking the same
question. And announced_subject names the routed bases instead of saying "the
portfolio" for a two-base commission.

Suite 1807/5 (from 1781, +26, 0 removed), golden demo-transcript.stdout
BYTE-UNCHANGED (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f),
ruff and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-15 06:02:46 +02:00
commit c8f0c8f7c4
12 changed files with 1098 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

63
src/portfolio_optimiser/generate.py
View file

@ -286,6 +286,7 @@ def _build_messages(
*,
approach: Approach | None = None,
prior_feedback: str | None = None,
parse_error: str | None = None,
) -> list[Message]:
"""Build the hypothesis prompt. When ``prior_rejection`` is set (Step 5, målbilde §5/§7),
append a revision block carrying ONLY the falsification *reason* verbatim — never the prior
@ -308,8 +309,17 @@ def _build_messages(
"your numbers were refuted"; conflating the two would tell the model the machine objected
when a person did. ``None`` -> the byte-identical base prompt, like the other two.
All three are composable, and the ORDER is fixed: base -> approach head -> rejection ->
feedback. A prompt can legitimately carry a rejection AND a feedback at once — that is the
When ``parse_error`` is set (P20/C1) a FOURTH block carries the reason the PREVIOUS reply
could not be parsed — the same "only the reason, never the JSON" rule the other two follow.
It is a different instruction from both: a rejection means the numbers were refuted and a
feedback means a person objected, while this one means nothing was ever read. MEASURED (P19
F4): ``_fetch_parsed`` retried with the byte-identical prompt, and one round-3 run
(``kontrakt-sorasen-04``) spent ELEVEN of its twelve rounds on replies that all failed the
same way — ``claimed_saving_nok: 0`` — because nothing ever told the model what was wrong.
``None`` -> the byte-identical base prompt, like the other three.
All four are composable, and the ORDER is fixed: base -> approach head -> rejection ->
feedback -> parse error. A prompt can legitimately carry a rejection AND a feedback at once — that is the
attempt after a revise whose bought attempt the validator then rejected: the human's
instruction STANDS until the human next answers, while the machine's reason is per-attempt
(only the most recent, as today).
@ -362,6 +372,14 @@ def _build_messages(
f"Expert feedback: {prior_feedback}\n"
"Produce a REVISED SavingsProposal that follows this feedback."
)
if parse_error is not None:
prompt += (
"\n\nYour previous reply could not be PARSED as a SavingsProposal, so it was "
"discarded before any validator saw it.\n"
f"Reason: {parse_error}\n"
"Reply with a SavingsProposal whose claimed_saving_nok is greater than 0 and whose "
"affected_items each carry code, quantity and unit_cost."
)
return [Message(role="user", contents=[prompt])]
@ -472,7 +490,13 @@ def _grounding_text(
*delivered.documents,
*(item.code for item in project.cost_items),
*(() if baseline is None else baseline.items),
)
),
# P20/B: the base's own vocabulary of clause numbers travels WITH the text it was read
# off. Carried through rather than recomposed: ``run_project`` walks the base once and
# composes both halves there, and a second derivation here would be free to disagree with
# the documents it is supposed to describe (kø-(p)). The two later sources are cost CODES,
# which declare nothing, so they contribute none.
declared_references=delivered.declared_references,
)
@ -615,10 +639,19 @@ async def generate_via_llm(
from discarding what it already knew. Never a malformed proposal; raises ``BudgetExceeded``
when the meter cap is crossed."""
async def _fetch_parsed(messages: list[Message]) -> SavingsProposal:
async def _fetch_parsed(build: Callable[[str | None], list[Message]]) -> SavingsProposal:
# Parse-robust: a malformed/text-leaked reply is retried; the meter caps total work.
#
# P20/C1: the retry is no longer BLIND. It takes a BUILDER rather than a finished message
# list, because the whole defect was that the same bytes were re-sent: measured, one
# round-3 run burned 11 of its 12 rounds on replies that all failed identically. The
# builder is the caller's own ``_build_messages`` binding, so this loop cannot compose a
# prompt the outer loop would not have composed (kø-(p)); the reason is per-RETRY, like
# ``prior_rejection`` is per-attempt, and starts empty so attempt 1 is byte-identical.
parse_error: str | None = None
while True:
meter.tick_round() # between-attempt bound (BudgetExceeded over cap)
messages = build(parse_error)
# Fase 1b, funn 1b: hand the model a GRAMMAR, not a prose request. The prompt's
# "Respond with ONLY a JSON object" line stays — a provider that ignores
# ``response_format`` (or a local model that does not implement it) must still be told
@ -633,10 +666,10 @@ async def generate_via_llm(
# Capture BEFORE the retry: this reply was paid for, and once ``continue`` runs the
# only record of what the model actually said is gone (Fase 1b, funn 1). Verbatim —
# the operator is diagnosing a format failure, so any shortening removes evidence.
reason = f"{type(exc).__name__}: {exc}"
if parse_failures is not None:
parse_failures.append(
ParseFailure(text=reply.text, error=f"{type(exc).__name__}: {exc}")
)
parse_failures.append(ParseFailure(text=reply.text, error=reason))
parse_error = reason
continue
last: Rejection | None = None
@ -672,14 +705,16 @@ async def generate_via_llm(
# accumulated history (bounded prompt growth).
if last is not None:
fed_back.append(last)
messages = _build_messages(
project,
context,
prior_rejection=last,
approach=approach,
prior_feedback=feedback,
candidate = await _fetch_parsed(
lambda parse_error: _build_messages(
project,
context,
prior_rejection=last,
approach=approach,
prior_feedback=feedback,
parse_error=parse_error,
)
)
candidate = await _fetch_parsed(messages)
if pending_revise is not None and reviews is not None:
reviews[pending_revise] = replace(reviews[pending_revise], honoured=True)
pending_revise = None