feat(1b): fang den råe modell-svarteksten ved parse-feil

Fase 1b funn 1 lukket. generate._fetch_parsed kastet hvert uparsebart modellsvar i
except: continue, så prosjektets første levende kjøring brant tolv runder på formatfeil
og etterlot null tegn av det modellen faktisk sa. Enhver videre betalt kjøring ville
vært gjetning.

HVOR teksten overflates er avgjort av en måling, ikke av symmetri med Steg 5:
meter.tick_round() raiser BudgetExceeded INNE i _fetch_parsed, og uten mandat fanger
ingen den, så på nøyaktig den stien fangsten finnes for returnerer generate_via_llm
ingenting. Et felt på GenerationResult ville vært blindt for den, og et outbox-artefakt
skrevet etter kjøringen likeså. Sinken speiler meter: en kaller-eid akkumulator hvis
innhold kalleren holder uansett hvordan løkka endte. Artefaktet skrives fra en finally,
ikke except BudgetExceeded, og kun når noe faktisk feilet.

Iron Law: testfila rød ved collection FØR modulen fantes. Seks mutasjoner mot HELE
suiten, alle røde, hver med sin egen signatur; grønn kontroll 859 passed / 4 skipped
(fra 854). Den skarpeste er trunkering som BEHOLDER sentinelen: da faller kun
verbatim-asserten, som er det som beviser at den ene testen bærer den egenskapen.

Samme økt: mutasjonsmålingen økt 34 utsatte for de to Fase 5-gatene er kjørt. Fire
preflight-mutasjoner mot hele suiten, alle røde på riktig test og ingen annen (detach
fallbacken 2 røde, snu presedensen, presence i stedet for truthiness, avslaget navngir
kun vårt navn). To handover-mutasjoner kjørt MÅLRETTET mot egen testfil under tidspress,
ikke mot hele suiten — uttalt, ikke skjult (drop uv.lock, bygg fra arbeidstreet i stedet
for tracked files). De to DEPLOY.md-mutasjonene gjenstår: git archive leser HEAD, ikke
arbeidstreet, så de krever en midlertidig commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WLQd8ojQ9xwxhB8vsETYBs
This commit is contained in:
Kjell Tore Guttormsen 2026-08-14 11:41:04 +02:00
commit e3718908d0
5 changed files with 413 additions and 7 deletions

View file

@ -44,6 +44,23 @@ class GenerationError(RuntimeError):
"""No parseable proposal could be produced within the attempt budget."""
@dataclass(frozen=True)
class ParseFailure:
"""One model reply that did NOT parse into the typed IR, kept VERBATIM (Fase 1b, funn 1).
``text`` is the reply exactly as the model produced it never truncated, stripped or
summarised. It is the thing the run PAID for and the only evidence of *why* the reply did not
parse; a paraphrase would make the next paid run a guess again, which is the defect this type
exists to close. ``error`` names the parse error itself (``json.JSONDecodeError`` vs a pydantic
``ValidationError`` are very different diagnoses: leaked prose vs a wrong-shaped object).
Collected into a CALLER-OWNED sink rather than returned see ``generate_via_llm``.
"""
text: str
error: str
@dataclass(frozen=True)
class GenerationResult:
"""What one ``generate_via_llm`` call produced: the outcome, and the falsification history that
@ -163,6 +180,7 @@ async def generate_via_llm(
max_attempts: int = 3,
baseline: CostBaseline | None = None,
approach: Approach | None = None,
parse_failures: list[ParseFailure] | None = None,
) -> GenerationResult:
"""Async LLM path: non-streaming chat -> parse -> validate, with TWO bounded retry kinds,
the meter checked in this loop:
@ -189,6 +207,19 @@ async def generate_via_llm(
falsified per ATTEMPT like any other rejection and its reason feeds the next attempt's prompt
through the SAME informed-refinement path (Step 5), which is why no new loop appears here.
``parse_failures`` (Fase 1b, funn 1) is a CALLER-OWNED sink: every reply that fails to parse is
appended to it VERBATIM, at the moment it fails. It is an out-parameter and not part of the
return value ON PURPOSE, and the reason is measured rather than stylistic. ``meter.tick_round``
raises ``BudgetExceeded`` inside the inner fetch loop, so on the path this capture exists for
a model whose replies never parse, which burns the round ledger this function raises and
returns NOTHING. That is exactly the live Fase-1b failure. A field on ``GenerationResult`` (the
Step-5 ``refinements`` shape) would be blind to it, as would any artefact written by the caller
*after* a successful return. The sink mirrors ``meter`` instead: a caller-owned accumulator this
loop mutates, whose contents the caller still holds however the loop ended. Step 5's "a returned
value cannot be silently lost by a caller that forgets to pass a collector" governs a value that
REACHES the caller; here it does not, so the rule is cited and departed from deliberately. That
a caller can forget is answered by a test on the wiring, not by a shape that cannot work.
Returns a ``GenerationResult``: the ``ValidatedProposal | Rejection`` outcome plus every
rejection that was fed back into a later attempt's prompt. Surfacing that history changes
nothing about the loop's BOUND — ``max_attempts`` and ``meter.tick_round`` are exactly as
@ -204,7 +235,14 @@ async def generate_via_llm(
_charge_usage(meter, reply)
try:
return _parse_ir(reply.text, project)
except (ValidationError, ValueError, TypeError):
except (ValidationError, ValueError, TypeError) as exc:
# Capture BEFORE the retry: this reply was paid for, and once ``continue`` runs the
# only record of what the model actually said is gone (Fase 1b, funn 1). Verbatim —
# the operator is diagnosing a format failure, so any shortening removes evidence.
if parse_failures is not None:
parse_failures.append(
ParseFailure(text=reply.text, error=f"{type(exc).__name__}: {exc}")
)
continue
last: Rejection | None = None