feat(1b): fang den råe modell-svarteksten ved parse-feil
Fase 1b funn 1 lukket. generate._fetch_parsed kastet hvert uparsebart modellsvar i except: continue, så prosjektets første levende kjøring brant tolv runder på formatfeil og etterlot null tegn av det modellen faktisk sa. Enhver videre betalt kjøring ville vært gjetning. HVOR teksten overflates er avgjort av en måling, ikke av symmetri med Steg 5: meter.tick_round() raiser BudgetExceeded INNE i _fetch_parsed, og uten mandat fanger ingen den, så på nøyaktig den stien fangsten finnes for returnerer generate_via_llm ingenting. Et felt på GenerationResult ville vært blindt for den, og et outbox-artefakt skrevet etter kjøringen likeså. Sinken speiler meter: en kaller-eid akkumulator hvis innhold kalleren holder uansett hvordan løkka endte. Artefaktet skrives fra en finally, ikke except BudgetExceeded, og kun når noe faktisk feilet. Iron Law: testfila rød ved collection FØR modulen fantes. Seks mutasjoner mot HELE suiten, alle røde, hver med sin egen signatur; grønn kontroll 859 passed / 4 skipped (fra 854). Den skarpeste er trunkering som BEHOLDER sentinelen: da faller kun verbatim-asserten, som er det som beviser at den ene testen bærer den egenskapen. Samme økt: mutasjonsmålingen økt 34 utsatte for de to Fase 5-gatene er kjørt. Fire preflight-mutasjoner mot hele suiten, alle røde på riktig test og ingen annen (detach fallbacken 2 røde, snu presedensen, presence i stedet for truthiness, avslaget navngir kun vårt navn). To handover-mutasjoner kjørt MÅLRETTET mot egen testfil under tidspress, ikke mot hele suiten — uttalt, ikke skjult (drop uv.lock, bygg fra arbeidstreet i stedet for tracked files). De to DEPLOY.md-mutasjonene gjenstår: git archive leser HEAD, ikke arbeidstreet, så de krever en midlertidig commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WLQd8ojQ9xwxhB8vsETYBs
This commit is contained in:
parent
5bd8e1caa1
commit
e3718908d0
5 changed files with 413 additions and 7 deletions
|
|
@ -44,6 +44,23 @@ class GenerationError(RuntimeError):
|
|||
"""No parseable proposal could be produced within the attempt budget."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ParseFailure:
|
||||
"""One model reply that did NOT parse into the typed IR, kept VERBATIM (Fase 1b, funn 1).
|
||||
|
||||
``text`` is the reply exactly as the model produced it — never truncated, stripped or
|
||||
summarised. It is the thing the run PAID for and the only evidence of *why* the reply did not
|
||||
parse; a paraphrase would make the next paid run a guess again, which is the defect this type
|
||||
exists to close. ``error`` names the parse error itself (``json.JSONDecodeError`` vs a pydantic
|
||||
``ValidationError`` are very different diagnoses: leaked prose vs a wrong-shaped object).
|
||||
|
||||
Collected into a CALLER-OWNED sink rather than returned — see ``generate_via_llm``.
|
||||
"""
|
||||
|
||||
text: str
|
||||
error: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GenerationResult:
|
||||
"""What one ``generate_via_llm`` call produced: the outcome, and the falsification history that
|
||||
|
|
@ -163,6 +180,7 @@ async def generate_via_llm(
|
|||
max_attempts: int = 3,
|
||||
baseline: CostBaseline | None = None,
|
||||
approach: Approach | None = None,
|
||||
parse_failures: list[ParseFailure] | None = None,
|
||||
) -> GenerationResult:
|
||||
"""Async LLM path: non-streaming chat -> parse -> validate, with TWO bounded retry kinds,
|
||||
the meter checked in this loop:
|
||||
|
|
@ -189,6 +207,19 @@ async def generate_via_llm(
|
|||
falsified per ATTEMPT like any other rejection — and its reason feeds the next attempt's prompt
|
||||
through the SAME informed-refinement path (Step 5), which is why no new loop appears here.
|
||||
|
||||
``parse_failures`` (Fase 1b, funn 1) is a CALLER-OWNED sink: every reply that fails to parse is
|
||||
appended to it VERBATIM, at the moment it fails. It is an out-parameter and not part of the
|
||||
return value ON PURPOSE, and the reason is measured rather than stylistic. ``meter.tick_round``
|
||||
raises ``BudgetExceeded`` inside the inner fetch loop, so on the path this capture exists for —
|
||||
a model whose replies never parse, which burns the round ledger — this function raises and
|
||||
returns NOTHING. That is exactly the live Fase-1b failure. A field on ``GenerationResult`` (the
|
||||
Step-5 ``refinements`` shape) would be blind to it, as would any artefact written by the caller
|
||||
*after* a successful return. The sink mirrors ``meter`` instead: a caller-owned accumulator this
|
||||
loop mutates, whose contents the caller still holds however the loop ended. Step 5's "a returned
|
||||
value cannot be silently lost by a caller that forgets to pass a collector" governs a value that
|
||||
REACHES the caller; here it does not, so the rule is cited and departed from deliberately. That
|
||||
a caller can forget is answered by a test on the wiring, not by a shape that cannot work.
|
||||
|
||||
Returns a ``GenerationResult``: the ``ValidatedProposal | Rejection`` outcome plus every
|
||||
rejection that was fed back into a later attempt's prompt. Surfacing that history changes
|
||||
nothing about the loop's BOUND — ``max_attempts`` and ``meter.tick_round`` are exactly as
|
||||
|
|
@ -204,7 +235,14 @@ async def generate_via_llm(
|
|||
_charge_usage(meter, reply)
|
||||
try:
|
||||
return _parse_ir(reply.text, project)
|
||||
except (ValidationError, ValueError, TypeError):
|
||||
except (ValidationError, ValueError, TypeError) as exc:
|
||||
# Capture BEFORE the retry: this reply was paid for, and once ``continue`` runs the
|
||||
# only record of what the model actually said is gone (Fase 1b, funn 1). Verbatim —
|
||||
# the operator is diagnosing a format failure, so any shortening removes evidence.
|
||||
if parse_failures is not None:
|
||||
parse_failures.append(
|
||||
ParseFailure(text=reply.text, error=f"{type(exc).__name__}: {exc}")
|
||||
)
|
||||
continue
|
||||
|
||||
last: Rejection | None = None
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue