feat(explore): den offline generalproeven kan AAPNE en base (MAJOR-1 b, ORDRE 20260902T151931Z)

`--scripted-replies` tok EN konstant streng per rolle, saa ingen skriptet rolle
kunne emittere et `function_call`: maalt 0 verktoeykall / 0 approaches / 1 runde
paa 4/4 baser, mens hvert andre felt i artefaktet saa ut som en kjoering som
hadde virket. En operatoer som foelger dette repoets egen maalestige — «bevis saa
mye som mulig gratis foer det dyre trinnet» — kunne ikke bevise at navigatoeren
aapner noe.

En UTFORSKNINGSROLLE kan naa faa en LISTE av trinn, der et trinn er en tekst
eller ETT verktoeykall (`{"call": "<tool>", "args": {...}}`). Formen utvider den
ENE kanoniske `_inner_get_response`-kroppen (S2.5-konsolideringen) — ikke en
andre klient: rapportens vedlegg-A-`RecordingClient` var maaleinstrument, og en
kopi av kroppen ville drevet fra de tre conftest-dublettene som rir paa den.
Manuset ligger VED SIDEN AV selektoren, ikke inni: en selektor returnerer `str`
ved kontrakt, og et verktoeykall er ikke tekst. Naar manuset er tomt svarer
selektoren som foer, saa et utloept manus degraderer til konstantformen, ikke til
stillhet.

Listeformen NEKTES for debattens roller ved navn (proposeren svarer `generate`s
eget kall, ikke en agent-loekke som kan kalle et verktoey mellom turer), og hvert
malformet trinn nektes ved navn — validering, ALDRI reparasjon
(`write_concept_file`-regelen). Et trinn som slipper gjennom naar
`step["call"]` og krasjer midt i kjoeringen, som er nettopp MAJOR-2-klassen
filas foerste halvdel lukket.

MAALT: ni mutasjoner, alle roede mot HELE suiten, hver med sitt eget vitne +
groenn kontroll 2 failed (F15-diffen, KJENT) / 1087 passed / 5 skipped:
M1 detach recorderen fra explore() OG resume (1 roed) · M2 dropp `tool_calls`
fra `trace_payload` (3) · M3 sorter+dedupliser sinken (2) · M4 registrer navnet
uten basen (3) · M5 nekt listeformen igjen (2) · M6 aksepter lista men emitter
tekst (1) · M7 toler et malformet trinn (1) · M8 la en debattrolle ta
listeformen (1) · M9 dropp ukjent-noekkel-whitelisten (1). Golden
`demo-transcript.stdout` BYTE-UENDRET (ea8c534773acdbe41ae68f2c55724d69aaf8be4f).

M7 FALSIFISERTE TESTEN FOERST (repoets vakuoes-gate-klasse, TOLVTE gang):
nekt-testen brukte `{"invoke": "list_bundles"}` og sto GROENN med forms-sjekken
detached — UKJENT-NOEKKEL-grenen fanget den i stedet, saa den nekten som var
under test hadde intet vitne. De to grenene oeves naa av to armer med hver sin
mutasjon (M7 og M9).

Aerlighetsgrense, uttalt: at en LEVENDE modell kaller verktoeyene er fortsatt
ikke bevist (samme klasse som structured-output-grensen) — dette gjoer den
GRATIS halvdelen av stigen ekte, ikke den betalte.

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-02 18:45:53 +02:00
commit 2654219e8c
3 changed files with 267 additions and 14 deletions

View file

@ -1618,13 +1618,52 @@ _SCRIPTED_BANNER = (
)
def _validate_script(role: str, steps: Any, path: str) -> None:
"""Refuse a malformed step script BY NAME (MAJOR-1 b). Validation, NEVER repair.
A step is either a plain reply (``str``) or one tool call
(``{"call": "<tool>", "args": {...}}``). Skipping a step nobody could parse would run a
rehearsal that proves less than the operator wrote while looking exactly like one that proved
it the failure mode this whole seam exists to make visible. An EMPTY list is refused for the
same reason: it is the constant form written obscurely, and it would fall through to the
default reply.
"""
if not isinstance(steps, list) or not steps:
raise ValueError(f"--scripted-replies[{role!r}] must be a non-empty list of steps ({path})")
for index, step in enumerate(steps):
if isinstance(step, str):
continue
if not isinstance(step, dict) or not isinstance(step.get("call"), str):
raise ValueError(
f"--scripted-replies[{role!r}] step {index} is neither a text reply nor a tool "
f'call {{"call": "<tool>", "args": {{...}}}} ({path})'
)
unknown = sorted(set(step) - {"call", "args"})
if unknown:
raise ValueError(
f"--scripted-replies[{role!r}] step {index} names unknown key(s) "
f"{', '.join(unknown)}; a step carries only 'call' and 'args' ({path})"
)
if step.get("args") is not None and not isinstance(step["args"], dict):
raise ValueError(
f"--scripted-replies[{role!r}] step {index}: 'args' must be an object ({path})"
)
def _load_scripted_replies(
path: str, required_roles: Sequence[str] = _SCRIPTED_ROLES
) -> dict[str, str]:
) -> dict[str, Any]:
"""Load the caller's scripted answers, fail-fast. Every role ``required_roles`` names must be
present AND a string: a missing role would otherwise surface as a ``KeyError`` deep inside
present AND readable: a missing role would otherwise surface as a ``KeyError`` deep inside
``scripted_factory``'s lookup, mid-run, long after the run appeared to start cleanly (MAJOR-2:
measured for the three ``explore()`` adds on top of the debate's own two)."""
measured for the three ``explore()`` adds on top of the debate's own two).
**An EXPLORATION role may also be given a step LIST** (MAJOR-1 b), because a single constant
string can never emit a ``function_call`` measured: 0 tool calls / 0 approaches / 1 round on
4/4 bases, an offline rehearsal that was vacuous by construction. The list form is refused for
the debate's roles BY NAME rather than accepted and ignored: the proposer answers
``generate``'s own call, not an agent loop that would invoke a tool between turns, so a script
of calls there describes a rehearsal that cannot happen."""
try:
raw = json.loads(Path(path).read_text(encoding="utf-8"))
except FileNotFoundError as exc:
@ -1633,12 +1672,22 @@ def _load_scripted_replies(
raise ValueError(f"--scripted-replies is not valid JSON ({path}): {exc}") from exc
if not isinstance(raw, dict):
raise ValueError(f"--scripted-replies must be a JSON object of role -> reply ({path})")
missing = [r for r in required_roles if not isinstance(raw.get(r), str)]
missing = [r for r in required_roles if not isinstance(raw.get(r), (str, list))]
if missing:
raise ValueError(
f"--scripted-replies needs a string reply for each of {', '.join(required_roles)}; "
f"missing or non-string: {', '.join(missing)} ({path})"
f"--scripted-replies needs a reply for each of {', '.join(required_roles)}; "
f"missing or of an unusable type: {', '.join(missing)} ({path})"
)
for role in required_roles:
if isinstance(raw[role], str):
continue
if role not in _EXPLORATION_SCRIPTED_ROLES:
raise ValueError(
f"--scripted-replies[{role!r}] is a step list, but that form is the exploration's: "
f"only {', '.join(_EXPLORATION_SCRIPTED_ROLES)} run inside an agent loop that can "
f"invoke a tool between turns ({path})"
)
_validate_script(role, raw[role], path)
return {role: raw[role] for role in required_roles}