fix(cli): a provider failure leaves the CLI as one line, and a guessed base id is correctable
Funn 99, measured offline against the artefacts the paid Q5=B run left behind — no paid
run here.
ROOT, verbatim from the records: the three failing quick_validate calls all sent
bundle_id="renholdstekniske_funksjonskrav" — a CONCEPT name guessed out of the seeded cut,
while the base's id is k2-trinn1-20260903. Both arguments parsed against the signature, so
it was _resolve_bundle's raise MAF counted, proven by quick_validations being EMPTY while
all three stand in tool_calls. Denominator: 12 tool calls, and those three came BEFORE
list_bundles.
The order's causal chain is FELLED: the quick_validate triple is records 4-6 and the run
continued for 13 more model calls; the triple immediately before the 400 is the navigator's
three read_file refusals on del-ii-bilag-7-prisskjema*. The limit fired TWICE.
(A) ChatClientException is caught on BOTH seams — the exploration dispatch and the full-run
dispatch — because the debate's own model calls go through the same provider. The line is
"run stopped:", not "run refused:" (a stated divergence from the order): the argv was fine
and tokens were already spent, which is the MAJOR-2 arm's own reason, verbatim. Caught
INSIDE the try/finally so the exploration artefact still lands.
(B) quick_validate answers an unknown base id with {"decision": "refused", ...} naming the
configured ids, and records it in the sink. MAF turns a tool raise into the opaque
"Error: Function failed." (_tools.py:1426), so the one thing the refusal knew and the model
did not never reached it — the replies show it guessing at the JSON format instead.
read_file/read_dir/read_bundle still raise: measured, reported, out of scope.
Seven mutations all red against the whole suite, green control 1529/5, golden ea8c534
unchanged. One existing gate REWRITTEN, not deleted; its second half is what keeps (B)
scoped. The test double raises from the reply_selector seam rather than a new
_inner_get_response body, so the S2.5 consolidation guard stays untouched.
Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
parent
4f23fa2a70
commit
078a099898
5 changed files with 543 additions and 5 deletions
|
|
@ -752,10 +752,23 @@ def test_an_unknown_knowledge_base_is_refused_by_name() -> None:
|
|||
Model-chosen arguments are untrusted input. Answering an unknown id with an empty result would
|
||||
let the manager conclude the base is empty rather than absent — the fourth face of the
|
||||
verification law, arrived at through a tool rather than a query.
|
||||
|
||||
REWRITTEN, not weakened (funn 99): ``quick_validate`` now answers with a ``refused`` VERDICT
|
||||
rather than a raise, because MAF converts a tool raise into the opaque ``"Error: Function
|
||||
failed."`` and counts three of them as grounds to stop function calling for the whole request —
|
||||
so the one thing this refusal knows and the model does not (which ids exist) never reached it.
|
||||
The property T20 was written for is untouched and asserted here in both halves: the answer is
|
||||
still a refusal, and it still names what IS configured. The ``navigator_tools`` half is what
|
||||
keeps the change SCOPED — those three tools still raise.
|
||||
"""
|
||||
validate = explore.quick_validate_tool(("/tmp/base-a",))
|
||||
verdict = validate.func(bundle_id="base-b", proposal_json="{}")
|
||||
assert verdict["decision"] == "refused"
|
||||
assert "base-b" in verdict["reason"] and "base-a" in verdict["reason"]
|
||||
|
||||
read_file = next(t for t in explore.navigator_tools(("/tmp/base-a",)) if t.name == "read_file")
|
||||
with pytest.raises(explore.ExplorationError) as excinfo:
|
||||
validate.func(bundle_id="base-b", proposal_json="{}")
|
||||
read_file.func(bundle_id="base-b", path="x.md")
|
||||
assert "base-a" in str(excinfo.value)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue