open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity
portfolio-optimiser/src/portfolio_optimiser/explore.py
Kjell Tore Guttormsen ac0bfdba27 feat(p21): a declaration that must have LOOKED, and a refusal that names the neighbours 
C1. Round 4 produced 13 declarations over six runs and NOT ONE named a fasit concept.
The distinct documents opened before each were 1,1,1,1,1,1,1,2,5,5,6,13,13: seven
declared the base's FIRST requirement after opening exactly ONE document.

The order offered two rules and asked which discriminates. Replayed against the real
listings: "the declared document must have come back from a read_dir filtered on a word
from the approach's label" refuses 13 of 13 -- including Soraasen's 12.11, the closest
any run came -- because ZERO of the 13 were reached through a filtered listing at all.
A gate that refuses every measured case, right and wrong alike, cannot discriminate.
"fewer than k distinct documents opened" at k=3 refuses 8 of 13 and keeps the five that
navigated. k=3, 4 and 5 refuse the SAME eight -- the distribution has a gap between 2
and 5 -- so the threshold is not on a cliff, and 3 is the lowest of that plateau.
DISTINCT paths, not calls, and capped by the base's own size so a small base stays
declarable.

C2. Over the same traces 18 of 143 path-bearing calls named a path the base does not
hold, ELEVEN of them one run walking R761/4-3, 4.3, 4-2, 4-1, 4-0, 4-5, 4-6 while the
real names are R761/4, R761/41, R761/42. The refusal already named the nearest listable
ancestor; now it also names up to five of that rung's own subdirectories, ranked by
longest common prefix with the segment that failed. ONE copy shared by both refusal
sites, built from context_files through in_dimension, so every name handed back resolves
and the verdict layer can never be advertised in an apology. MEASURED after: 16 of 18.

Load-bearing MEASURED, four mutations all red against the WHOLE suite, green control
1863/5, golden byte-unchanged: C3(i) the declaration gate detached (2 red) . C3(ii) the
neighbour list empty (6) . C3(iii) built from files (1, the verdict arm alone) . C3(iv)
count CALLS instead of distinct documents (1, the repetition arm alone).

Three existing arms REWRITTEN, not weakened: all three read one document and declared,
which is the measured failure class exactly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 11:38:57 +02:00

2509 lines
128 KiB
Python
Raw Blame History

"""U4 + U13-synchronous — the Magentic exploration loop, sitting OVER the normative pipeline.
**One sentence of design, and it is not to be reopened without a measurement.** Magentic is laid
over the eight-step loop, never inside Step 3: ``prompt + knowledge bases -> Mandate ->
run_project(mandate=...)``, with that last call byte-for-byte the one it already was. The manager
is free to choose which base to open and which hypothesis to shape next; what leaves that freedom
is a ``mandate.Mandate`` — approaches worth *testing*. The exploration's own final answer is RAW,
never a proposal, and ``explore()`` writes to neither the outbox nor the wiki. Every number that
survives is gated by ``validate_proposal`` inside ``run_project``, in the same blocking gate as
today, and ``shared/method-spec.md`` §3's maker-checker debate is untouched (it is commons-owned
and normative; replacing it would need an amendment, not a module).
**Three levels of guarantee, stated so nobody reads more into the loop than is there.** The
``quick_validate`` tool the hypothesiser calls in-loop is level 1: the SAME ``validate_proposal``,
against the SAME baseline, but its verdict is advisory — it never becomes provenance. Level 2 is
the pipeline, where each approach is validated and stamped. Level 3 is write access, which only
the pipeline has.
This module imports ``agent_framework.orchestrations`` and therefore may never be imported from
``okf.py`` / ``mandate.py`` / ``hitl.py``, which are held framework-neutral by
``tests/test_okf.py::test_okf_is_maf_free``.
"""
from __future__ import annotations
import json
import sys
from collections.abc import Awaitable, Callable, Mapping, Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Final, Literal, TextIO
from agent_framework import (
Agent,
BaseChatClient,
FileCheckpointStorage,
FunctionInvocationContext,
FunctionMiddleware,
FunctionTool,
tool,
)
from agent_framework.orchestrations import (
MagenticBuilder,
MagenticOrchestratorEventType,
MagenticPlanReviewResponse,
)
from pydantic import BaseModel, Field, ValidationError, model_validator
from portfolio_optimiser import okf
from portfolio_optimiser.backends import Profile
from portfolio_optimiser.budget import Budget, BudgetExceeded, BudgetMiddleware, TokenMeter
from portfolio_optimiser.ir import SavingsProposal
from portfolio_optimiser.mandate import (
OWN_PROPOSAL_ID,
Approach,
BindingRequirement,
Mandate,
)
from portfolio_optimiser.retrieval import safe_resolve
from portfolio_optimiser.tracing import exploration_tracer
from portfolio_optimiser.validator import Rejection, validate_proposal
class ExplorationContract(BaseModel):
"""The stated bounds of ONE exploration. Every field is required; none has a default.
Mirrors ``contracts.TerminationContract`` in spirit and goes further in one respect: there is
nothing to inherit. ``MagenticBuilder`` defaults ``max_round_count`` to ``None`` (unbounded)
and ``max_reset_count`` to ``None`` (unlimited), so a field left out here would not fall back
to something conservative — it would fall back to the one shape ``shared/method-spec.md`` §8
forbids outright, an unbounded loop. A default would also assert an intent nobody stated,
which is the ground ``ProvenanceStamp.cost_baseline_anchored`` is required on.
``max_plan_revisions`` earns its place by measurement, not symmetry (§ F, A3): a plan-review
``revise`` costs two manager calls, emits **no** progress ledger and consumes **no** round,
then asks again. Under the round cap alone an always-revising expert is an unbounded spend
that the round counter never sees. ``0`` is meaningful and allowed: the plan must be approved
as first written or the exploration stops.
"""
#: Hard cap on orchestration rounds (``MagenticBuilder(max_round_count=...)``).
max_rounds: int = Field(gt=0)
#: Hard cap on tokens, enforced by ``BudgetMiddleware`` on EVERY agent, manager included.
max_tokens: int = Field(gt=0)
#: Consecutive no-progress rounds tolerated before the manager resets and replans. ``0`` is
#: allowed and means "reset on the first round that reports no progress": the orchestrator's
#: check is STRICT (``stall_count > max_stall_count``, ``_magentic.py:1118``) and the counter is
#: incremented ahead of it, so zero is strictness rather than self-defeat.
max_stall_count: int = Field(ge=0)
#: Resets tolerated before the exploration is declared stalled. Must be POSITIVE, and that is a
#: MEASUREMENT: the limit check is ``reset_count >= max_reset_count`` (``:1243``) against a
#: counter starting at zero, so a cap of zero is already met before the first round. Measured
#: against the installed stack, ``max_reset_count=0`` produces only the ``facts`` and ``plan``
#: manager calls, **zero** progress-ledger events, and the canonical "maximum reset count"
#: termination — an exploration that explored nothing, dressed as a stall that never happened.
max_reset_count: int = Field(gt=0)
#: Plan revisions the reviewer may ask for before the exploration stops. See the class note.
max_plan_revisions: int = Field(ge=0)
#: Whether a human/persona signs the plan off before the loop runs (the U13 synchronous door).
enable_plan_review: bool
@model_validator(mode="after")
def _a_revision_cap_needs_a_review_to_cap(self) -> "ExplorationContract":
"""A revision cap with the review switched off bounds an event that cannot occur.
Refused rather than dropped: a caller who wrote ``max_plan_revisions=3`` believes they
bounded something, and silently ignoring it is the failure mode this repo's flag contract
exists to prevent (``--embedder-config requires --semantic-retrieval``, same rule). The
coherent way to say "no reviews" is ``enable_plan_review=False, max_plan_revisions=0``.
"""
if not self.enable_plan_review and self.max_plan_revisions:
raise ValueError(
f"max_plan_revisions={self.max_plan_revisions} bounds plan revisions, but "
"enable_plan_review is false so no plan review is ever requested and no revision "
"can occur; set max_plan_revisions=0 or enable the review"
)
return self
def exploration_notice(result: "ExplorationResult") -> str:
"""The ONE line an exploration says about itself before the pipeline takes over.
Always a line, never an omission — unlike ``cost_baseline_notice`` and
``skipped_links_notice``, which qualify a run that happened anyway. This one is printed only on
a surface where ``--explore`` was asked for, so there is no run it could speak for silently.
``stop`` is named when there is one, because a mandate shaped by a loop that gave up is a
smaller mandate than the same loop finishing would have produced, and nothing else on stdout
would say so.
"""
ended = "concluded" if result.stop is None else f"stopped ({result.stop})"
return (
f"Exploration: {ended} after {len(result.ledger_log)} round(s); "
f"{len(result.mandate.approaches)} approach(es) to evaluate"
)
def parked_notice(parked: "ParkedExploration", *, run_id: str) -> str:
"""The ONE thing a parked exploration says at the terminal it stopped on.
Always a line, for the reason ``exploration_notice`` always is: it is printed only where the
asynchronous door was asked for. It names the run id, because that is the single coordinate
``--resume`` takes, and it is the one an operator will be looking for weeks later.
It does NOT print the plan. The plan is in the question artefact, where the expert who has to
read it will be looking — and this terminal belongs to whoever STARTED the run, who is not
necessarily them.
"""
return (
f"Exploration parked: plan review {parked.index} of run {run_id!r} is waiting for a "
f"human. The question is in {run_id}-plan-review.json; answer it with "
f"{run_id}-plan-review-answer.json in a review inbox, then --resume {run_id}"
)
def load_exploration_contract(path: str | Path) -> ExplorationContract:
"""Fail-fast standalone loader for an exploration's bounds (mirrors ``mandate.load_mandate``).
The bounds are authoritative startup input, and the ONE thing that keeps a Magentic loop from
being unbounded — so a missing or malformed file refuses the run rather than degrading to
defaults, which is the shape ``ExplorationContract`` refuses to have in the first place.
:raises FileNotFoundError: ``path`` does not point at an existing file.
:raises pydantic.ValidationError: the content is not JSON, or violates the contract.
"""
p = Path(path)
if not p.is_file():
raise FileNotFoundError(f"exploration config not found: {str(path)!r}")
return ExplorationContract.model_validate_json(p.read_text(encoding="utf-8"))
#: The manager's role name. Not a participant: the manager plans, picks the next speaker and
#: keeps the progress ledger, and ``BudgetMiddleware`` rides on it like on everyone else (A1,
#: measured green — without that, the most talkative agent in the loop would be the one outside
#: the token cap).
MANAGER_ROLE: Final = "manager"
#: Reads the knowledge bases PROGRESSIVELY (§3 Steg 1) — index, then whatever the index links.
NAVIGATOR_ROLE: Final = "navigator"
#: Shapes ONE candidate direction at a time and may have its numbers advisory-checked in-loop.
HYPOTHESISER_ROLE: Final = "hypothesiser"
#: The participants, in build order. The manager's ``next_speaker`` is validated against exactly
#: these names: a ledger naming anyone else makes the orchestrator emit a final answer having
#: asked NOBODY (measured, ``_magentic.py:1128-1131``), which is a plausible answer produced by
#: zero work — the hazard class this repo retired ``E2`` for.
PARTICIPANT_ROLES: Final = (NAVIGATOR_ROLE, HYPOTHESISER_ROLE)
#: The span every exploration records its decisions under. One span per exploration, with the
#: manager's decisions as span EVENTS on it, because those are moments INSIDE one activity rather
#: than activities of their own — and because a reader wants them in order, on one timeline.
EXPLORATION_SPAN: Final = "exploration"
#: The line prefix that turns a hypothesiser turn into a candidate direction. A marker rather than
#: "parse every turn as JSON" because most turns legitimately are not hypotheses — the agent also
#: reasons out loud. That distinction is what lets an UNPARSEABLE marked line be a hard error
#: instead of a silence: with no marker there is nothing to be silent about.
HYPOTHESIS_MARKER: Final = "HYPOTHESIS:"
_INSTRUCTIONS: Final = {
NAVIGATOR_ROLE: (
"You read the project's knowledge bases, one level at a time. Use list_bundles to see "
"what exists, read_bundle to open ONE of them and see its top level, read_dir to open a "
"directory that listing named, and read_file to read a document you picked. read_bundle "
"and read_dir return LISTINGS, never the documents — a knowledge base can hold hundreds, "
"so descend to the part that matters instead of asking for all of it. A listing is a "
"WINDOW: it reports 'total' for the level and gives you 'limit' of them from 'offset', so "
"when 'total' is large do not page through it — pass read_dir a 'filter' word and read the "
"'total_matches' it reports. Quote only what read_file gave you, and never guess at "
"content you have not read: a path you invent is refused, it does not find a neighbour."
),
HYPOTHESISER_ROLE: (
"You shape ONE candidate cost-saving direction at a time from what the navigator found. "
"BEFORE you commit to a direction, name the ONE requirement in the knowledge base that "
"BINDS it: pass read_dir a 'filter' word taken from the approach's own label — "
"filter='rundkjoring' finds the level's requirements about roundabouts, and one of them is "
"the 'Krav 4.1.2-1' you are looking for — read it with read_file, then call "
"declare_requirement with the base id, that path and the requirement's own number. The "
"reply gives back the document's own title and number: if they are not about your measure, "
"you declared the wrong requirement and should filter again. A direction with no "
"requirement behind it is a guess. "
"You may call quick_validate to sanity-check a candidate's numbers; its verdict is "
"ADVISORY and is not the project's decision. When you commit to a direction, end your "
f'turn with a line of the form: {HYPOTHESIS_MARKER} {{"label": "<short name>", '
'"rationale": "<why this project, in your own words>", "requirement": '
'{"path": "<the path you read>", "ref": "<the requirement number>"}}. If the base '
'genuinely holds no requirement for this direction, send "requirement": null and '
'"why_none": "<why the base has none>" instead — the field is never omitted.'
),
}
class ExplorationError(RuntimeError):
"""The exploration cannot be honoured as configured, or produced something unreadable."""
class VerdictLayerRefused(ValueError):
"""A navigator asked ``read_file`` for a document belonging to the ``type: verdict`` layer.
Prior expert judgements reach a hypothesis through ONE door — the gated ExpeL fold inside
``run_project``, keyed on the candidate (S3.2) and folded before generation. Everything else in
the bundle is context an agent may navigate; a verdict is not, because a run that reads its own
corpus of past judgements as ordinary knowledge has routed around the gate that decides which
of them are relevant, and self-contamination is exactly what målbilde §4 excludes the layer to
prevent.
A ``ValueError``, the ``DimensionScopeRefused``/``BundlePathNotFound`` precedent: the caller is
a model choosing a path, so the refusal lands on the CLI's refusal tuple and hosting's 400 arm
rather than the crash channel. Like the dimension scope it is a refused read inside a run that
is otherwise fine, never an exploration that cannot be honoured as configured.
"""
class DimensionScopeRefused(ValueError):
"""A navigator asked for a document belonging to ANOTHER dimension than the run is scoped to.
A ``ValueError``, the ``BundlePathNotFound``/``BundleIdMismatch`` precedent: the caller is a
model choosing a path, so the refusal must land on the CLI's refusal tuple and hosting's 400
arm rather than the crash channel. It is deliberately NOT an ``ExplorationError``
(a ``RuntimeError``): this is a refused read inside a run that is otherwise fine, not an
exploration that cannot be honoured as configured.
"""
class DirectoryPathRefused(ValueError):
"""``read_file`` was asked for a DIRECTORY — the wrong rung of the navigation ladder.
Measured against a live model on K2 (``docs/2026-09-06-major2-levende-k2.md`` § 3/§ 4, finding
(c)): a listing hands back subdirectories and concept documents as two distinct keys, and the
model asked ``read_file`` for one of the directories anyway, three times in one run. Before
this the answer was ``IsADirectoryError`` — an ``OSError``, so it left by the crash channel,
and it named neither what the path was nor which tool reads it.
A ``ValueError``, the ``BundlePathNotFound``/``DimensionScopeRefused`` precedent: the caller is
a model choosing a path, so a refusal belongs on the CLI's refusal tuple and hosting's 400 arm.
The message names ``read_dir`` because a refusal that only says "no" leaves the caller with the
same next move it just made.
"""
@dataclass(frozen=True)
class LedgerEntry:
"""One progress-ledger round, reduced to the five fields the manager steers on (C.2).
``speaker_known`` is not part of the ledger MAF produces — it is this layer's reading of it,
and the reason the whole entry is recorded rather than counted. A ``next_speaker`` matching no
participant does not raise: the orchestrator logs a warning and jumps straight to a final
answer, so the run returns a plausible result that nobody worked for. Recording the fact is
what lets ``explore()`` refuse to hand that result onward as if it had been explored.
"""
round_index: int
is_request_satisfied: bool
is_in_loop: bool
is_progress_being_made: bool
next_speaker: str
instruction_or_question: str
speaker_known: bool
@dataclass(frozen=True)
class PlanReview:
"""One synchronous plan-review round trip (U13, door 2 of § C.6).
``is_stalled`` is carried because the same request type serves two very different moments:
the initial sign-off, and a re-review after the manager reset and replanned. A reviewer that
cannot tell them apart cannot answer the second one usefully.
"""
index: int
plan: str
is_stalled: bool
decision: Literal["approve", "revise"]
feedback: str = ""
@dataclass(frozen=True)
class QuickValidation:
"""One advisory (level 1) verdict, as the tool answered it.
``proposal_json`` is kept VERBATIM for the reason the parse-failure capture keeps the raw reply
verbatim: the operative question after a run is what the hypothesiser actually asked about, and
a re-serialised form would answer a different one.
"""
bundle_id: str
proposal_json: str
verdict: Mapping[str, Any]
@dataclass(frozen=True)
class ToolCall:
"""One exploration tool invocation: what was asked for, never what came back.
``bundle_id`` is the base the call named, and ``path`` the document or directory inside it —
each ``""`` for a tool that does not take it (``list_bundles`` takes neither, ``read_bundle``
takes only the base). The RESULT is deliberately absent: it is the base's content, which is the
very thing that is too big to ride along (MAJOR-3 measured it at 73-93 % of all prompt tokens),
and a trace carrying it would be a second copy of the context rather than a record of the run.
``path`` was added in S7a-3 pkt. 3, and the measurement that forced it is scale: over the three
example bases the name and the base id were enough to see what a run had done, but over K2's
**629 concepts** a trace reading ``read_file, k2`` twice answers nothing about WHICH two were
opened. It is recorded for ``read_file`` and for ``read_dir`` — a trace that named the documents
but not the directories would say where a run ended without saying how it got there.
"""
name: str
bundle_id: str
path: str
#: P19 DEL C — HOW the listing was asked for, not just WHICH one. P18 gave ``read_dir`` a
#: window (``filter``/``offset``/``limit``) and the trace could not say whether a model used
#: it: five of 31 documents read in round 2 lay outside the default window, so the window HAD
#: been widened, and nothing said with which knob. Empty/zero mean "not passed" — the
#: ``bundle_id``/``path`` rule one field over, and unambiguous here because ``limit`` is
#: clamped to at least one wherever it is given.
filter: str = ""
offset: int = 0
limit: int = 0
@dataclass(frozen=True)
class DeclaredRequirement:
"""One requirement a navigating role DECLARED as binding, after reading it (P19 DEL A).
Recorded on a CALLER-OWNED sink for the reason ``ToolCall`` is: the declaration is made mid-run
by a tool, and a budget stop after it constructs no result at all — so a returned value would
lose exactly the evidence a paid run was bought for.
"""
bundle_id: str
path: str
ref: str
class RequirementNotRead(ValueError):
"""A role declared a binding requirement it never opened (P19 A2).
**A refusal the model can act on, never a raise that ends the run.** The declaration is a claim
about the corpus, and the cheapest falsifier of it is the run's own read trace: a path that is
not among this run's ``read_file`` calls was not read, whatever the declaration says. Answering
that as a refused TURN — ``quick_validate``'s mechanism, one field over — leaves the model able
to go and read it; raising would end a run over a mistake that costs one tool call to fix.
A ``ValueError``, the ``BundlePathNotFound``/``DimensionScopeRefused`` precedent: the caller is
a model choosing a path, so if it ever escapes a tool body it belongs on the CLI's refusal
tuple and hosting's 400 arm rather than the crash channel.
"""
def _string_argument(arguments: Any, key: str) -> str:
"""One named argument of a call, from either shape ``FunctionInvocationContext`` allows.
``arguments`` is typed ``BaseModel | Mapping[str, Any]`` (measured against the installed
signature), so both are read rather than one being assumed. A tool without the parameter — or
a value that is not a string — yields ``""``: the recorder describes the call, and inventing a
value for an argument that was never passed would be the false-attribution that
``ToolCallRecorder`` refuses for unconfigured tools.
ONE reader for both fields (kø-(p)): two copies of "read this key out of either shape" would be
free to disagree about what an absent argument means.
"""
if isinstance(arguments, Mapping):
value: Any = arguments.get(key)
else:
value = getattr(arguments, key, None)
return value if isinstance(value, str) else ""
def _number_argument(arguments: Any, key: str) -> int:
"""One NUMERIC argument of a call — ``_string_argument``'s sibling, and deliberately separate.
A model may send ``limit`` as ``10`` or as ``"10"`` (both reach a tool through the same wire),
so a recorder that read only the first shape would say a paging call was unpaged. Anything that
is neither is ``0``: the recorder describes the call, and inventing a number for an argument
nobody passed would be the false attribution ``_string_argument`` refuses for its own field.
``bool`` is excluded explicitly because it is an ``int`` in Python and a flag is not a window.
"""
if isinstance(arguments, Mapping):
value: Any = arguments.get(key)
else:
value = getattr(arguments, key, None)
if isinstance(value, bool):
return 0
if isinstance(value, int):
return value
if isinstance(value, str) and value.strip().lstrip("-").isdigit():
return int(value.strip())
return 0
class ExplorationToolRecorder(FunctionMiddleware):
"""Records WHICH exploration tool an agent actually called, in the order it called them.
**A sibling of ``mcp_tools.ToolCallRecorder``, never a reuse of it** — the same shape, one
layer over: that one observes the debate's EXTERNAL tool calls, this one the exploration's
IN-PROCESS ones. Their invariants are opposites and generalising one to serve both would break
the other. ``ToolCallRecorder`` filters to configured servers, de-duplicates per
``(server, tool)`` and returns SORTED, because its record is an egress claim stamped into a
byte-deterministic artefact. This one keeps every call in INVOCATION ORDER without dedup,
because the question it answers is the opposite one: did the navigator open anything, and in
what sequence. A sorted, de-duplicated set cannot tell a rehearsal that read a base from one
that only listed them, which is the whole of MAJOR-1.
It observes only — ``call_next`` is always awaited, and nothing here can block, alter or
short-circuit an invocation. A trace that changed the run it traces would not be a trace.
"""
def __init__(self, sink: list[ToolCall]) -> None:
self._sink = sink
async def process(
self, context: FunctionInvocationContext, call_next: Callable[[], Awaitable[None]]
) -> None:
name = getattr(getattr(context, "function", None), "name", None)
if isinstance(name, str):
arguments = getattr(context, "arguments", None)
self._sink.append(
ToolCall(
name=name,
bundle_id=_string_argument(arguments, "bundle_id"),
path=_string_argument(arguments, "path"),
filter=_string_argument(arguments, "filter"),
offset=_number_argument(arguments, "offset"),
limit=_number_argument(arguments, "limit"),
)
)
await call_next()
@dataclass
class ExplorationTrace:
"""The CALLER-owned accumulator for everything one exploration produced along the way.
**Why a caller-owned sink and not a return value** — the same measurement that shaped the
parse-failure capture (Fase 1b, funn 1), one layer up. ``explore()`` raises ``BudgetExceeded``
at its round cap, and a token cap fires from inside ``BudgetMiddleware`` mid-run; on both paths
no ``ExplorationResult`` is ever constructed. § C.2 requires the exploration artefact to be
readable "uansett hvilken vakt som fyrte", so the evidence has to live somewhere that survives
the ending — which a return value, by definition, does not. Step 5's opposite rule ("a returned
value, never an out-parameter") governs a value that DOES reach the caller; copying it blindly
here would rebuild the very defect it was written against.
``ExplorationResult.ledger_log`` and ``.plan_reviews`` are built FROM these lists rather than
accumulated beside them: two containers holding one fact drift (kø-(p)), and a drifted pair
would let the returned result and the written artefact describe different runs.
"""
ledger: list[LedgerEntry] = field(default_factory=list)
plan_reviews: list[PlanReview] = field(default_factory=list)
quick_validations: list[QuickValidation] = field(default_factory=list)
#: Every exploration tool call, in order (MAJOR-1). It sits BESIDE ``quick_validations``
#: rather than inside it: that list is the level-1 VERDICTS the hypothesiser saw, this is
#: whether any base was opened at all. An offline rehearsal that called nothing looks exactly
#: like a successful one on every other field, which is what made the dress rehearsal vacuous
#: by construction and unreadable after the fact.
tool_calls: list[ToolCall] = field(default_factory=list)
#: Every requirement a role DECLARED as binding, in declaration order (P19 DEL A). Beside
#: ``tool_calls`` rather than derived from it: the trace says which documents were opened, this
#: says which one the model committed to as the thing that binds — two different facts, and the
#: second cannot be inferred from the first.
requirements: list[DeclaredRequirement] = field(default_factory=list)
#: Tokens spent so far, refreshed as the loop turns rather than written once at the end. The
#: meter is internal to ``explore``, so this is the only way the artefact can report a spend —
#: and updating it per iteration is what makes it readable for a run a cap cut short, which is
#: the same reason the sink exists at all. Across a park it is what the resumed leg adds to.
tokens_spent: int = 0
def tool_call_payload(calls: Sequence[ToolCall]) -> list[dict[str, Any]]:
"""The ONE rendering of a tool trace into plain data, in CALL ORDER.
Two surfaces now write one: ``trace_payload`` for ``{run_id}-exploration.json`` and
``run_project`` for ``{run_id}-debate.json`` (S2c). Two copies of "what a recorded call looks
like" would drift into two answers about the same fact, which is the kø-(p) defect — and here
the drift would land in the artefacts an operator reads to find out what a paid run opened.
Plain mappings only, so the RAW output layer stays MAF-free (``outbox.py`` may not import
this module).
"""
return [
{
"name": call.name,
"bundle_id": call.bundle_id,
"path": call.path,
# P19 DEL C: HOW the level was asked for. Always present, zero/empty when not passed —
# the ``write_debate_tools`` rule one field down: an absent key and "not narrowed" must
# not be the same reading.
"filter": call.filter,
"offset": call.offset,
"limit": call.limit,
}
for call in calls
]
def trace_payload(
trace: ExplorationTrace,
*,
stop: str | None,
completed: bool,
mandate: Mandate | None,
prepass: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
"""The ONE rendering of a trace into plain data for ``outbox.write_exploration``.
Plain mappings only, so the RAW output layer stays MAF-free (the ``write_parse_failures``
precedent — ``outbox.py`` may not import this module).
``completed`` is a required field rather than an inference from ``stop``. With no result there
is no stop, and a ``stop: null`` meaning BOTH "concluded normally" and "we never found out"
is exactly the silence ``ProvenanceStamp.cost_baseline_anchored`` was made required to close.
``mandate`` is required for the same reason, applied one field over: before this the
approaches a loop SHAPED reached only the stdout announcement (``mandate.announce``), so a
caller who kept the artefact but not the terminal had no way to learn what the run had
decided to evaluate (measured on K2, S7b's own uttalte grense). ``None`` is not "zero
approaches" — ``ExplorationResult.mandate`` always carries at least the seeded ones, so the
only way to reach here with no mandate is a run that never produced one (a cap that fired, or
a park), which is exactly what ``completed=False`` already says. Collapsing that into an
empty list would make "the loop formed no approaches" and "the loop never got that far"
unreadable from each other.
``prepass`` is the declared cut this exploration was SEEDED with (``--prepass-seed``), already
rendered by ``prepass.declaration_payload`` so this module and the outbox both stay free of
that dependency. It DEFAULTS to ``None``, unlike ``completed`` and ``mandate`` above: absence
here is an honest positive statement — no cut was given — which is ``Bundle.skipped``'s empty
tuple rather than ``cost_baseline_anchored``'s required boolean, and it is the same decision
``RunResult.prepass`` already made one surface over.
"""
return {
"completed": completed,
"stop": stop,
"prepass": dict(prepass) if prepass is not None else None,
"tokens_spent": trace.tokens_spent,
"approaches": [
{
"id": approach.id,
"label": approach.label,
"description": approach.description,
"affected_codes": list(approach.affected_codes),
"claimed_saving_nok": approach.claimed_saving_nok,
"bundle_id": approach.bundle_id,
}
for approach in (mandate.approaches if mandate is not None else ())
],
"rounds": [
{
"round_index": entry.round_index,
"is_request_satisfied": entry.is_request_satisfied,
"is_in_loop": entry.is_in_loop,
"is_progress_being_made": entry.is_progress_being_made,
"next_speaker": entry.next_speaker,
"instruction_or_question": entry.instruction_or_question,
"speaker_known": entry.speaker_known,
}
for entry in trace.ledger
],
"plan_reviews": [
{
"index": review.index,
"plan": review.plan,
"is_stalled": review.is_stalled,
"decision": review.decision,
"feedback": review.feedback,
}
for review in trace.plan_reviews
],
"quick_validations": [
{
"bundle_id": call.bundle_id,
"proposal_json": call.proposal_json,
"verdict": dict(call.verdict),
}
for call in trace.quick_validations
],
"tool_calls": tool_call_payload(trace.tool_calls),
# P19 DEL A: what this run declared as binding, beside what it opened. An empty list is an
# honest positive statement — the run declared nothing — which is ``Bundle.skipped``'s
# empty tuple rather than a field that has to be inferred from an absence.
"requirements": requirement_payload(trace.requirements),
}
#: Why an exploration ended without a satisfied request. ``None`` means it concluded normally.
#: Resource exhaustion is NOT here: tokens and rounds raise ``BudgetExceeded`` (the 429 channel),
#: because "we ran out" and "we finished, unsatisfied" are the two things S3.4 split apart and a
#: single field would fuse back together.
ExplorationStop = Literal["stalled", "plan_revisions_exhausted", "unknown_speaker"]
@dataclass(frozen=True)
class ExplorationResult:
"""What one exploration produced. The mandate is the product; the rest is the evidence.
``mandate`` is ALWAYS present, including on a stop: an exploration that was cut short still
carries the expert's seed approaches forward, because door 1 of § C.6 is a preservation rule
and not a reward for finishing. ``stop`` is what says the mandate is smaller than it might
have been, and why.
"""
mandate: Mandate
ledger_log: tuple[LedgerEntry, ...]
stop: ExplorationStop | None
plan_reviews: tuple[PlanReview, ...]
@dataclass(frozen=True)
class PlanReviewRequest:
"""What the reviewer is shown before the loop is allowed to run."""
index: int
plan: str
current_progress: str
is_stalled: bool
@dataclass(frozen=True)
class PlanReviewDecision:
"""The reviewer's answer. ``feedback is None`` means approve.
Two named constructors mirroring ``MagenticPlanReviewResponse.approve()/.revise()``, so a
persona or operator answering a review never has to import ``agent_framework`` — the adapter
is one-directional and lives in exactly one place.
"""
feedback: str | None
@staticmethod
def approve() -> PlanReviewDecision:
return PlanReviewDecision(feedback=None)
@staticmethod
def revise(feedback: str) -> PlanReviewDecision:
if not feedback.strip():
raise ValueError("a revision must say what to revise; use approve() to sign off")
return PlanReviewDecision(feedback=feedback)
#: The synchronous HITL seam (U13). Given the request, answer it. Called in-process, so the
#: exploration blocks on it exactly as a human at a terminal would block the loop.
PlanReviewer = Callable[[PlanReviewRequest], PlanReviewDecision]
class PlanReviewInputError(ExplorationError):
"""A terminal plan review was left without an answer: the input ended mid-review.
A distinct type rather than a distinguishing message, for the reason ``_classify_stop`` reads
counts instead of the termination prose: a caller deciding what happened should never have to
match wording. It is an ``ExplorationError`` (a ``RuntimeError``) because the run FAILED — the
caller's argv was fine and the loop had already started spending; the same channel an
unreadable marked hypothesis leaves by.
"""
#: The closed answer vocabulary of the terminal door. Two words, matched structurally.
_APPROVE_ANSWER: Final = "approve"
_REVISE_ANSWER: Final = "revise"
def terminal_plan_reviewer(
*, stream_in: TextIO | None = None, stream_out: TextIO | None = None
) -> PlanReviewer:
"""A ``PlanReviewer`` that asks the operator at a terminal and reads their typed answer.
This is the door that makes målbilde §3's "still spørsmål, be om svar, bruke svarene" reachable
without importing the package (F4): ``run.py``'s ``--plan-review`` builds one of these and
hands it to ``explore()``. Blocking is not an oversight — ``explore()`` calls the reviewer
synchronously (it is not awaited), so the loop waits on the human exactly as the ``PlanReviewer``
contract says. That is also the reason the hosted surface keeps refusing the review: there,
blocking the reviewer would block the event loop that answers ``/readiness``.
**The streams are resolved at CALL time, not here** (the ``shared_root()`` idiom): a factory
that captured ``sys.stdin`` at construction could not be driven by a caller — or a test — that
replaces the stream afterwards, and the only way left to exercise the door would be a
subprocess.
**Fail-closed on the operator's own input.** ``approve`` signs off; ``revise <what to change>``
sends the words back to the manager. Anything else — a blank line, a typo, a bare ``revise`` —
is asked AGAIN, never taken as a decision. End of input raises ``PlanReviewInputError``:
reading silence as approval would let an autonomous loop run on a plan no human signed, and do
it invisibly. Validation, NEVER repair (the ``write_concept_file`` rule).
"""
def review(request: PlanReviewRequest) -> PlanReviewDecision:
source = sys.stdin if stream_in is None else stream_in
sink = sys.stdout if stream_out is None else stream_out
stalled = " (a RE-PLAN after a stall)" if request.is_stalled else ""
print(f"\nPLAN REVIEW #{request.index + 1}{stalled}", file=sink)
print("--- the plan the exploration would run ---", file=sink)
print(request.plan, file=sink)
print("--- progress so far ---", file=sink)
# Stated, never omitted, and never a repr of absence. An expert about to sign has to be
# able to tell "the loop has not reported progress yet" from "this section was dropped".
print(
request.current_progress
if request.current_progress.strip()
else "(no progress ledger yet)",
file=sink,
)
while True:
print(
f'Answer "{_APPROVE_ANSWER}" to sign it off, '
f'or "{_REVISE_ANSWER} <what to change>": ',
file=sink,
)
sink.flush()
line = source.readline()
if line == "":
raise PlanReviewInputError(
"the plan review reached end of input without an answer. Silence is not a "
"sign-off: the exploration will not run a plan nobody approved"
)
answer = line.strip()
if answer == _APPROVE_ANSWER:
return PlanReviewDecision.approve()
verb, _, feedback = answer.partition(" ")
if verb == _REVISE_ANSWER and feedback.strip():
return PlanReviewDecision.revise(feedback.strip())
print(f"Not an answer: {answer!r}.", file=sink)
return review
# ---------------------------------------------------------------------------------------------
# U12 — the ASYNCHRONOUS half of the same door: a review answered days later, in another process.
# ---------------------------------------------------------------------------------------------
#: The two types a plan-review checkpoint carries, in ``"module:qualname"`` form.
#:
#: **This tuple is the whole of the measured trap** (plan § F, A4; re-measured against the
#: installed source, ``_workflows/_checkpoint.py:386-388``): ``FileCheckpointStorage`` runs a
#: restricted unpickler, and ``list_checkpoints`` swallows a blocked type into a ``logger.warning``
#: and returns an EMPTY list. Omit either name and the checkpoint is written but comes back
#: unreadable, so a resume fails as an ABSENCE — "nothing to resume" — rather than as an error.
#: ONE copy, used by every process that touches this storage, because both the writing process and
#: the resuming one must declare them and a second copy is the drift kø-(p) exists to prevent.
_ALLOWED_CHECKPOINT_TYPES: Final[tuple[str, ...]] = (
"agent_framework_orchestrations._magentic:MagenticPlanReviewRequest",
"agent_framework_orchestrations._magentic:MagenticPlanReviewResponse",
)
def checkpoint_storage(checkpoint_dir: str | Path) -> FileCheckpointStorage:
"""The ONE construction site for the exploration's checkpoint storage.
A caller that built its own ``FileCheckpointStorage`` would have to remember the allow-list
above, and forgetting it is invisible (see ``_ALLOWED_CHECKPOINT_TYPES``). Routing every
construction through here makes "both processes declare the types" a structural property
rather than a convention two call sites have to keep.
"""
return FileCheckpointStorage(
str(checkpoint_dir), allowed_checkpoint_types=list(_ALLOWED_CHECKPOINT_TYPES)
)
class CheckpointUnreadable(ExplorationError):
"""The exploration stopped at a review but left nothing a later process could resume from.
Raised where the framework is silent: an empty listing means the checkpoint could not be read
back, and parking anyway would hand an expert a question whose answer can never be applied.
Failing here is the fourth face of the verification law written into our own surface.
"""
class ParkedStateError(ExplorationError):
"""A parked-state file that cannot be read as one.
Fail-fast, NOT the tolerant RAW-inbox rule: this file is the run's own suspended state, the
same class as the spend file ``read_spend`` refuses to read loosely. Treating a malformed one
as "no parked run" would silently drop an exploration a human is waiting to answer.
"""
@dataclass(frozen=True)
class ParkedExploration:
"""Everything needed to resume one suspended exploration in a process that never saw it.
Two files cross the boundary and they own different halves. MAF's checkpoint holds the
WORKFLOW's state (the manager's ledgers, the pending request); this holds the EXPLORATION
LAYER's — what has been spent, what the loop has already found, and which question is open.
Neither can reconstruct the other, so both are named here rather than one being inferred.
``tokens_spent`` and ``ledger`` are carried for a reason that is not bookkeeping. Both budget
channels live in the process: a resume with a fresh ``TokenMeter`` and an empty ledger would
get the whole cap AGAIN, once per park — unbounded consumption behind guards that all report
themselves satisfied, which is the class S3.4 split apart. Carrying them makes the cap span
the suspension.
``hypotheses`` are the marked turns the loop produced BEFORE parking. They are verbatim, and
they are carried for the same reason: minting the mandate from only what the resuming process
observed would silently drop everything found before a stalled re-review.
"""
prompt: str
request_id: str
checkpoint_id: str
index: int
plan: str
current_progress: str
is_stalled: bool
bundle_dirs: tuple[str, ...]
contract: ExplorationContract
ledger: tuple[LedgerEntry, ...]
plan_reviews: tuple[PlanReview, ...]
hypotheses: tuple[str, ...]
tokens_spent: int
replans: int
class PlanReviewParked(ExplorationError):
"""The exploration is suspended at a plan review, waiting for a human.
**A raise rather than a return, and the argument is the 429 one** (kø-(y), 14.08). A parked run
produced NO mandate: the loop is stopped mid-plan and nothing has been explored yet. Handing
back an ``ExplorationResult`` would let an automated caller book "explored" for a run that
explored nothing — the same reason an exhausted budget is not a 200 even though it is not a
crash either. The coordinates travel as STRUCTURE on ``parked``, never as ``str(exc)``.
"""
def __init__(self, parked: ParkedExploration) -> None:
super().__init__(
f"exploration parked at plan review {parked.index} "
f"(request {parked.request_id}, checkpoint {parked.checkpoint_id})"
)
self.parked = parked
def parked_payload(parked: ParkedExploration) -> dict[str, Any]:
"""The ONE rendering of a parked exploration into plain data for ``outbox.write_plan_review``.
Plain mappings only, so the RAW output layer stays MAF-free — the ``trace_payload`` precedent,
for the same reason: ``outbox.py`` may not import this module.
"""
return {
"prompt": parked.prompt,
"request_id": parked.request_id,
"checkpoint_id": parked.checkpoint_id,
"index": parked.index,
"plan": parked.plan,
"current_progress": parked.current_progress,
"is_stalled": parked.is_stalled,
"bundle_dirs": list(parked.bundle_dirs),
"contract": parked.contract.model_dump(),
"ledger": [
{
"round_index": entry.round_index,
"is_request_satisfied": entry.is_request_satisfied,
"is_in_loop": entry.is_in_loop,
"is_progress_being_made": entry.is_progress_being_made,
"next_speaker": entry.next_speaker,
"instruction_or_question": entry.instruction_or_question,
"speaker_known": entry.speaker_known,
}
for entry in parked.ledger
],
"plan_reviews": [
{
"index": review.index,
"plan": review.plan,
"is_stalled": review.is_stalled,
"decision": review.decision,
"feedback": review.feedback,
}
for review in parked.plan_reviews
],
"hypotheses": list(parked.hypotheses),
"tokens_spent": parked.tokens_spent,
"replans": parked.replans,
}
def load_parked(payload: Mapping[str, Any]) -> ParkedExploration:
"""Read a parked-state payload back, fail-fast (``ParkedStateError`` on anything missing).
Validation, never repair (the ``write_concept_file`` rule): a payload that has lost, say, its
``checkpoint_id`` describes a suspension nobody can lift, and defaulting it would produce a
resume that looks like one and is not.
"""
try:
return ParkedExploration(
prompt=str(payload["prompt"]),
request_id=str(payload["request_id"]),
checkpoint_id=str(payload["checkpoint_id"]),
index=int(payload["index"]),
plan=str(payload["plan"]),
current_progress=str(payload["current_progress"]),
is_stalled=bool(payload["is_stalled"]),
bundle_dirs=tuple(str(d) for d in payload["bundle_dirs"]),
contract=ExplorationContract.model_validate(payload["contract"]),
ledger=tuple(
LedgerEntry(
round_index=int(row["round_index"]),
is_request_satisfied=bool(row["is_request_satisfied"]),
is_in_loop=bool(row["is_in_loop"]),
is_progress_being_made=bool(row["is_progress_being_made"]),
next_speaker=str(row["next_speaker"]),
instruction_or_question=str(row["instruction_or_question"]),
speaker_known=bool(row["speaker_known"]),
)
for row in payload["ledger"]
),
plan_reviews=tuple(
PlanReview(
index=int(row["index"]),
plan=str(row["plan"]),
is_stalled=bool(row["is_stalled"]),
decision="revise" if row["decision"] == "revise" else "approve",
feedback=str(row["feedback"]),
)
for row in payload["plan_reviews"]
),
hypotheses=tuple(str(h) for h in payload["hypotheses"]),
tokens_spent=int(payload["tokens_spent"]),
replans=int(payload["replans"]),
)
except (KeyError, TypeError, ValueError, ValidationError) as exc:
raise ParkedStateError(f"parked exploration state is unreadable: {exc}") from exc
# ---------------------------------------------------------------------------------------------
# The tools. Level 1 of the three-guarantee table: real computation, ADVISORY verdicts.
# ---------------------------------------------------------------------------------------------
def _bundle_index(bundle_dirs: Sequence[str]) -> dict[str, str]:
"""Map each knowledge base's id to its directory — the DECLARED id where the base declares one,
the directory's basename otherwise (``okf.reconcile_bundle_id``'s rule, S7a-3 pkt. 1).
**Why this door resolves the declaration at all — a CONSEQUENCE of the slacken, not scope
creep.** This used to be ``Path(raw).name`` while ``run_mandate_across_bundles`` used
``reconcile_bundle_id(raw).id``. While a declared id that disagreed with its mount was refused
outright the two could not differ. With declared-wins they can, and then ``explore()`` mints
approaches naming the MOUNT while the dispatcher routes by the DECLARATION — an exploration
whose own mandate is unroutable.
**Unreadable falls back to the basename rather than raising, and that is measured.**
``test_explore_loadbearing.py`` configures ``/tmp/base-a`` and ``/tmp/one/shared-name`` —
directories that do not exist — and expects an ExplorationError about IDs, not an I/O error.
Nothing is widened by the fallback: this index answers "which ids may be NAMED", and a base
nobody can read is refused a moment later by whichever door actually opens it.
A duplicate id is REFUSED rather than resolved by order: the id is what the manager
names a base by, and two bases answering to one name would let it read A while believing it
read B — the S3.2 key-collision class, one layer up.
"""
index: dict[str, str] = {}
for raw in bundle_dirs:
try:
bundle_id = okf.reconcile_bundle_id(raw).id
except ValueError:
bundle_id = Path(raw).name
if bundle_id in index:
raise ExplorationError(
f"two knowledge bases share the id {bundle_id!r} ({index[bundle_id]!r} and "
f"{raw!r}); the manager names a base by that id, so it must be unique"
)
index[bundle_id] = raw
return index
def _resolve_bundle(index: Mapping[str, str], bundle_id: str) -> str:
if bundle_id not in index:
known = ", ".join(sorted(index)) or "(none configured)"
raise ExplorationError(f"unknown knowledge base {bundle_id!r}; configured: {known}")
return index[bundle_id]
def _declared_document(index: Mapping[str, str], bundle_id: str, path: str) -> tuple[str, str]:
"""``(title, reference_number)`` of the document at ``path``, or ``("", "")`` when the base has
no navigated concept under that name.
Read off the SAME ``Bundle.context_files`` every listing rung is built from (MAJOR-3/S7a-3), so
a declaration can never be answered with the title of a ``type: verdict`` document — the one
layer no listing names and ``read_file`` refuses outright.
A path the base does not carry as a concept — ``index.md`` is the reachable case — answers with
two empty strings rather than raising: the declaration itself has already been accepted by the
read-trace check above, and turning "I cannot restate your title" into a refusal would fail a
declaration the run's own trace proves was read.
"""
bundle = okf.navigate_bundle(_resolve_bundle(index, bundle_id))
for file in bundle.context_files:
if file.name == path:
return okf.unquote_scalar(file.frontmatter.get("title", "")), okf.reference_number(file)
return "", ""
#: P21/C1 — how many DISTINCT documents a run must have opened before a declaration of the binding
#: requirement is worth recording. MEASURED, never chosen: see ``_declare_requirement`` for the
#: full distribution and for why the alternative rule the order offered was rejected. Capped by the
#: base's own document count at the call site, so a small base stays declarable.
_MIN_DOCUMENTS_READ: Final = 3
#: Characters of the root index body one catalogue entry may carry. The catalogue's job is to let a
#: manager pick a base, not to read one, so the excerpt is a fixed-size window rather than a share
#: of the base: cost then scales with how many bases are configured, which the operator chose, and
#: not with how much they contain, which they did not. The number is deliberately small — a headline
#: and a first line or two — and the ceiling that guards it lives in the TEST, not here, because
#: raising this constant is the regression the gate exists to catch.
_CATALOGUE_EXCERPT_CHARS: Final = 200
def _index_excerpt(body: str) -> tuple[str, bool]:
"""A bounded VERBATIM prefix of an index body, plus whether anything was cut.
Cut at the last line break inside the budget, never mid-line: half a markdown link is a target a
model may well try to follow, and a path that never existed is worse than no path. A first line
longer than the whole budget has no break to cut at, and is cut hard — the bound is the promise.
Returns the body UNTOUCHED and ``False`` when it already fits: an index that was never cut must
not be reported as cut (omission, never a lie in either direction — ``cost_baseline_notice``'s
rule applied to a value instead of a line).
"""
if len(body) <= _CATALOGUE_EXCERPT_CHARS:
return body, False
cut = body.rfind("\n", 0, _CATALOGUE_EXCERPT_CHARS + 1)
return (body[:cut] if cut > 0 else body[:_CATALOGUE_EXCERPT_CHARS]), True
#: The refusal classes the navigator's read tools turn into an ANSWER instead of an exception,
#: named ONE BY ONE and never as a base class. ``ExplorationError`` is itself a ``RuntimeError``
#: subclass, so an arm written as ``except RuntimeError`` would also swallow a failure nobody
#: named -- turning an unknown fault into a confident-looking answer, which is worse than the
#: opaque string this conversion removes. MEASURED: inside these three tool bodies the only
#: reachable ``ExplorationError`` is ``_resolve_bundle``'s unknown-base refusal.
_RETURNABLE_REFUSALS: Final = (
ExplorationError,
RequirementNotRead,
okf.BundleIdMismatch,
okf.BundlePathNotFound,
okf.DocumentPathRefused,
DirectoryPathRefused,
VerdictLayerRefused,
DimensionScopeRefused,
)
def _refusal_kind(exc: Exception) -> str:
"""The refusal's KIND, from the exception's own class name -- one source, never a second table.
F3 measured that ``read_dir`` must tell "this is a document, use the other rung" apart from
"this does not exist", and before this change a caller could tell them apart by CLASS. A
returned refusal would lose that distinction unless it carries it, so it does.
"""
return type(exc).__name__
def _neighbours(bundle_dir: str, path: str, dimension: str | None) -> tuple[str, tuple[str, ...]]:
"""``(nearest listable ancestor, up to five of its subdirectories)`` for a path that is absent.
ONE navigation for both halves, and BOTH read off ``okf`` rather than reimplemented here: this
module and ``directory_listing``'s own refusal answer the same question about the same base, and
two copies of "which directory did they mean" would be free to give a caller two answers about
one level (kø-(p)).
"""
bundle = okf.navigate_bundle(bundle_dir)
return (
okf.nearest_listable_directory(bundle, path, dimension=dimension),
okf.nearest_subdirectories(bundle, path, dimension=dimension),
)
def _refused_mapping(exc: Exception) -> dict[str, Any]:
"""A listing tool's refusal: a mapping with no key a successful listing has.
``directory_listing`` answers with ``path`` / ``directories`` / ``documents``, so ``refused``
and ``refusal`` cannot be mistaken for a result -- the property that makes a returned refusal
worth anything at all.
"""
return {"refused": str(exc), "refusal": _refusal_kind(exc)}
def _refused_text(exc: Exception) -> str:
"""``read_file``'s refusal, as text, because that tool answers with text.
**Why not a mapping here.** Making ``read_file`` return a dict would change the shape of every
SUCCESSFUL read, and therefore every prompt byte S2c measured when it gave the debate these
tools. The refusal is a string with a fixed leading sentinel instead.
**Honesty limit, stated.** A document whose first characters were exactly this sentinel would
be indistinguishable from a refusal. That is a real gap and it is recorded rather than papered
over; what IS guaranteed is the direction that matters -- a refusal never carries the bytes of
the document it refused, which is the property the verdict and dimension gates exist for.
"""
return f"REFUSED ({_refusal_kind(exc)}): {exc}"
def navigator_tools(
bundle_dirs: Sequence[str],
*,
dimension: str | None = None,
opened: list[ToolCall] | None = None,
requirements: list[DeclaredRequirement] | None = None,
) -> list[FunctionTool]:
"""The navigator's tools: survey the catalogue, open one base, read one document — and, when
the caller offers the two sinks, DECLARE the requirement that binds a direction.
Progressive disclosure, not stuffing (målbilde §2/§4): ``list_bundles`` never returns content,
only what each base IS — and crucially whether it ships a ``cost-baseline.json``, because a base
without one cannot have its hypotheses reconciled against the project's real cost lines, and a
manager that does not know which bases are anchored cannot plan around it (§ C.7).
**The catalogue costs O(bases), never O(corpus)**, and that is the rung's whole point. It used
to return the WHOLE root index body per base plus one object per unfollowed cross-link — both
corpus-sized — so the cheapest rung of the ladder was the most expensive call in the loop:
112 116 o200k_base tokens over three flat Vegnormal bases, 124 942 over the 171 branch bases
that replaced them (measured 25.-26.08; ``tests/test_catalogue_cost_loadbearing.py``). Each
entry is now bounded by construction: a ``_CATALOGUE_EXCERPT_CHARS``-long VERBATIM prefix of the
index body plus counts. The full index stays exactly one ``read_file(id, "index.md")`` away, so
this is a disclosure level, not data loss.
**Truncation is ANNOUNCED, never silent** — ``index_truncated`` is a field beside the excerpt,
not a marker glued into it, for the reason ``BudgetExceeded`` carries its triple as fields
(kø-(y)): a consumer that has to re-parse prose to learn whether it is holding the whole thing
has been handed a diagnostic it cannot act on. And an unfollowed cross-link keeps a COUNT here
(session 51's "a skip is tolerated but no longer silent") while its per-link detail stays where
it is actionable, on ``RunResult.skipped_links`` / ``DryRunReport.skipped_links``.
**``read_bundle`` is the SAME rung one level down, and for the same measured reason.** It used
to return ``okf.bundle_context`` — the whole navigated base. Measured before the change
(``docs/2026-09-02-read-bundle-kontekstkostnad.md``): 3 861 / 10 406 / 12 595 o200k_base tokens
for the three example bases, and because the exploration's participants share one conversation
history, that single ``function_result`` rides in FIVE later prompts — 54-59 % of every
prompt-token in one CLI ``--explore`` run, none of it asked for twice. It now returns the
catalogue form: one entry per concept document (``name``, ``type``, ``title``, ``chars``), with
``read_file`` as the next rung, so a navigator pays for the documents it opened rather than for
the ones it did not. The root index body is deliberately NOT carried — the tunnel base's alone
is 4 763 characters, and ``list_bundles`` already excerpts it while ``read_file(id, "index.md")``
still returns it whole. Ceiling in ``tests/test_read_bundle_cost_loadbearing.py``, never here.
The listing is built from ``Bundle.context_files``, which EXCLUDES the ``type: verdict`` layer
by construction — prior verdicts reach a hypothesis only through the gated ExpeL fold inside
``run_project``, never by being read as context here.
**``dimension`` scopes BOTH rungs, and both halves are the promise** (§4.1a, carried over in
S2c when the DEBATE started navigating instead of being handed ``bundle_context``). The filter
used to live in the rendering; with navigation it has to live in the tools, and a listing that
hides a foreign-dimension document while ``read_file`` still serves it by path is a filter in
name only — a model-chosen path is untrusted input, so the gate belongs where the bytes leave.
``None`` (the exploration's own call) admits everything, byte-identical to before.
**``declare_requirement`` exists only when the caller passes BOTH sinks** (P19 DEL A), and that
is what keeps every other call site byte-identical — including the tool-set assertions three
older gates make. ``opened`` is the SAME list ``ExplorationToolRecorder`` appends to (an alias,
never a copy — the ``_drive`` rule), because the refusal this tool exists for is answered by
the run's own read trace and a second record of it would be free to disagree with the first.
Passing one without the other is refused at construction: a log that cannot see what was opened
would accept every declaration, which is the vacuous-gate class.
"""
if (opened is None) != (requirements is None):
raise ExplorationError(
"navigator_tools takes 'opened' and 'requirements' together or not at all: a "
"requirement sink with no read trace could not refuse an undeclared read, and a read "
"trace with no sink would record nothing"
)
index = _bundle_index(bundle_dirs)
@tool(
name="list_bundles",
description=(
"List the knowledge bases available to this exploration: id, the OPENING of what the "
"index says, how many documents and prior expert verdicts the base holds, whether it "
"ships a cost baseline (without one, numbers cannot be reconciled against the "
"project's own), and how many cross-links could not be followed. The excerpt is cut "
"when index_truncated is true; read_file(id, 'index.md') gives the whole index."
),
)
def list_bundles() -> list[dict[str, Any]]:
catalogue: list[dict[str, Any]] = []
for bundle_id, bundle_dir in index.items():
bundle = okf.navigate_bundle(bundle_dir)
excerpt, truncated = _index_excerpt(bundle.index_summary)
catalogue.append(
{
"id": bundle_id,
"index_excerpt": excerpt,
"index_truncated": truncated,
"documents": sum(
1 for f in bundle.context_files if okf.in_dimension(f, dimension)
),
"verdict_count": len(bundle.verdicts),
# Tolerant on CONTENT, fail-fast on the PATH: an operator's bad directory is
# refused by navigate_bundle above, while a navigable base that simply has no
# baseline is legitimate (load_optional_cost_baseline's own contract).
"cost_baseline": okf.load_optional_cost_baseline(bundle_dir) is not None,
"unreachable_links": len(bundle.skipped),
}
)
return catalogue
@tool(
name="read_bundle",
description=(
"Open ONE knowledge base by id and list its TOP LEVEL: the directories it is divided "
"into (each with how many concept documents its whole subtree holds) and the documents "
"that sit directly at the top, each with name, declared type, title and size in "
"characters. Paths are usable as they are — read_dir(id, path) opens a directory, "
"read_file(id, name) returns one document whole."
),
)
def read_bundle(bundle_id: str) -> dict[str, Any]:
try:
return _read_bundle(bundle_id)
except _RETURNABLE_REFUSALS as exc:
return _refused_mapping(exc)
def _read_bundle(bundle_id: str) -> dict[str, Any]:
bundle_dir = _resolve_bundle(index, bundle_id)
bundle = okf.navigate_bundle(bundle_dir)
# The base is OPENED here, so this is where it must be able to say what it IS (S7a-3
# pkt. 1). A declared id that disagrees with the MOUNT is no longer refused — that is a
# filesystem accident, recorded by the run rather than blocked here — but two concepts
# declaring two different corpora is a base no fallback can settle.
okf.assert_declared_ids_agree(bundle)
# ONE renderer for both rungs (kø-(p)): this tool and ``read_dir`` differ only in WHICH
# level they ask for, and two copies of a listing rule would drift into two answers about
# one bundle. ``okf`` owns it, so the context seam stays framework-neutral.
return okf.directory_listing(bundle, dimension=dimension)
@tool(
name="read_dir",
description=(
"Open ONE directory inside a knowledge base, by base id and the path a previous "
"listing gave you. Answers in the same shape as read_bundle: the directories one level "
"further down, and the concept documents that sit in this one. An unknown path is "
"refused rather than answered as an empty directory. The answer is a WINDOW: 'total' "
"is how many entries the level holds, 'offset'/'limit' say which of them you were "
"given (limit is capped, so ask for the next page instead of a bigger one). Use "
"'filter' to ask for the entries whose title, requirement number or path contains a "
"word -- e.g. read_dir(bundle_id, 'krav/N100', filter='rundkjoring') answers with the "
"6 of 445 documents about roundabouts and reports total_matches: 6. A filter that "
"matches nothing is an answer (total_matches: 0), not a refusal."
),
)
def read_dir(
bundle_id: str,
path: str,
filter: str | None = None,
offset: int = 0,
limit: int | None = None,
) -> dict[str, Any]:
try:
bundle_dir = _resolve_bundle(index, bundle_id)
bundle = okf.navigate_bundle(bundle_dir)
okf.assert_declared_ids_agree(bundle)
return okf.directory_listing(
bundle,
path,
dimension=dimension,
filter=filter,
offset=offset,
limit=limit,
)
except _RETURNABLE_REFUSALS as exc:
return _refused_mapping(exc)
@tool(
name="read_file",
description="Read ONE document inside a knowledge base, by base id and relative path.",
)
def read_file(bundle_id: str, path: str) -> str:
try:
return _read_file(bundle_id, path)
except _RETURNABLE_REFUSALS as exc:
return _refused_text(exc)
def _read_file(bundle_id: str, path: str) -> str:
bundle_dir = _resolve_bundle(index, bundle_id)
# safe_resolve is the ONE in-/out-of-bundle test in this repo, and it is fail-closed. A
# model-chosen path is untrusted input by definition, so it goes through the same gate the
# navigation walk uses rather than a second, laxer check.
resolved = Path(safe_resolve(bundle_dir, path))
# The wrong RUNG, answered as such (finding (c) of the live K2 run). This is BEFORE the
# verdict gate for a mechanical reason as well as a readable one: ``declares_verdict_type``
# reads the path's frontmatter, so on a directory it would raise ``IsADirectoryError``
# first and the caller would get the OSError this branch exists to replace.
if resolved.is_dir():
raise DirectoryPathRefused(
f"{path!r} in knowledge base {bundle_id!r} is a directory, not a document; "
"use read_dir to list what it holds, then read_file on one of the names it gives"
)
# P18/A3: a path that does not exist, answered as such. MEASURED over P16's four paid runs:
# 10 of 24 ``read_file`` calls named a path the base does not hold (8 distinct -- one is a
# single-character UUID slip, ``4d7f`` for the real ``4e7f``), and each one left the model
# with MAF's opaque "Error: Function failed." while counting toward the three consecutive
# errors that end a request. The nearest EXISTING directory is named because that is the
# one thing the caller can act on: it is the argument for the rung that lists real names.
#
# Narrow BY CONSTRUCTION, and that is the half this replaces rather than weakens: only a
# path that is absent is translated. Any other ``OSError`` -- an unreadable file, a broken
# link -- still propagates untouched, because a refusal is a statement about the CALLER's
# path and a failure to read something that IS there is not one.
if not resolved.exists():
ancestor, neighbours = _neighbours(bundle_dir, path, dimension)
# P21/C2: the nearest listable ancestor AND up to five of its own subdirectories. The
# ancestor alone says which rung to go back to; the neighbours say which names that
# rung actually uses — measured, one run spent eleven calls walking ``R761/4-3``,
# ``4.3``, ``4-2``, ``4-1``, ``4-0``, ``4-5``, ``4-6`` while the real names were
# ``R761/4``, ``R761/41``, ``R761/42``. Omitted when the ancestor has no
# subdirectories: an empty list would be a sentence with nothing in it.
nearby = f" (its subdirectories include {', '.join(neighbours)})" if neighbours else ""
raise okf.BundlePathNotFound(
f"knowledge base {bundle_id!r} has no document {path!r}; nearest directory that "
f"holds documents: {ancestor!r}{nearby} — list it "
"with read_dir (it takes a filter) and read_file one of the names it gives"
)
# The verdict layer, refused HOWEVER the path was found (order 20260904T172353Z). No
# listing names it — ``context_files`` drops it at every level, so ``read_bundle`` and
# ``read_dir`` never mention one — but a GUESSED path reached it, and reaching it that way
# walks around the gated ExpeL fold that is the only sanctioned route from a past judgement
# into a hypothesis. Measured on the fixture base before the gate: all 2 883 characters.
#
# The rule lives HERE, in the tool, and therefore in exactly one place for both callers —
# the exploration (``--explore``) and, since S2c, the debate. Not in the rendering, which
# S2c measured to be "a filter in name only" while this rung still serves the bytes; not in
# prompt text, which is advice to an untrusted chooser rather than a gate. It is BEFORE the
# dimension check because the layer is refused unconditionally: ``dimension=None`` is the
# exploration's own call and admits every dimension, so a gate placed after that branch
# would be absent from precisely the caller it was written for.
if okf.declares_verdict_type(resolved):
raise VerdictLayerRefused(
f"document {path!r} in knowledge base {bundle_id!r} is a prior expert verdict; "
"judgements reach a hypothesis only through the gated ExpeL fold, never by being "
"read as bundle knowledge"
)
if dimension is not None:
# The SECOND half of the scope (§4.1a). A listing that hides a document while this rung
# still serves it by path is a filter in name only, and the caller here is a model that
# can name a path no listing gave it. Only a NAVIGATED concept file is judged: the walk
# is what knows a file's declared dimension, and a path outside it is already refused —
# or, for ``index.md``, is navigation rather than scoped knowledge.
bundle = okf.navigate_bundle(bundle_dir)
foreign = next(
(
f
for f in bundle.context_files
if Path(safe_resolve(bundle_dir, f.name)) == resolved
and not okf.in_dimension(f, dimension)
),
None,
)
if foreign is not None:
raise DimensionScopeRefused(
f"document {path!r} in knowledge base {bundle_id!r} declares dimension "
f"{foreign.frontmatter.get('dimension')!r}; this run is scoped to "
f"{dimension!r} and reads only knowledge in scope"
)
return resolved.read_text(encoding="utf-8")
@tool(
name="declare_requirement",
description=(
"Declare the ONE requirement in a knowledge base that BINDS the direction you are "
"about to commit to, by base id, the bundle-relative path read_file gave you, and the "
"requirement's own number as its frontmatter states it. You must have READ the "
"document with read_file first: a declaration naming a path this run never opened is "
"refused, and reading it is the correction. Use read_dir with a 'filter' word to find "
"it (a word from the approach's own label works: filter='rundkjoring' -> "
"'Krav 4.1.2-1'), read_file to read it, then declare it. The reply gives back the "
"document's own "
"title and number, so you can see whether you declared the requirement you meant: a "
"declaration of a requirement that is not about the measure is worth nothing."
),
)
def declare_requirement(bundle_id: str, path: str, ref: str) -> dict[str, Any]:
try:
return _declare_requirement(bundle_id, path, ref)
except _RETURNABLE_REFUSALS as exc:
return _refused_mapping(exc)
def _declare_requirement(bundle_id: str, path: str, ref: str) -> dict[str, Any]:
assert opened is not None and requirements is not None # the constructor guard above
# The base is resolved by the SAME index every read rung uses, so an unknown base is
# refused here exactly as it is there rather than being accepted into the record.
resolved_dir = _resolve_bundle(index, bundle_id)
read_paths = [call.path for call in opened if call.name == "read_file" and call.path]
if path not in read_paths:
raise RequirementNotRead(
f"{path!r}; this run has opened {len(read_paths)} document(s) with read_file, and "
"this is not one of them. Read it first — a requirement nobody read cannot bind a "
"direction"
)
# P21/C1: the declaration must have LOOKED. MEASURED over round 4's six debate traces --
# 13 declarations, and the distinct documents opened before each were
# 1,1,1,1,1,1,1,2,5,5,6,13,13. Seven of them opened exactly ONE document, the base's front
# matter, and declared its FIRST requirement (``Krav 1.2-1``, ``Krav 1.1-1``,
# ``Krav 1.1.1-1``); none of the 13 named a fasit concept. Three of five paid runs read a
# single document all run.
#
# THE ORDER OFFERED A SECOND RULE AND THE MEASUREMENT CHOSE BETWEEN THEM. The alternative
# -- "the declared document must have been returned by a ``read_dir`` filtered on a word
# from the approach's label" -- was replayed against the real listings and refuses 13 of
# 13, including Soraasen's ``12.11``, which the order names as the closest any run came.
# A gate that refuses every measured case, right and wrong alike, cannot discriminate: it
# is the vacuous gate's mirror image. This one refuses 8 of 13 and keeps the five that
# navigated, ``12.11`` among them.
#
# THE THRESHOLD IS NOT ON A CLIFF: k = 3, 4 and 5 all refuse the same eight, because the
# measured distribution has a gap between 2 and 5. Three is the lowest of that plateau,
# which is the least this can refuse while still separating the two measured classes.
#
# CAPPED BY THE BASE ITSELF. A base with two concept documents can be read whole in two,
# and a floor above its size would make declaration impossible there -- a gate that can
# only refuse. The cap is read off ``context_files``, the same property every listing rung
# is built from, so the verdict layer is outside the denominator exactly as it is outside
# the listings.
distinct = len(set(read_paths))
in_base = len(okf.navigate_bundle(resolved_dir).context_files)
floor = min(_MIN_DOCUMENTS_READ, in_base)
if distinct < floor:
raise RequirementNotRead(
f"{path!r}; this run has opened {distinct} distinct document(s) of the "
f"{in_base} in {bundle_id!r}, and a binding requirement declared after {distinct} "
f"is a guess rather than a finding — read at least {floor} before declaring one. "
"Use read_dir with a 'filter' word from the approach's own label to find the "
"candidates, then read_file the ones that could bind it"
)
requirements.append(DeclaredRequirement(bundle_id=bundle_id, path=path, ref=ref))
# P20/A1: give back the DOCUMENT's own title and number, read off the base rather than
# echoed from the arguments. MEASURED (P19 round 3, P17b): 13 declarations over 5 runs and
# NOT ONE named a fasit concept — the tool answered ``{"declared": true, ...}`` to every
# declaration, so a model that had declared the wrong requirement was told it had succeeded.
# ``okf.reference_number`` is the ONE reader of "which requirement is this" (kø-(p)), and
# ``binds`` says out loud what the declaration is for: without it the reply is data with no
# instruction, and the instruction is the whole correction.
declared = _declared_document(index, bundle_id, path)
return {
"declared": True,
"bundle_id": bundle_id,
"path": path,
"ref": ref,
"title": declared[0],
"req_number": declared[1],
"binds": (
f"This declaration says {ref} is the requirement the proposal rests on; a "
"declaration of a requirement that is not about the measure is worth nothing."
),
}
tools = [list_bundles, read_bundle, read_dir, read_file]
if requirements is not None:
tools.append(declare_requirement)
return tools
def requirement_payload(declared: Sequence[DeclaredRequirement]) -> list[dict[str, Any]]:
"""The ONE rendering of declared requirements into plain data, in DECLARATION order.
Two artefacts carry it — ``{run_id}-exploration.json`` and ``{run_id}-debate.json`` — and two
copies of "what a declaration looks like" would drift into two answers about one run, which is
the kø-(p) defect landing in exactly the files an operator reads after a paid run.
"""
return [{"bundle_id": d.bundle_id, "path": d.path, "ref": d.ref} for d in declared]
def _refused(
reason: str,
*,
sink: list[QuickValidation] | None,
bundle_id: str,
proposal_json: str,
) -> dict[str, Any]:
"""The refused verdict, recorded on the same rule every other branch is (see below).
``anchored`` is ``False`` and that is a statement of fact, not a default: no base was
resolved, so no ``cost-baseline.json`` was read and this verdict was reached without the
project's own cost lines — exactly what the field says everywhere else it appears.
"""
verdict: dict[str, Any] = {"decision": "refused", "reason": reason, "anchored": False}
if sink is not None:
sink.append(
QuickValidation(bundle_id=bundle_id, proposal_json=proposal_json, verdict=verdict)
)
return verdict
def quick_validate_tool(
bundle_dirs: Sequence[str], *, sink: list[QuickValidation] | None = None
) -> FunctionTool:
"""The hypothesiser's in-loop deterministic check — level 1, and advisory by construction.
It is the SAME ``validate_proposal`` against the SAME baseline the pipeline will use, so the
numbers it reports are real; what it never becomes is provenance. ``ProvenanceStamp`` is
written by ``run_project`` alone, and nothing here reaches it.
Measured cheap (S5: 13.6 ms median through stage 0 + the CBC solve + a 512-sample Monte Carlo,
on an anchored base with assumption bands), so it may be called freely in the loop and the
contract carries no latency budget.
``anchored`` is reported alongside the verdict for the same reason
``ProvenanceStamp.cost_baseline_anchored`` is a required field: a verdict reached without the
project's own cost lines is a weaker claim, and one that does not say so is the silence S4.0's
visibility work closed.
``sink`` is the caller's ``ExplorationTrace.quick_validations``. These verdicts are the one
thing ``ExplorationResult`` deliberately does not carry — they are level-1 advisory, and their
home is the ``{run_id}-exploration.json`` artefact — so without a sink they would leave no
trace of having been asked for at all.
"""
index = _bundle_index(bundle_dirs)
@tool(
name="quick_validate",
description=(
"Run the deterministic validator over a candidate, as an ADVISORY check. Pass the "
"knowledge base id and the candidate as IR JSON. The verdict is not the project's "
"decision — the pipeline re-validates and stamps."
),
)
def quick_validate(bundle_id: str, proposal_json: str) -> dict[str, Any]:
verdict: dict[str, Any]
try:
bundle_dir = _resolve_bundle(index, bundle_id)
except ExplorationError as exc:
# A base id the model guessed wrong is a thing it can CORRECT — so it comes back as a
# verdict naming the configured ids, never as a raise. MEASURED (funn 99, Q5=B on K2):
# three consecutive calls carried ``bundle_id="renholdstekniske_funksjonskrav"``, a
# concept name guessed out of the seeded cut, and MAF turned each raise into the opaque
# ``"Error: Function failed."`` (``_tools.py:1426`` — the detail is suppressed unless
# ``include_detailed_errors``), so the ONE thing this refusal knows and the model did
# not — which ids exist — never reached it. The replies show the consequence: it went
# on guessing at the JSON format. Three in a row is
# ``DEFAULT_MAX_CONSECUTIVE_ERRORS_PER_REQUEST`` (``_tools.py:96``), after which MAF
# stops all function calling for the request. This is NOT a general softening of the
# navigator's refusals: ``read_file``/``read_dir``/``read_bundle`` still raise, and
# their raises are counted the same way (measured, reported, out of this order's scope).
return _refused(str(exc), sink=sink, bundle_id=bundle_id, proposal_json=proposal_json)
baseline = okf.load_optional_cost_baseline(bundle_dir)
try:
proposal = SavingsProposal.model_validate_json(proposal_json)
except ValidationError as exc:
verdict = {
"decision": "unparseable",
"reason": str(exc),
"anchored": baseline is not None,
}
else:
outcome = validate_proposal(proposal, baseline=baseline)
if isinstance(outcome, Rejection):
verdict = {
"decision": "rejected",
"reason": outcome.reason,
"anchored": baseline is not None,
}
else:
verdict = {
"decision": "validated",
"reason": "",
"anchored": baseline is not None,
"p10": outcome.p10,
"p50": outcome.p50,
"p90": outcome.p90,
}
# Recorded AFTER the verdict is decided and on EVERY branch — an unparseable candidate is
# as much a thing the hypothesiser asked about as a validated one. A refused bundle id is
# recorded too, on that same rule: the comment that used to stand here ("nothing was
# validated") was written for a RAISE, which left no verdict at all. Now that there IS one,
# keeping it out would make a refused call the single quick_validate outcome invisible in
# ``quick_validations`` — and an operator reading that list could not tell "never called"
# from "called three times with an id that does not exist", which is exactly the read this
# defect needed two artefacts to reconstruct.
if sink is not None:
sink.append(
QuickValidation(bundle_id=bundle_id, proposal_json=proposal_json, verdict=verdict)
)
return verdict
return quick_validate
# ---------------------------------------------------------------------------------------------
# The workflow. One builder, one build, ONE run per exploration (B7 / § C.4).
# ---------------------------------------------------------------------------------------------
def fresh_exploration_workflow(
client_factory: Callable[[str], BaseChatClient],
*,
contract: ExplorationContract,
bundle_dirs: Sequence[str] = (),
middleware: Sequence[Any] | None = None,
quick_validate_sink: list[QuickValidation] | None = None,
tool_call_sink: list[ToolCall] | None = None,
requirement_sink: list[DeclaredRequirement] | None = None,
checkpoint_dir: str | None = None,
) -> Any:
"""Build a FRESH Magentic workflow with fresh agents and fresh clients (mirrors
``workflow.fresh_workflow``).
**Fresh builder per exploration, never a reused one.** A built Magentic workflow is
single-use: a second ``run()`` on it raises ``RuntimeError`` having made zero model calls
(measured, E1) — which is a safe failure, unlike GroupChat 1.9.0's silently empty second run.
Reuse is prevented here rather than relied upon to fail.
``manager_agent_factory=`` rather than ``manager_agent=``: the eager form constructs the
manager once and hands the same instance to every ``build()`` (``:1683``, ``:1729-1730``),
which on orchestrations 1.0.0 leaked one exploration's task into the next. Orchestrations
1.0.1 removed the manager's persistent session, so that leak is GONE and **no test here can
tell the two forms apart today** (measured, § F A8: 4/4 → 0/5). The factory form is used
anyway because it costs nothing and does not depend on an upstream regression fix staying
fixed — but it is stated plainly rather than gated, because a gate that cannot go red proves
nothing.
Every agent carries the SAME ``middleware`` list, manager included. That is the whole of the
token guarantee: agent-level ``ChatMiddleware`` does fire on the manager's own calls (measured
A1), and the manager is the most talkative participant in the loop.
"""
# P19 DEL A: the declaration tool reaches BOTH roles, and that is a measurement rather than
# generosity. The instruction that asks for a binding requirement is the HYPOTHESISER's — it is
# the role that commits to a direction — while the ``read_file`` calls the refusal checks are
# the NAVIGATOR's. Giving it to the navigator alone would leave the committing role unable to
# state its own commitment; to the hypothesiser alone, unable to declare what its partner read.
navigator = list(
navigator_tools(bundle_dirs, opened=tool_call_sink, requirements=requirement_sink)
)
hypothesiser_tools: list[Any] = [quick_validate_tool(bundle_dirs, sink=quick_validate_sink)]
hypothesiser_tools += [t for t in navigator if getattr(t, "name", "") == "declare_requirement"]
tools_by_role: dict[str, list[Any]] = {
NAVIGATOR_ROLE: navigator,
HYPOTHESISER_ROLE: hypothesiser_tools,
}
participants = [
Agent(
client_factory(role),
_INSTRUCTIONS[role],
name=role,
description=_INSTRUCTIONS[role],
tools=tools_by_role[role],
middleware=middleware,
)
for role in PARTICIPANT_ROLES
]
def _manager_agent() -> Agent:
return Agent(
client_factory(MANAGER_ROLE),
"You plan and coordinate an exploration of a project's knowledge bases to find "
"cost-saving directions worth testing. You never state a saving figure yourself.",
name=MANAGER_ROLE,
description="plans the exploration",
middleware=middleware,
)
builder = MagenticBuilder(
participants=participants,
manager_agent_factory=_manager_agent,
max_round_count=contract.max_rounds,
max_stall_count=contract.max_stall_count,
max_reset_count=contract.max_reset_count,
enable_plan_review=contract.enable_plan_review,
)
if checkpoint_dir is not None:
# Measured (spike S4, and re-measured in the plan's own E-table correction): it is the
# BUILDER's ``.with_checkpointing`` that is load-bearing, not ``checkpoint_storage=`` on
# ``run()`` — dropping the latter leaves the whole suite green. Both processes call THIS
# function, so the graph they build is identical, which is what lets a checkpoint written
# by one be restored by the other (``_runner.py:275-279`` matches on the graph signature).
builder = builder.with_checkpointing(checkpoint_storage(checkpoint_dir))
return builder.build()
def _truthy(answer: Any) -> bool:
"""Read a ledger answer exactly as the orchestrator does.
``MagenticProgressLedgerItem.answer`` is ``str | bool`` and is NOT normalised per field
(``:292``); the orchestrator then tests it with plain Python truthiness (``:1106``, ``:1112``).
A smarter coercion here — treating the string ``"false"`` as false, say — would describe a run
the orchestrator did not have. This layer reports what the loop DID, so it copies the loop's
own rule rather than improving on it.
"""
return bool(answer)
def _plan_text(content: Any) -> str:
"""The task-ledger plan as text. ``PLAN_CREATED``/``REPLANNED`` carry a ``Message``."""
return str(getattr(content, "text", "") or "")
def _progress_text(content: Any) -> str:
"""The progress ledger as text — and ABSENCE as the empty string, never ``"None"``.
``MagenticProgressLedger | None`` went through a bare ``str()`` at both construction sites, so
in every run measured so far (the value is ``None`` until the manager has emitted a ledger) the
expert was shown, and the parked question file stored, the four characters ``None``. That is
worse than saying nothing: it looks like content. The data layer states absence by being
absent — ``Bundle.skipped``'s empty-tuple rule — and the TERMINAL is where it is put in words,
because silence at a gate somebody signs is what ``PlanReviewInputError`` already refuses.
"""
return "" if content is None else str(content)
def _absorb(
result: Any,
*,
ledger_log: list[LedgerEntry],
hypotheses: list[str],
seen: set[int],
span: Any,
) -> int:
"""Fold one ``run()``'s events into the running log and onto the trace. Returns this call's
replans.
``seen`` de-duplicates by object identity because MAF may surface the same payload under more
than one event type; the alternative — pinning one event-type name — would silently under- or
over-count if that changed. Participant output is matched on ``executor_id``, which is the
agent's name, exactly as ``run._authored_texts`` matches the debate's ``author_name``.
**The three span events land here** (U14's deferred half, now that it has a call site). They
carry the manager's DECISIONS — which plan, who was asked, whether the request was satisfied —
as separate typed attributes rather than a rendered sentence, so a collector can query them.
Stated honesty limit: they are recorded as the event stream is folded, i.e. after ``run()``
returns, so their ORDER is faithful and their timestamps are not the moments the manager acted.
Emitting live would mean driving the loop through ``run_stream``, which is a different
measurement from the one the plan-review round trip was proved against.
"""
replans = 0
for event in result:
data = getattr(event, "data", None)
if data is None or id(data) in seen:
continue
event_type = getattr(data, "event_type", None)
if event_type is not None:
seen.add(id(data))
if event_type == MagenticOrchestratorEventType.PLAN_CREATED:
span.add_event("plan_created", {"plan": _plan_text(data.content)})
elif event_type == MagenticOrchestratorEventType.REPLANNED:
replans += 1
span.add_event("replanned", {"plan": _plan_text(data.content)})
elif event_type == MagenticOrchestratorEventType.PROGRESS_LEDGER_UPDATED:
ledger = data.content
speaker = str(ledger.next_speaker.answer)
entry = LedgerEntry(
round_index=len(ledger_log) + 1,
is_request_satisfied=_truthy(ledger.is_request_satisfied.answer),
is_in_loop=_truthy(ledger.is_in_loop.answer),
is_progress_being_made=_truthy(ledger.is_progress_being_made.answer),
next_speaker=speaker,
instruction_or_question=str(ledger.instruction_or_question.answer),
speaker_known=speaker in PARTICIPANT_ROLES,
)
ledger_log.append(entry)
span.add_event(
"progress_ledger_updated",
{
"round_index": entry.round_index,
"is_request_satisfied": entry.is_request_satisfied,
"is_in_loop": entry.is_in_loop,
"is_progress_being_made": entry.is_progress_being_made,
"next_speaker": entry.next_speaker,
"speaker_known": entry.speaker_known,
"instruction_or_question": entry.instruction_or_question,
},
)
continue
if getattr(data, "executor_id", None) == HYPOTHESISER_ROLE:
response = getattr(data, "agent_response", None)
text = getattr(response, "text", None)
if text:
seen.add(id(data))
hypotheses.append(text)
return replans
class HypothesisParseError(ExplorationError):
"""A hypothesiser turn carried the marker and then something unreadable.
Fail-closed, and the marker is what makes that affordable: most turns legitimately are not
hypotheses, so an unmarked turn is not a failure and there is nothing to be silent about. A
MARKED line that will not parse is a claim the loop tried to make and could not — raising it
is the ``write_concept_file`` rule (validation, never repair) rather than the tolerant RAW
inbox rule, because this is the product of the run, not a folder anyone may drop things in.
"""
def _resolve_hypothesis_bundle(raw: str, bundle_ids: Sequence[str]) -> str:
"""Which knowledge base a marked hypothesis belongs to (§ C.7), fail-closed.
Three rules, and the middle one is the whole reason the field can default at all:
* a stated id must be one that was CONFIGURED — refused by name, never resolved by position,
exactly as ``_resolve_bundle`` refuses an unknown base for the read tools. A hypothesis that
names a base must not be treated more leniently than a read of one.
* a silent marker with at most one base configured resolves to that base (or to ``""`` with
none). With one configured base there is no other value the field could take, so this is the
only answer rather than a guess.
* a silent marker with SEVERAL configured refuses. A marked line is a claim the loop committed
to; one that cannot be routed is a claim it could not finish making, and the marker is
precisely what makes that affordable to refuse — an unmarked turn is not a hypothesis, so
there is no silence being turned into an error.
"""
if raw:
if raw not in bundle_ids:
known = ", ".join(bundle_ids) or "(none configured)"
raise ExplorationError(
f"hypothesis names knowledge base {raw!r}, which is not configured for this "
f"exploration; configured: {known}"
)
return raw
if len(bundle_ids) <= 1:
return bundle_ids[0] if bundle_ids else ""
raise HypothesisParseError(
f"a marked hypothesis must name its knowledge base when several are configured "
f'({", ".join(bundle_ids)}): add "bundle_id" to the marker payload. Routing it here '
"would evaluate a direction against a project nobody asked about"
)
def _requirement_of(data: Mapping[str, Any], line: str) -> BindingRequirement | None:
"""The marked hypothesis's binding requirement, or ``None`` when the base genuinely has none.
**The field is never OMITTED** (P19 A1): a missing key is a hard error of the same class as an
unreadable marked line, because the marker is what makes fail-closed affordable — the model
committed to a direction, and "which requirement binds it" is part of that commitment rather
than an optional extra. An EXPLICIT ``null`` is legal and needs ``why_none``: a base that holds
no requirement for a direction is a finding worth stating, and one stated without a reason is
indistinguishable from the model having skipped the question.
"""
if "requirement" not in data:
raise HypothesisParseError(
"a marked hypothesis must carry a 'requirement' — either "
'{"path": ..., "ref": ...} for the document that binds it, or null together with '
f"'why_none'. A direction with no requirement behind it is a guess; got: {line}"
)
raw = data["requirement"]
if raw is None:
if not data.get("why_none"):
raise HypothesisParseError(
"a marked hypothesis with 'requirement': null must say 'why_none' — the base "
f"holding no requirement is a finding, and an unexplained null is a silence: {line}"
)
return None
if not isinstance(raw, dict) or not raw.get("path") or not raw.get("ref"):
raise HypothesisParseError(
"a marked hypothesis's 'requirement' needs a non-empty 'path' and 'ref'; a half-named "
f"requirement reads as a citation and points at nothing: {line}"
)
return BindingRequirement(path=str(raw["path"]), ref=str(raw["ref"]))
def _parse_hypotheses(
texts: Sequence[str], bundle_ids: Sequence[str]
) -> list[tuple[str, str, str, BindingRequirement | None]]:
"""Every marked ``(label, rationale, bundle_id, requirement)`` the hypothesiser committed to,
in turn order. The base is resolved HERE rather than at minting time so an unroutable claim is
refused while the line that made it is still in hand for the error message."""
found: list[tuple[str, str, str, BindingRequirement | None]] = []
for text in texts:
for line in text.splitlines():
stripped = line.strip()
if not stripped.startswith(HYPOTHESIS_MARKER):
continue
payload = stripped[len(HYPOTHESIS_MARKER) :].strip()
try:
data = json.loads(payload)
except json.JSONDecodeError as exc:
raise HypothesisParseError(
f"hypothesiser marked a line as a hypothesis and it is not JSON: {exc}; "
f"line was: {stripped}"
) from exc
if not isinstance(data, dict) or not data.get("label") or not data.get("rationale"):
raise HypothesisParseError(
"a marked hypothesis needs a non-empty 'label' and 'rationale'; got: "
f"{stripped}"
)
found.append(
(
str(data["label"]),
str(data["rationale"]),
_resolve_hypothesis_bundle(str(data.get("bundle_id") or ""), bundle_ids),
_requirement_of(data, stripped),
)
)
return found
def refuse_unroutable_seeds(seeds: Sequence[Approach], bundle_ids: Sequence[str]) -> None:
"""Refuse an expert's seed that could not be dispatched — BEFORE the first model call.
Validation, ALDRI reparasjon, and here the second half is the load-bearing one: a seed is the
expert's own words and § C.6 door 1 is a PRESERVATION rule, so filling in a missing
``bundle_id`` on their behalf would put their name on a routing decision they did not make.
The single-base default belongs to ``route_by_bundle``, at consumption, where it is unambiguous
by construction — never to a rewrite of the input.
Called before the loop starts rather than after it returns, and that placement is the økt-57
outbox-hoist precedent: at the exception alone, a refusal after the exploration has spent its
whole budget looks identical to one before it spent anything. The mandate is undispatchable
either way; what is at stake is whether the expert pays to find that out.
"""
for seed in seeds:
if seed.bundle_id:
if seed.bundle_id not in bundle_ids:
known = ", ".join(bundle_ids) or "(none configured)"
raise ExplorationError(
f"seed approach {seed.id!r} names knowledge base {seed.bundle_id!r}, which is "
f"not configured for this exploration; configured: {known}"
)
elif len(bundle_ids) > 1:
raise ExplorationError(
f"seed approach {seed.id!r} names no knowledge base and {len(bundle_ids)} are "
f"configured ({', '.join(bundle_ids)}); set Approach.bundle_id so the mandate can "
"be dispatched to the base the hypothesis is actually about"
)
def _mint_approaches(
seeds: Sequence[Approach],
discovered: Sequence[tuple[str, str, str, BindingRequirement | None]],
) -> tuple[Approach, ...]:
"""Seeds FIRST, untouched, then one approach per discovered direction.
Seeds lead because the pipeline evaluates approaches in order under one shared budget: a
direction the domain expert asked for must be reached before the loop's own findings spend
what is left. The minted ids skip anything a seed already claims, so a seed literally named
``hypothesis-1`` cannot collide — ``Mandate`` refuses duplicate ids at construction, and
turning an expert's naming choice into a hard failure would be a refusal that teaches nothing.
"""
taken = {approach.id for approach in seeds}
minted: list[Approach] = list(seeds)
counter = 0
for label, rationale, bundle_id, requirement in discovered:
counter += 1
candidate = f"hypothesis-{counter}"
while candidate in taken or candidate == OWN_PROPOSAL_ID:
counter += 1
candidate = f"hypothesis-{counter}"
taken.add(candidate)
# description is the hypothesiser's own words, VERBATIM: ``generate._build_messages`` feeds
# an Approach.description to the proposer unchanged, and the reason a direction is worth
# trying is exactly the half a model cannot re-derive from the cost table.
#
# ``bundle_id`` is stamped on a MINTED approach because there is nothing here to preserve —
# the opposite call from the seeds above, which pass through untouched (§ C.6 door 1).
# ``requirement`` is stamped on a MINTED approach for the same reason ``bundle_id`` is:
# there is nothing here to preserve. A SEED passes through untouched (§ C.6 door 1) — an
# expert who named no requirement is not to be given one on their behalf.
minted.append(
Approach(
id=candidate,
label=label,
description=rationale,
bundle_id=bundle_id,
requirement=requirement,
)
)
return tuple(minted)
def _pending_plan_reviews(result: Any) -> list[Any]:
return [event for event in result if event.type == "request_info"]
async def explore(
prompt: str,
*,
contract: ExplorationContract,
bundle_dirs: Sequence[str] = (),
profile: Profile | str = Profile.LOCAL,
client_factory: Callable[[str], BaseChatClient] | None = None,
seed_approaches: Sequence[Approach] = (),
plan_reviewer: PlanReviewer | None = None,
meter: TokenMeter | None = None,
success_criteria: str = "",
trace: ExplorationTrace | None = None,
checkpoint_dir: str | None = None,
seed_context: str = "",
) -> ExplorationResult:
"""Explore the knowledge bases and return the ``Mandate`` the pipeline should evaluate.
Writes NOTHING. No outbox artefact, no wiki promotion, no verdict — level 3 of the guarantee
table belongs to ``run_project`` alone, and an exploration that could write would be a route
around the gate that makes an answer checkable.
**Three stops, three channels, deliberately not one.** Tokens raise ``BudgetExceeded`` exactly
as the debate does, so the hosted surface's 429 arm needs no new case. Rounds raise it too,
with ``kind="exploration_rounds"`` — the orchestration itself raises nothing at its round cap
(measured, § F): it returns a canonical assistant message that is indistinguishable from
success at the transport, so this layer produces the typed stop. Everything semantic —
stalling out, an exhausted revision cap, a ledger naming nobody — is a VALUE in ``stop``,
because those are outcomes of the exploration rather than exhaustion of a resource, and S3.4
split those two apart for a reason.
``seed_approaches`` are the domain expert's own hypotheses (door 1 of § C.6). They are in the
returned mandate whatever the loop found, including when the loop found nothing and including
when it stopped early.
**Multi-base (§ C.7).** Every approach in the returned mandate carries ``bundle_id``, and
``mandate.route_by_bundle`` partitions the commission by it so ``run.run_mandate_across_bundles``
can run the pipeline once per base. ``run_project`` itself still takes ONE ``bundle_dir``, and
that is the measured shape rather than a step not yet taken: it derives the project, the
validator's cost baseline, the read context and the ExpeL query key from THE bundle and returns
one stamped ``RunResult``, so a second directory on that signature would force a silent
pick-one for all four.
Honesty limits, stated rather than implied. (1) The ``quick_validate`` verdicts the
hypothesiser saw are not in
``ExplorationResult``: they are level-1 advisory, and their home is the
``{run_id}-exploration.json`` artefact the CLI wiring writes — pass an ``ExplorationTrace`` to
collect them. (2) The exploration roles resolve through ``resolve_model``'s ``default``
fallback unless an operator maps them explicitly.
``trace`` is the caller's accumulator and is the ONLY way to see what a run that RAISED
produced: both budget channels destroy the ``ExplorationResult`` before it exists. When it is
omitted a private one is used, so the returned result is unchanged for every existing caller.
``seed_context`` is material the CALLER has already verified and wants the loop to START from
— today, one contract-conformant OKF pre-pass cut (``--prepass-seed``). It joins the TASK
MESSAGE and deliberately NOT ``objective``: the objective is what a person commissioned and
what ``Mandate.announce`` prints back to them, so folding ten thousand tokens of excerpts into
it would make the commission unreadable and would put the cut's text into every artefact that
quotes the objective. Empty by default, and an empty string leaves the task message
byte-identical to what it has always been — which is what makes "without the flag, nothing
changed" a property rather than a promise.
**This is a starting point, never a boundary.** The navigator tools are built exactly as they
are without it, because an exploration that could not read past its seed would be the OTHER
arm (``--prepass-payload``, where the cut REPLACES the base and the tools are withdrawn), and
building both behaviours behind one name is how a flag stops meaning anything.
"""
if plan_reviewer is not None and checkpoint_dir is not None:
raise ExplorationError(
"a plan_reviewer and a checkpoint_dir are two doors onto one review: the first answers "
"it in this process, the second parks it for another one. Refused rather than ranked, "
"because silently preferring either would block a caller that asked for the other"
)
if contract.enable_plan_review and plan_reviewer is None and checkpoint_dir is None:
raise ExplorationError(
"enable_plan_review is set but no plan_reviewer was given and no checkpoint_dir was "
"offered to park it: the exploration would stop at a review nobody can answer, which "
"is a hang rather than a result"
)
if checkpoint_dir is not None and not contract.enable_plan_review:
raise ExplorationError(
"a checkpoint_dir was given but enable_plan_review is false, so nothing would ever "
"park and the storage would be written and never read (a cap on an event that cannot "
"happen, refused for the reason max_plan_revisions is)"
)
if plan_reviewer is not None and not contract.enable_plan_review:
raise ExplorationError(
"a plan_reviewer was given but enable_plan_review is false, so no review is ever "
"requested and the reviewer would never be called (refused, never silently ignored)"
)
# The base ids, resolved ONCE and before anything is built: they are what a seed and a marked
# hypothesis are both routed against, and ``_bundle_index`` is the one place that decides them
# (a duplicate basename refuses here rather than letting the manager read A believing it read B).
bundle_ids = tuple(_bundle_index(bundle_dirs))
refuse_unroutable_seeds(seed_approaches, bundle_ids)
if meter is None:
meter = TokenMeter(Budget(max_tokens=contract.max_tokens, max_rounds=contract.max_rounds))
if client_factory is None:
# Imported HERE, not at module scope: ``run`` imports this module for its ``--explore``
# door, so a top-level import would be circular. One copy of the backend factory, never a
# second (the (p) rule) — the same move ``run.main`` makes for ``scripted_factory``.
from portfolio_optimiser.run import _default_factory
client_factory = _default_factory(profile)
if trace is None:
trace = ExplorationTrace()
workflow = fresh_exploration_workflow(
client_factory,
contract=contract,
bundle_dirs=bundle_dirs,
middleware=[BudgetMiddleware(meter), ExplorationToolRecorder(trace.tool_calls)],
quick_validate_sink=trace.quick_validations,
# ALIASES of the trace's own lists, never copies (the ``_drive`` rule): the refusal reads
# the same read trace the recorder writes, so the two can never disagree about what this
# run opened.
tool_call_sink=trace.tool_calls,
requirement_sink=trace.requirements,
checkpoint_dir=checkpoint_dir,
)
hypothesis_texts: list[str] = []
# ONE span for the whole exploration, opened before the first model call and closed however
# the loop ends — including on a BudgetExceeded, which the span records rather than swallows.
# With no provider installed this is a no-op tracer and every event is discarded, which is
# exactly what "tracing is off" has meant since U14.
with exploration_tracer().start_as_current_span(EXPLORATION_SPAN) as span:
# The seed rides in the TASK MESSAGE, never in ``prompt`` itself: ``_finish`` below builds
# the mandate's ``objective`` from ``prompt``, and a commission whose objective carried the
# whole cut would be unreadable to the person who wrote it.
result = await workflow.run(f"{prompt}\n\n{seed_context}" if seed_context else prompt)
stop, _ = await _drive(
workflow,
result,
contract=contract,
trace=trace,
hypotheses=hypothesis_texts,
seen=set(),
replans=0,
span=span,
plan_reviewer=plan_reviewer,
checkpoint_dir=checkpoint_dir,
prompt=prompt,
bundle_dirs=bundle_dirs,
meter=meter,
)
return _finish(
prompt=prompt,
stop=stop,
trace=trace,
hypotheses=hypothesis_texts,
bundle_ids=bundle_ids,
seed_approaches=seed_approaches,
success_criteria=success_criteria,
)
async def _drive(
workflow: Any,
result: Any,
*,
contract: ExplorationContract,
trace: ExplorationTrace,
hypotheses: list[str],
seen: set[int],
replans: int,
span: Any,
plan_reviewer: PlanReviewer | None,
checkpoint_dir: str | None,
prompt: str,
bundle_dirs: Sequence[str],
meter: TokenMeter,
) -> tuple[ExplorationStop | None, int]:
"""Drive a built workflow from one ``run()`` result to an ending, answering plan reviews.
**ONE copy of this loop, shared by ``explore`` and ``resume_exploration``.** The asynchronous
door is not a second loop that happens to look like the first: a resumed exploration answers
reviews, absorbs ledgers, mints nothing, and classifies its stop by exactly the same rules, and
two copies of that would drift the moment one of them was corrected (kø-(p)).
Raises ``PlanReviewParked`` when the asynchronous door is armed — the loop stops mid-plan and
the caller decides where to write the question.
"""
# ONE accumulator per fact, held by the caller (see ``ExplorationTrace``). The local names are
# aliases, never copies — a second list here is the kø-(p) drift this shape exists to prevent.
ledger_log = trace.ledger
plan_reviews = trace.plan_reviews
stop: ExplorationStop | None = None
while True:
trace.tokens_spent = meter.tokens
replans += _absorb(
result,
ledger_log=ledger_log,
hypotheses=hypotheses,
seen=seen,
span=span,
)
pending = _pending_plan_reviews(result)
if not pending:
break
request = pending[0]
review = request.data
if plan_reviewer is None:
# The asynchronous door (U12). ``checkpoint_dir`` is what armed it, and the guard in
# ``explore`` refused every other way of arriving here with no reviewer.
assert checkpoint_dir is not None
raise PlanReviewParked(
await _park(
workflow,
request,
checkpoint_dir=checkpoint_dir,
prompt=prompt,
bundle_dirs=bundle_dirs,
contract=contract,
trace=trace,
hypotheses=hypotheses,
meter=meter,
replans=replans,
)
)
decision = plan_reviewer(
PlanReviewRequest(
index=len(plan_reviews),
plan=_plan_text(review.plan),
current_progress=_progress_text(review.current_progress),
is_stalled=_truthy(review.is_stalled),
)
)
if decision.feedback is None:
plan_reviews.append(
PlanReview(
index=len(plan_reviews),
plan=_plan_text(review.plan),
is_stalled=_truthy(review.is_stalled),
decision="approve",
)
)
response = MagenticPlanReviewResponse.approve()
else:
# The revision is recorded whether or not it is APPLIED: ``plan_reviews`` is the
# record of what the reviewer DECIDED, and ``stop`` is what says the last one was
# refused.
applied = sum(1 for entry in plan_reviews if entry.decision == "revise")
plan_reviews.append(
PlanReview(
index=len(plan_reviews),
plan=_plan_text(review.plan),
is_stalled=_truthy(review.is_stalled),
decision="revise",
feedback=decision.feedback,
)
)
if applied >= contract.max_plan_revisions:
# Measured (§ F, A3): a revise costs two manager calls, emits no progress
# ledger and consumes no round, then asks AGAIN. Under the round cap alone this
# loop never terminates. Stopping is the honest move — forcing an approve the
# reviewer did not give would be repair, and repair of a human's decision most
# of all.
stop = "plan_revisions_exhausted"
break
response = MagenticPlanReviewResponse.revise(decision.feedback)
result = await workflow.run(responses={request.request_id: response})
trace.tokens_spent = meter.tokens
if stop is None:
stop = _classify_stop(ledger_log, replans=replans, contract=contract)
return stop, replans
async def _park(
workflow: Any,
request: Any,
*,
checkpoint_dir: str,
prompt: str,
bundle_dirs: Sequence[str],
contract: ExplorationContract,
trace: ExplorationTrace,
hypotheses: Sequence[str],
meter: TokenMeter,
replans: int,
) -> ParkedExploration:
"""Freeze the suspended exploration and name the checkpoint a later process resumes from.
``get_latest`` rather than the last entry of a listing: it picks by timestamp
(``_checkpoint.py:424``), while the listing's order is whatever ``Path.glob`` returned.
The checkpoint named here must be the one that carries THIS question, and that is asserted
against the declared ``pending_request_info_events`` field rather than inferred from the
listing being non-empty. **The weaker form went inert under core 1.16.0 (F15, measured):** a
park now writes TWO checkpoints, and only one of them carries a plan-review type. A
mis-declared ``_ALLOWED_CHECKPOINT_TYPES`` therefore no longer empties the listing — it drops
exactly the checkpoint that matters, ``get_latest`` returns the OTHER one, and the old
``latest is None`` guard passed while the question file named a checkpoint that can never bear
the answer. That is the silent dead question this layer exists to refuse, so the guard now
checks the property it always meant rather than the symptom that used to imply it.
Refusing is where this layer is louder than the framework: a blocked deserialisation is
swallowed into a warning upstream, so the alternative to raising here is a question whose
answer can never be applied — an exploration that fails as an absence, days later, to somebody
who has already written their answer.
"""
latest = await checkpoint_storage(checkpoint_dir).get_latest(workflow_name=workflow.name)
if latest is None or str(request.request_id) not in latest.pending_request_info_events:
raise CheckpointUnreadable(
f"the exploration reached a plan review but no checkpoint carrying it could be read "
f"back from {checkpoint_dir!r}: without one the review can never be resumed, so it is "
f"refused here rather than written as a question nobody can answer"
)
review = request.data
return ParkedExploration(
prompt=prompt,
request_id=str(request.request_id),
checkpoint_id=str(latest.checkpoint_id),
index=len(trace.plan_reviews),
plan=_plan_text(review.plan),
current_progress=_progress_text(review.current_progress),
is_stalled=_truthy(review.is_stalled),
bundle_dirs=tuple(bundle_dirs),
contract=contract,
ledger=tuple(trace.ledger),
plan_reviews=tuple(trace.plan_reviews),
hypotheses=tuple(hypotheses),
tokens_spent=meter.tokens,
replans=replans,
)
def _finish(
*,
prompt: str,
stop: ExplorationStop | None,
trace: ExplorationTrace,
hypotheses: Sequence[str],
bundle_ids: Sequence[str],
seed_approaches: Sequence[Approach],
success_criteria: str,
) -> ExplorationResult:
"""Mint the mandate from a finished drive. Shared, for the reason ``_drive`` is."""
discovered = _parse_hypotheses(hypotheses, bundle_ids) if stop != "unknown_speaker" else []
return ExplorationResult(
mandate=Mandate(
objective=prompt,
approaches=_mint_approaches(seed_approaches, discovered),
allow_own_proposals=True,
success_criteria=success_criteria,
),
ledger_log=tuple(trace.ledger),
stop=stop,
plan_reviews=tuple(trace.plan_reviews),
)
async def resume_exploration(
parked: ParkedExploration,
decision: PlanReviewDecision,
*,
checkpoint_dir: str,
profile: Profile | str = Profile.LOCAL,
client_factory: Callable[[str], BaseChatClient] | None = None,
seed_approaches: Sequence[Approach] = (),
success_criteria: str = "",
trace: ExplorationTrace | None = None,
) -> ExplorationResult:
"""Answer a parked plan review and drive the exploration onward, in a process that never ran it.
Everything about the workflow is rebuilt from ``parked`` rather than from argv: the prompt, the
bounds and the bases are what the suspended run used, and the graph must match the checkpoint's
signature (``_runner.py:275-279``) for the restore to be accepted at all. A caller that had to
re-supply them could get one of them wrong and would find out as a restore failure days later.
**The budget spans the suspension.** The meter starts at ``parked.tokens_spent`` and the ledger
at ``parked.ledger``, so the round cap and the token cap measure the whole exploration rather
than this leg of it. Without that a park would hand back a full budget every time it happened.
Parking AGAIN is a normal outcome, not a failure: a revision makes the manager replan and ask
about the NEW plan, which is the second half of "be om svar, BRUKE svarene" on this time-scale.
It leaves by ``PlanReviewParked`` exactly as the first park did.
"""
if trace is None:
trace = ExplorationTrace()
# The carried state is put back BEFORE anything runs: ``_drive`` reads these as its own running
# log, and ``_classify_stop`` counts the ledger to decide whether the ROUND cap bound.
trace.ledger.extend(parked.ledger)
trace.plan_reviews.extend(parked.plan_reviews)
hypothesis_texts = list(parked.hypotheses)
bundle_ids = tuple(_bundle_index(parked.bundle_dirs))
refuse_unroutable_seeds(seed_approaches, bundle_ids)
meter = TokenMeter(
Budget(max_tokens=parked.contract.max_tokens, max_rounds=parked.contract.max_rounds)
)
# Through ``charge``, not by assigning ``tokens``: charging re-tests the cap, so a suspension
# that already spent everything refuses HERE instead of buying one more leg of the loop.
meter.charge(parked.tokens_spent)
if client_factory is None:
from portfolio_optimiser.run import _default_factory
client_factory = _default_factory(profile)
workflow = fresh_exploration_workflow(
client_factory,
contract=parked.contract,
bundle_dirs=parked.bundle_dirs,
middleware=[BudgetMiddleware(meter), ExplorationToolRecorder(trace.tool_calls)],
quick_validate_sink=trace.quick_validations,
# ALIASES of the trace's own lists, never copies (the ``_drive`` rule): the refusal reads
# the same read trace the recorder writes, so the two can never disagree about what this
# run opened.
tool_call_sink=trace.tool_calls,
requirement_sink=trace.requirements,
checkpoint_dir=checkpoint_dir,
)
# The revision cap counted across the SUSPENSION, over the reviews carried in ``parked``.
# Measured, and it is why the carry-over is load-bearing rather than tidy: a revise costs two
# manager calls, emits no progress ledger and consumes no round (§ F, A3), so a cap that reset
# at every park would leave the asynchronous door with no bound at all — an expert could revise
# forever, one process at a time, under guards that all report themselves satisfied. The same
# arithmetic ``_drive`` does for the synchronous door, and the same refusal to repair: the
# decision is RECORDED and the loop stops, never forced into an approve nobody gave.
applied = sum(1 for entry in parked.plan_reviews if entry.decision == "revise")
trace.plan_reviews.append(
PlanReview(
index=parked.index,
plan=parked.plan,
is_stalled=parked.is_stalled,
decision="approve" if decision.feedback is None else "revise",
feedback=decision.feedback or "",
)
)
if decision.feedback is not None and applied >= parked.contract.max_plan_revisions:
return _finish(
prompt=parked.prompt,
stop="plan_revisions_exhausted",
trace=trace,
hypotheses=hypothesis_texts,
bundle_ids=bundle_ids,
seed_approaches=seed_approaches,
success_criteria=success_criteria,
)
response = (
MagenticPlanReviewResponse.approve()
if decision.feedback is None
else MagenticPlanReviewResponse.revise(decision.feedback)
)
with exploration_tracer().start_as_current_span(EXPLORATION_SPAN) as span:
result = await workflow.run(
responses={parked.request_id: response},
checkpoint_id=parked.checkpoint_id,
checkpoint_storage=checkpoint_storage(checkpoint_dir),
)
stop, _ = await _drive(
workflow,
result,
contract=parked.contract,
trace=trace,
hypotheses=hypothesis_texts,
seen=set(),
replans=parked.replans,
span=span,
plan_reviewer=None,
checkpoint_dir=checkpoint_dir,
prompt=parked.prompt,
bundle_dirs=parked.bundle_dirs,
meter=meter,
)
return _finish(
prompt=parked.prompt,
stop=stop,
trace=trace,
hypotheses=hypothesis_texts,
bundle_ids=bundle_ids,
seed_approaches=seed_approaches,
success_criteria=success_criteria,
)
def _classify_stop(
ledger_log: Sequence[LedgerEntry], *, replans: int, contract: ExplorationContract
) -> ExplorationStop | None:
"""Decide what a finished ``run()`` actually was. Structural, never prose-matched.
The orchestration ends four ways and reports three of them with an ordinary-looking result:
a satisfied request, the round cap, the reset cap, and a ``next_speaker`` matching nobody.
Only the first is success. The discriminators used here are counts and ledger flags — the
termination MESSAGE is deliberately not matched, because it is an f-string built inline
(``:1253``) with no constant to pin against, and a model's own final answer could contain the
same words.
A ledger naming nobody is checked FIRST and outranks everything else. It is the only ending
in which the orchestrator produced an answer having asked no participant at all (``:1128``),
so the run has a result that no work stands behind — the same class as the retired E2 finding,
and not something a later, softer verdict should be allowed to paper over.
"""
if any(not entry.speaker_known for entry in ledger_log):
return "unknown_speaker"
if ledger_log and ledger_log[-1].is_request_satisfied:
return None
if len(ledger_log) >= contract.max_rounds:
raise BudgetExceeded("exploration_rounds", contract.max_rounds, len(ledger_log))
# Whatever is left ended without satisfying the request and without exhausting the round cap:
# the manager reset until it ran out of resets, or terminated with nothing to show. Both are
# the same fact for a caller — the exploration gave up — so they share one token rather than
# inventing a distinction the ledger cannot support.
return "stalled"
Reference in a new issue View git blame Copy permalink