open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(explore): U4+U13 del 3 - multi-base er en PARTISJON, ikke en videre signatur (ORDRE 20260825T080753Z) [skip-docs]

Browse source 
Ordrens premiss («endrer run_project sin signatur») er FALSIFISERT foer bygging,
og ordren ba selv om nettopp den sjekken. run_project kan ikke ta mer enn en
bundle_dir: paa bundle-stien avleder den FIRE enkeltverdier fra DEN basen -
prosjektet (_project_from_bundle fail-faster naar basens egen project_id ikke er
den forespurte), validatorens stage-0-baseline, agentenes lesekontekst og
ExpeL-noekkelen - og returnerer ETT stemplet RunResult. En andre katalog paa den
signaturen ville tvunget et stille velg-en for alle fire. Planens egen setning
sier det samme lest naert: «pipelinen kjoeres per bundle som i dag
(run_portfolio-formen)» = N kall, ikke ett kall med N.

Levert form, tre soemmer:
- mandate.Approach.bundle_id (default "" - hvert mandat skrevet foer i dag er
  fortsatt gyldig og dispatchbart uendret)
- mandate.route_by_bundle - ren partisjon, fail-fast paa et mandat som ikke kan
  utfoeres som skrevet (load_mandate-regelen). En base som ingen approach navngir
  kjoeres ikke; med NOEYAKTIG en base absorberer den alt uten navn, som ikke er en
  gjetning men det eneste mulige svaret.
- run.run_mandate_across_bundles - dispatchen. INGEN project_id-parameter:
  hver base sitt prosjekt leses fra DEN basens egen IR-projeksjon, altsaa den
  verdien _project_from_bundle allerede fail-faster mot. En delt VerdictStore
  traades paa tvers (kryss-base-laering, run_portfolio-formen), og med
  portfolio_meter gjelder de to S3.4-tennene som HAR mening her: oppstartsnekt
  (BudgetRefused) og aldri-startet + budget_stop + not_evaluated-rader.

INGEN eksisterende kaller endrer signatur - CLI, hosting og simulation sender
fortsatt en base hver, og kan fortsatt gjoere det.

explore(): hver myntet approach baerer bundle_id; markoeren kan navngi basen;
en umerket markoer med flere baser NEKTES (HypothesisParseError), med en base
resolveres den. Froe-approaches VALIDERES men skrives ALDRI om (§ C.6 doer 1 er
en bevaringsregel) - og valideringen skjer FOER foerste modellkall, samme
oekt-57-hoist-grunn: ved unntaket alene ser en nekt etter forbruket identisk ut.
explore()-docstringens gamle aerlighets-grense («venter paa at run_project tar
mer enn en bundle_dir») er RETTET - den ville vaert usann fra i dag.

Load-bearing MAALT, 12 mutasjoner alle roede mot HELE suiten, groenn kontroll
1020/5 (fra 999/5 + 21 nye tester); golden demo-transcript.stdout BYTE-UENDRET
(ea8c534773acdbe41ae68f2c55724d69aaf8be4f): detach myntet bundle_id (2) ·
stille gjennomfall ved >1 base (1) · ukjent id resolvert etter rekkefoelge (1) ·
froe-sjekk etter forbruket (2) · ruteren gjetter foerste base (1) · uroutbar
approach droppet (1) · dispatchen kollapser til en base (5) · project_id fra
foerste base (2) · detach aldri-startet-tannen (1) · unreached urapportert (1) ·
fersk store per base (1) · detach oppstartsnekten (2).

EN MUTASJON FALSIFISERTE TESTEN FOERST (repoets vakuoes-gate-klasse, aattende
gang): store-testen sammenlignet med ==, og VerdictStore er en pydantic-modell
med VERDI-likhet - tre ulike tomme stores er alle like, saa «fersk store per
base» lot hele suiten staa groenn. Delt INSTANS er paastanden, saa testen
asserterer naa paa `is`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3YHobQC3WzYVoxSgZus4d
This commit is contained in:
Kjell Tore Guttormsen 2026-08-25 13:35:39 +02:00
commit 18af86e422
6 changed files with 1051 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

118
src/portfolio_optimiser/explore.py
View file

@ -703,9 +703,46 @@ class HypothesisParseError(ExplorationError):
"""
def _parse_hypotheses(texts: Sequence[str]) -> list[tuple[str, str]]:
"""Every marked ``(label, rationale)`` pair the hypothesiser committed to, in turn order."""
found: list[tuple[str, str]] = []
def _resolve_hypothesis_bundle(raw: str, bundle_ids: Sequence[str]) -> str:
"""Which knowledge base a marked hypothesis belongs to (§ C.7), fail-closed.
Three rules, and the middle one is the whole reason the field can default at all:
* a stated id must be one that was CONFIGURED — refused by name, never resolved by position,
exactly as ``_resolve_bundle`` refuses an unknown base for the read tools. A hypothesis that
names a base must not be treated more leniently than a read of one.
* a silent marker with at most one base configured resolves to that base (or to ``""`` with
none). With one configured base there is no other value the field could take, so this is the
only answer rather than a guess.
* a silent marker with SEVERAL configured refuses. A marked line is a claim the loop committed
to; one that cannot be routed is a claim it could not finish making, and the marker is
precisely what makes that affordable to refuse — an unmarked turn is not a hypothesis, so
there is no silence being turned into an error.
"""
if raw:
if raw not in bundle_ids:
known = ", ".join(bundle_ids) or "(none configured)"
raise ExplorationError(
f"hypothesis names knowledge base {raw!r}, which is not configured for this "
f"exploration; configured: {known}"
)
return raw
if len(bundle_ids) <= 1:
return bundle_ids[0] if bundle_ids else ""
raise HypothesisParseError(
f"a marked hypothesis must name its knowledge base when several are configured "
f'({", ".join(bundle_ids)}): add "bundle_id" to the marker payload. Routing it here '
"would evaluate a direction against a project nobody asked about"
)
def _parse_hypotheses(
texts: Sequence[str], bundle_ids: Sequence[str]
) -> list[tuple[str, str, str]]:
"""Every marked ``(label, rationale, bundle_id)`` triple the hypothesiser committed to, in turn
order. The base is resolved HERE rather than at minting time so an unroutable claim is refused
while the line that made it is still in hand for the error message."""
found: list[tuple[str, str, str]] = []
for text in texts:
for line in text.splitlines():
stripped = line.strip()
@ -724,12 +761,48 @@ def _parse_hypotheses(texts: Sequence[str]) -> list[tuple[str, str]]:
"a marked hypothesis needs a non-empty 'label' and 'rationale'; got: "
f"{stripped}"
)
found.append((str(data["label"]), str(data["rationale"])))
found.append(
(
str(data["label"]),
str(data["rationale"]),
_resolve_hypothesis_bundle(str(data.get("bundle_id") or ""), bundle_ids),
)
)
return found
def refuse_unroutable_seeds(seeds: Sequence[Approach], bundle_ids: Sequence[str]) -> None:
"""Refuse an expert's seed that could not be dispatched — BEFORE the first model call.
Validation, ALDRI reparasjon, and here the second half is the load-bearing one: a seed is the
expert's own words and § C.6 door 1 is a PRESERVATION rule, so filling in a missing
``bundle_id`` on their behalf would put their name on a routing decision they did not make.
The single-base default belongs to ``route_by_bundle``, at consumption, where it is unambiguous
by construction — never to a rewrite of the input.
Called before the loop starts rather than after it returns, and that placement is the økt-57
outbox-hoist precedent: at the exception alone, a refusal after the exploration has spent its
whole budget looks identical to one before it spent anything. The mandate is undispatchable
either way; what is at stake is whether the expert pays to find that out.
"""
for seed in seeds:
if seed.bundle_id:
if seed.bundle_id not in bundle_ids:
known = ", ".join(bundle_ids) or "(none configured)"
raise ExplorationError(
f"seed approach {seed.id!r} names knowledge base {seed.bundle_id!r}, which is "
f"not configured for this exploration; configured: {known}"
)
elif len(bundle_ids) > 1:
raise ExplorationError(
f"seed approach {seed.id!r} names no knowledge base and {len(bundle_ids)} are "
f"configured ({', '.join(bundle_ids)}); set Approach.bundle_id so the mandate can "
"be dispatched to the base the hypothesis is actually about"
)
def _mint_approaches(
seeds: Sequence[Approach], discovered: Sequence[tuple[str, str]]
seeds: Sequence[Approach], discovered: Sequence[tuple[str, str, str]]
) -> tuple[Approach, ...]:
"""Seeds FIRST, untouched, then one approach per discovered direction.
@ -742,7 +815,7 @@ def _mint_approaches(
taken = {approach.id for approach in seeds}
minted: list[Approach] = list(seeds)
counter = 0
for label, rationale in discovered:
for label, rationale, bundle_id in discovered:
counter += 1
candidate = f"hypothesis-{counter}"
while candidate in taken or candidate == OWN_PROPOSAL_ID:
@ -752,7 +825,12 @@ def _mint_approaches(
# description is the hypothesiser's own words, VERBATIM: ``generate._build_messages`` feeds
# an Approach.description to the proposer unchanged, and the reason a direction is worth
# trying is exactly the half a model cannot re-derive from the cost table.
minted.append(Approach(id=candidate, label=label, description=rationale))
#
# ``bundle_id`` is stamped on a MINTED approach because there is nothing here to preserve —
# the opposite call from the seeds above, which pass through untouched (§ C.6 door 1).
minted.append(
Approach(id=candidate, label=label, description=rationale, bundle_id=bundle_id)
)
return tuple(minted)
@ -792,13 +870,19 @@ async def explore(
returned mandate whatever the loop found, including when the loop found nothing and including
when it stopped early.
Honesty limits, stated rather than implied. (1) A run is bound to ONE mandate; multi-base
dispatch (``Approach.bundle_id``, § C.7) waits for ``run_project`` to accept more than one
``bundle_dir``, and shipping the field before its consumer would be a shape guessed instead of
measured. (2) The ``quick_validate`` verdicts the hypothesiser saw are not in
**Multi-base (§ C.7).** Every approach in the returned mandate carries ``bundle_id``, and
``mandate.route_by_bundle`` partitions the commission by it so ``run.run_mandate_across_bundles``
can run the pipeline once per base. ``run_project`` itself still takes ONE ``bundle_dir``, and
that is the measured shape rather than a step not yet taken: it derives the project, the
validator's cost baseline, the read context and the ExpeL query key from THE bundle and returns
one stamped ``RunResult``, so a second directory on that signature would force a silent
pick-one for all four.
Honesty limits, stated rather than implied. (1) The ``quick_validate`` verdicts the
hypothesiser saw are not in
``ExplorationResult``: they are level-1 advisory, and their home is the
``{run_id}-exploration.json`` artefact the CLI wiring writes — pass an ``ExplorationTrace`` to
collect them. (3) The exploration roles resolve through ``resolve_model``'s ``default``
collect them. (2) The exploration roles resolve through ``resolve_model``'s ``default``
fallback unless an operator maps them explicitly.
``trace`` is the caller's accumulator and is the ONLY way to see what a run that RAISED
@ -816,6 +900,12 @@ async def explore(
"requested and the reviewer would never be called (refused, never silently ignored)"
)
# The base ids, resolved ONCE and before anything is built: they are what a seed and a marked
# hypothesis are both routed against, and ``_bundle_index`` is the one place that decides them
# (a duplicate basename refuses here rather than letting the manager read A believing it read B).
bundle_ids = tuple(_bundle_index(bundle_dirs))
refuse_unroutable_seeds(seed_approaches, bundle_ids)
if meter is None:
meter = TokenMeter(Budget(max_tokens=contract.max_tokens, max_rounds=contract.max_rounds))
if client_factory is None:
@ -912,7 +1002,9 @@ async def explore(
if stop is None:
stop = _classify_stop(ledger_log, replans=replans, contract=contract)
discovered = _parse_hypotheses(hypothesis_texts) if stop != "unknown_speaker" else []
discovered = (
_parse_hypotheses(hypothesis_texts, bundle_ids) if stop != "unknown_speaker" else []
)
return ExplorationResult(
mandate=Mandate(
objective=prompt,

88
src/portfolio_optimiser/mandate.py
View file

@ -29,6 +29,7 @@ Two refusals are load-bearing, both at construction time:
from __future__ import annotations
from collections.abc import Sequence
from dataclasses import dataclass
from pathlib import Path
from typing import Literal
@ -50,6 +51,14 @@ class Approach(BaseModel):
id: str = Field(min_length=1)
label: str = Field(min_length=1)
description: str = ""
#: Which knowledge base this approach belongs to (§ C.7), by the base's id — the directory's
#: BASENAME, exactly as ``explore._bundle_index`` names it. DEFAULTS to empty, meaning "no base
#: named": a legitimate statement when the run has only one base to name, and what keeps every
#: mandate written before multi-base existed valid and dispatchable unchanged.
#:
#: The field is a ROUTING key, never a claim about content. It says which pipeline the approach
#: must be evaluated in, and ``route_by_bundle`` is the one place that reads it.
bundle_id: str = ""
class Mandate(BaseModel):
@ -88,6 +97,85 @@ class Mandate(BaseModel):
return self
class MandateRoutingError(ValueError):
"""A commission that cannot be executed against the bases it was given (§ C.7).
**A ``ValueError`` by construction, and that is a measurement rather than a taxonomy note.**
økt 57 paid for the opposite: ``ExplorationError`` is a ``RuntimeError`` and therefore fell
outside ``run.main``'s ``(ValueError, FileNotFoundError, ValidationError)`` refusal tuple and
outside ``hosting``'s 400 arm, so a caller's configuration mistake would have left as a
traceback on one surface and a 500 — the crash channel — on the other. A routing refusal is
exactly that class of caller mistake, so it is born inside both nets instead of being
retrofitted into them later.
"""
def route_by_bundle(mandate: Mandate, bundle_ids: Sequence[str]) -> tuple[tuple[str, Mandate], ...]:
"""Partition one commission into one sub-mandate PER knowledge base (§ C.7).
This is the whole of "multi-base", and it is deliberately a partition rather than a widening.
``run_project`` derives four single-valued things from THE bundle it is given — the project
(``_project_from_bundle`` fail-fasts when the bundle's own ``project_id`` is not the requested
one), the validator's stage-0 cost baseline, the agents' read context and the ExpeL query key —
and returns ONE ``RunResult`` with ONE ``ProvenanceStamp``. A second ``bundle_dir`` on that
signature would force a silent pick-one for all four. § C.7 says the same thing in its own
words: *pipelinen kjøres per bundle som i dag* — N calls, not one call taking N.
Order is taken from ``bundle_ids``, never from first appearance among the approaches, so the
dispatch's spend order is a property of how the run was configured rather than of how a model
happened to sequence its hypotheses.
**Fail-fast on a commission that cannot be executed as written**, mirroring ``load_mandate``'s
contract for exactly its reason: a run must never proceed on a *silently degraded* commission,
because the coverage report would then describe work nobody ordered. Two ways that happens, and
both refuse by name rather than resolving by position (the S3.2 key-collision class):
* an approach naming a base that was not configured;
* an approach naming NO base while more than one is configured — with a single base there is no
other value the field could take, so resolving it there is the only answer rather than a
guess, and it is what keeps every pre-multi-base mandate dispatchable unchanged.
A base that no approach names is NOT run: a run costs money and the commission ordered nothing
for it. The single-base case keeps its own rule above, so an own-proposals-only mandate still
reaches the one base it could possibly mean.
:raises MandateRoutingError: no bases configured, or an approach that cannot be routed.
"""
if not bundle_ids:
raise MandateRoutingError(
"a mandate cannot be routed against zero knowledge bases: an empty plan reads as "
"'there was nothing to do', which is indistinguishable from a commission that was "
"fully evaluated against nothing"
)
known = tuple(bundle_ids)
sole = known[0] if len(known) == 1 else None
grouped: dict[str, list[Approach]] = {bundle_id: [] for bundle_id in known}
for approach in mandate.approaches:
target = approach.bundle_id or sole
if target is None:
raise MandateRoutingError(
f"approach {approach.id!r} names no knowledge base and {len(known)} are "
f"configured ({', '.join(known)}); which one it belongs to is not something this "
"layer may decide on the expert's behalf"
)
if target not in grouped:
raise MandateRoutingError(
f"approach {approach.id!r} names knowledge base {target!r}, which is not "
f"configured for this run; configured: {', '.join(known)}"
)
grouped[target].append(approach)
return tuple(
(
bundle_id,
mandate.model_copy(update={"approaches": tuple(grouped[bundle_id])}),
)
for bundle_id in known
if grouped[bundle_id] or sole is not None
)
@dataclass(frozen=True)
class ApproachOutcome:
"""What became of ONE commissioned approach — one row of the run's coverage report.

183
src/portfolio_optimiser/run.py
View file

@ -70,8 +70,10 @@ from portfolio_optimiser.mandate import (
Approach,
ApproachOutcome,
Mandate,
MandateRoutingError,
announce,
load_mandate,
route_by_bundle,
settle,
)
from portfolio_optimiser.mcp_tools import (
@ -1343,6 +1345,187 @@ async def run_portfolio(
return base
@dataclass(frozen=True)
class BundleRun:
"""One knowledge base's run inside a multi-base dispatch (§ C.7).
``bundle_id`` and ``project_id`` are carried BESIDE the result rather than read back off it,
because they are what the dispatch ROUTED on: the id is how the mandate named the base, and the
project is what that base's own IR projection says it is. A reader pairing a coverage row back
to a base should not have to re-derive either.
"""
bundle_id: str
bundle_dir: str
project_id: str
result: RunResult
@dataclass(frozen=True)
class MultiBaseResult:
"""What one multi-base dispatch produced: one run per base the commission named (§ C.7).
A DISTINCT type from ``PortfolioResult``, and deliberately so. ``PortfolioResult`` keys on
"one ``RunResult`` per PROJECT in input order" and its ``runs``/``failures`` partition *the
projects that were actually submitted*; this keys on the KNOWLEDGE BASE the mandate routed each
approach to. Two bases may legitimately describe one project, which the project-keyed shape
cannot express at all — ``run_portfolio`` reads each base off ``projects[pid].bundle_dir``, so
a pid admits exactly one base there. Reusing the type would fuse two axes.
``unreached`` is the ``not_evaluated`` rule at the dispatch layer: when the global cap stops the
pass, every approach in a base that was never started is reported as unreached rather than
omitted. An omitted row is indistinguishable from an approach nobody commissioned, which is the
silence the coverage report exists to remove.
"""
runs: tuple[BundleRun, ...]
store: VerdictStore
stopped_early: bool = False
budget_stop: BudgetStop | None = None
unreached: tuple[ApproachOutcome, ...] = ()
async def run_mandate_across_bundles(
mandate: Mandate,
bundle_dirs: Sequence[str],
profile: Profile | str = Profile.LOCAL,
*,
verdict_input: dict[str, str],
store: VerdictStore | None = None,
verdict_dir: str | None = None,
dimension: Dimension | None = None,
client_factory: Callable[[str], BaseChatClient] | None = None,
max_rounds: int = _DEFAULT_MAX_ROUNDS,
max_tokens: int = _DEFAULT_MAX_TOKENS,
top_k: int = 3,
portfolio_meter: PortfolioMeter | None = None,
) -> MultiBaseResult:
"""Evaluate ONE commission across SEVERAL knowledge bases — the multi-base dispatch (§ C.7).
``mandate.route_by_bundle`` partitions the commission by ``Approach.bundle_id``, and this runs
the EXISTING ``run_project`` once per base with that base's own sub-mandate. Composition, never
a widening: ``run_project`` derives the project, the validator's stage-0 baseline, the agents'
read context and the ExpeL query key from THE bundle it is handed, and returns one stamped
``RunResult`` — a second ``bundle_dir`` on that signature would force a silent pick-one for all
four. § C.7 words it the same way: *pipelinen kjøres per bundle som i dag*.
**There is no ``project_id`` argument, and that is the design rather than an omission.** Each
base's project is read from THAT base's own IR projection (``okf.load_ir_projection``), which is
the same value ``_project_from_bundle`` already fail-fasts against — so a caller-supplied
constant could only ever be right for one base out of N. Turning the existing fail-fast into the
routing key removes the guess entirely.
ONE ``VerdictStore`` is threaded across every base, exactly as ``run_portfolio`` threads one
across every project: a verdict captured while evaluating base k must be able to reach base
k+1's hypothesis, and a fresh store per base would leave the loop looking wired while carrying
nothing between the runs.
**The budget has the two S3.4 teeth that apply here, and no more.** With a ``portfolio_meter``:
a remainder that cannot fund one run refuses at STARTUP (``BudgetRefused`` — a pass with room
for zero runs is a caller error, not a result), and a base that cannot be funded is NEVER
STARTED, the pass stopping with ``budget_stop`` and every unreached approach reported. Never
started is the whole point: a base merely aborted mid-run has already cost real model calls.
The wave-reservation tooth has no counterpart here — this dispatch is SEQUENTIAL, so there is
no wave of runs funded off one pre-wave remainder to over-commit.
Honesty limits, stated rather than implied. (1) A base that RAISES propagates; the
collect-and-continue policy belongs to ``run_portfolio``, where the caller submitted a batch of
independent projects, whereas here the caller asked for ONE commission to be evaluated.
(2) Without a ``portfolio_meter`` the pass's ceiling is the number of routed bases times
``max_tokens``, each run bounded on its own — the global ledger is opt-in, and this does not
re-implement it (``_run_meter`` is the one copy of the binding rule). (3) The outbox is NOT
wired: N runs need N ``run_id``s, and minting them here would default a key this repo requires
a caller to supply, for byte-determinism. A caller who needs artefacts per base calls
``run_project`` itself with the sub-mandates ``route_by_bundle`` hands back.
:raises MandateRoutingError: the commission cannot be routed against ``bundle_dirs``.
:raises BudgetRefused: a global remainder that cannot fund a single run.
"""
by_id: dict[str, str] = {}
for raw in bundle_dirs:
bundle_id = Path(raw).name
if bundle_id in by_id:
# The same refusal ``explore._bundle_index`` makes, for the same reason: the id is how
# the mandate names a base, so two bases answering to one name would let an approach be
# evaluated against A while the report says B (the S3.2 key-collision class).
raise MandateRoutingError(
f"two knowledge bases share the id {bundle_id!r} ({by_id[bundle_id]!r} and "
f"{raw!r}); an approach names a base by that id, so it must be unique"
)
by_id[bundle_id] = raw
routed = route_by_bundle(mandate, tuple(by_id))
if portfolio_meter is not None and not portfolio_meter.can_fund_run():
raise BudgetRefused(portfolio_meter.remaining(), portfolio_meter.required_per_run)
store = store if store is not None else VerdictStore(verdicts=[])
runs: list[BundleRun] = []
unreached: list[ApproachOutcome] = []
budget_stop: BudgetStop | None = None
for index, (bundle_id, sub_mandate) in enumerate(routed):
if portfolio_meter is not None and not portfolio_meter.can_fund_run():
budget_stop = BudgetStop(
limit_tokens=portfolio_meter.budget.max_total_tokens,
spent_tokens=portfolio_meter.spent,
remaining_tokens=portfolio_meter.remaining(),
required_tokens=portfolio_meter.required_per_run,
)
unreached.extend(
ApproachOutcome(
id=approach.id,
label=approach.label,
status="not_evaluated",
detail=(f"budget exhausted before knowledge base {pending_id!r} was run"),
)
for pending_id, pending in routed[index:]
for approach in pending.approaches
)
break
bundle_dir = by_id[bundle_id]
# ONE reading of the base's own project id, used both to ADDRESS the run and to LABEL it.
# A second lookup for the label would be the kø-(p) duplicate free to drift from the value
# the run was actually dispatched with.
project_id = str(okf.load_ir_projection(bundle_dir)["project_id"])
result = cast(
RunResult,
await run_project(
project_id,
profile,
docs_dir=bundle_dir,
bundle_dir=bundle_dir,
verdict_input=verdict_input,
verdict_dir=verdict_dir,
dimension=dimension,
store=store,
client_factory=client_factory,
max_rounds=max_rounds,
max_tokens=max_tokens,
top_k=top_k,
mandate=sub_mandate,
meter=_run_meter(None, portfolio_meter, max_rounds),
),
)
runs.append(
BundleRun(
bundle_id=bundle_id,
bundle_dir=bundle_dir,
project_id=project_id,
result=result,
)
)
return MultiBaseResult(
runs=tuple(runs),
store=store,
stopped_early=budget_stop is not None,
budget_stop=budget_stop,
unreached=tuple(unreached),
)
# The roles ``debate``/``generate`` ask the factory for. Fixed here so a malformed replies file is
# caught at the door instead of mid-run.
_SCRIPTED_ROLES = ("proposer", "checker")

2
tests/test_explore_callsites_loadbearing.py
View file

@ -540,7 +540,7 @@ def test_a_seeded_mandate_still_leads_the_shaped_one() -> None:
A refusal that pointed at a door which did not open would be worse than no message at all.
"""
seed = Approach(id="expert-1", label="expert's own", description="the domain expert asked")
minted = explore._mint_approaches((seed,), [(_LABEL, "shaped in the loop")])
minted = explore._mint_approaches((seed,), [(_LABEL, "shaped in the loop", "")])
assert [a.id for a in minted] == ["expert-1", "hypothesis-1"]
assert isinstance(Mandate(objective="o", approaches=minted, allow_own_proposals=True), Mandate)

8
tests/test_explore_loadbearing.py
View file

@ -947,9 +947,9 @@ def test_a_marked_line_that_will_not_parse_is_a_hard_error() -> None:
than following the tolerant RAW-inbox rule, which belongs to folders anyone may drop files in.
"""
with pytest.raises(explore.HypothesisParseError):
explore._parse_hypotheses([f"{explore.HYPOTHESIS_MARKER} not json at all"])
explore._parse_hypotheses([f"{explore.HYPOTHESIS_MARKER} not json at all"], ())
with pytest.raises(explore.HypothesisParseError):
explore._parse_hypotheses([f'{explore.HYPOTHESIS_MARKER} {{"label": "no rationale"}}'])
explore._parse_hypotheses([f'{explore.HYPOTHESIS_MARKER} {{"label": "no rationale"}}'], ())
def test_unmarked_prose_is_not_a_failure() -> None:
@ -958,7 +958,9 @@ def test_unmarked_prose_is_not_a_failure() -> None:
Without it, T24 would pass on an implementation that refused every hypothesiser turn that was
not a hypothesis, which would make the loop unusable and the strictness meaningless.
"""
assert explore._parse_hypotheses(["I looked at the index and nothing stands out yet."]) == []
assert (
explore._parse_hypotheses(["I looked at the index and nothing stands out yet."], ()) == []
)
def test_a_review_nobody_can_answer_is_refused_before_the_first_model_call() -> None:

669
tests/test_multibase_loadbearing.py Normal file
View file

@ -0,0 +1,669 @@
"""U4 + U13, del 3 — MULTI-BASE (``Approach.bundle_id``, plan § C.7).
**The premise this file corrects, measured before anything was built.** § C.7 and økt 56's own
honesty limit read as though the deliverable were "``run_project`` accepts more than one
``bundle_dir``". It cannot, and the refusal is structural rather than stylistic: on the bundle
path ``run_project`` derives FOUR single-valued things from THE bundle — the project
(``_project_from_bundle``, which fail-fasts when ``validator-input.json``'s ``project_id`` is not
the requested one), the validator's stage-0 cost baseline (S4.0's whole point being that the gate
is anchored to THAT project's real cost lines), the agents' read context, and the ExpeL query key
— and it returns ONE ``RunResult`` with ONE ``ProvenanceStamp``. A second ``bundle_dir`` would
force a silent pick-one for all four, which is the guessed-shape class this repo refuses.
§ C.7's own sentence says the same thing once read closely: *"pipelinen kjøres per bundle som i dag
(``run_portfolio``-formen)"* — N calls, one per base, not one call taking N. So the delivered shape
is: each approach RECORDS its base, a pure router PARTITIONS the mandate by base, and a thin
dispatch runs the existing ``run_project`` once per base. **No existing caller's signature changes**
— CLI, hosting and simulation each still pass one base, and each still may.
Three seams, each with its own detach signature:
* ``mandate.Approach.bundle_id`` — the field, defaulting to ``""`` so every mandate written before
this session stays valid;
* ``mandate.route_by_bundle`` — the partition, fail-fast on a commission that cannot be executed as
written (``load_mandate``'s rule: a run must never proceed on a silently degraded commission);
* ``run.run_mandate_across_bundles`` — the dispatch, whose ``project_id`` per base comes from THAT
base's own IR projection and never from a caller-supplied constant.
"""
from __future__ import annotations
import json
import shutil
from collections.abc import Callable
from pathlib import Path
from typing import Any
import pytest
from agent_framework import BaseChatClient
from portfolio_optimiser import explore, run as run_module
from portfolio_optimiser.budget import BudgetRefused, PortfolioBudget, PortfolioMeter
from portfolio_optimiser.explore import ExplorationContract, ExplorationError, HypothesisParseError
from portfolio_optimiser.mandate import (
Approach,
Mandate,
MandateRoutingError,
route_by_bundle,
)
from portfolio_optimiser.run import RunResult, run_mandate_across_bundles
from portfolio_optimiser.simulation import ScriptedChatClient
from portfolio_optimiser.verdicts import VerdictStore
_EXAMPLES = Path(__file__).resolve().parents[1] / "shared" / "examples"
#: Three bases with three DISTINCT project ids — which is what makes "the project comes from the
#: base, not from the caller" a claim a test can actually falsify.
_BYGG = _EXAMPLES / "bygg-energi-mikro" # BYGG-KONTOR-NORD
_TUNNEL = _EXAMPLES / "tunnel-hauglia" # TUNNEL-HAUGLIA
_VEGLYS = _EXAMPLES / "veglys-fv-soer" # VEGLYS-FV-SOER
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (sim)"}
# ---------------------------------------------------------------------------------------------
# 1. The field. mandate.py stays pydantic + stdlib (D7-portable, test_okf_is_maf_free).
# ---------------------------------------------------------------------------------------------
def test_an_approach_records_which_knowledge_base_it_belongs_to() -> None:
"""T1: ``Approach`` carries ``bundle_id``, and it DEFAULTS to the empty string.
The default is what keeps every mandate JSON written before this session valid, and every
existing ``Approach(...)`` constructor call unaffected — the same reason ``RunResult.coverage``
defaults. Empty means "no base named", which is a legitimate statement when the run has only
one base to name.
"""
assert Approach(id="a", label="A").bundle_id == ""
assert Approach(id="a", label="A", bundle_id="tunnel-hauglia").bundle_id == "tunnel-hauglia"
# ---------------------------------------------------------------------------------------------
# 2. The router. Pure, framework-neutral, and fail-fast on a commission it cannot execute.
# ---------------------------------------------------------------------------------------------
def _mandate(*approaches: Approach, own: bool = True) -> Mandate:
return Mandate(objective="find savings", approaches=approaches, allow_own_proposals=own)
def test_the_router_partitions_the_mandate_one_sub_mandate_per_named_base() -> None:
"""T2: each base gets a sub-mandate carrying ONLY its own approaches, in ``bundle_ids`` order.
Order is taken from the configured bases rather than from first appearance among the
approaches, so the dispatch's spend order is a property of how the run was configured and not
of how a model happened to sequence its hypotheses.
"""
a = Approach(id="a", label="A", bundle_id="tunnel-hauglia")
b = Approach(id="b", label="B", bundle_id="veglys-fv-soer")
c = Approach(id="c", label="C", bundle_id="tunnel-hauglia")
routed = route_by_bundle(_mandate(a, b, c), ("veglys-fv-soer", "tunnel-hauglia"))
assert [bundle_id for bundle_id, _ in routed] == ["veglys-fv-soer", "tunnel-hauglia"]
assert [ap.id for ap in routed[0][1].approaches] == ["b"]
assert [ap.id for ap in routed[1][1].approaches] == ["a", "c"]
# The commission's own fields travel with every partition: each sub-run is still working on the
# same objective and under the same "and/or your own" permission.
assert routed[0][1].objective == "find savings"
assert routed[0][1].allow_own_proposals is True
def test_a_single_base_absorbs_every_unassigned_approach() -> None:
"""T3: with exactly ONE base configured, an approach naming none routes to it.
Not a guess — with one configured base there is no other value the field could take, and
``_bundle_index`` already guarantees the id is unique. This is what keeps a single-base
mandate (every mandate that exists today) dispatchable unchanged.
"""
routed = route_by_bundle(_mandate(Approach(id="a", label="A")), ("tunnel-hauglia",))
assert [bundle_id for bundle_id, _ in routed] == ["tunnel-hauglia"]
assert [ap.id for ap in routed[0][1].approaches] == ["a"]
def test_an_unassigned_approach_among_several_bases_is_refused_not_guessed() -> None:
"""T4: the discriminator for T3 — with TWO bases, an unnamed approach REFUSES the dispatch.
Silently sending it to the first base would evaluate a commissioned direction against a
project nobody asked about and report it as done. Refusing is ``load_mandate``'s rule applied
one layer on: a run must never proceed on a silently degraded commission, because the coverage
report would then describe work nobody ordered.
"""
with pytest.raises(MandateRoutingError) as exc:
route_by_bundle(_mandate(Approach(id="a", label="A")), ("tunnel-hauglia", "veglys-fv-soer"))
# The REPR, never the bare letter: "a" is a substring of almost any English sentence, so an
# assertion on it would hold against a refusal raised for an entirely different reason.
assert "'a'" in str(exc.value)
def test_an_approach_naming_an_unconfigured_base_is_refused_by_name() -> None:
"""T5: an approach whose ``bundle_id`` matches no configured base refuses, naming both.
Resolving it by position instead would be the S3.2 key-collision class: an approach evaluated
against a base it does not belong to, with nothing in the report saying so.
"""
approach = Approach(id="a", label="A", bundle_id="does-not-exist")
with pytest.raises(MandateRoutingError) as exc:
route_by_bundle(_mandate(approach), ("tunnel-hauglia", "veglys-fv-soer"))
message = str(exc.value)
assert "does-not-exist" in message
assert "tunnel-hauglia" in message
def test_the_routing_refusal_is_a_value_error() -> None:
"""T6: ``MandateRoutingError`` subclasses ``ValueError`` — a TYPE claim, not a taxonomy note.
økt 57 measured the cost of getting this wrong the other way: ``ExplorationError`` is a
``RuntimeError`` and therefore fell outside ``run.main``'s ``(ValueError, FileNotFoundError,
ValidationError)`` refusal tuple and outside hosting's 400 arm, so a caller's configuration
mistake would have left as a traceback and a 500. A routing refusal is exactly that class of
caller mistake, so it is born inside both nets rather than retrofitted into them.
"""
assert issubclass(MandateRoutingError, ValueError)
def test_routing_against_no_base_at_all_is_refused() -> None:
"""T7: zero configured bases refuses rather than returning an empty plan.
An empty plan reads as "there was nothing to do", which is indistinguishable from a mandate
that was fully evaluated against nothing — the omitted-row silence ``ApproachOutcome``'s
``not_evaluated`` status exists to remove.
"""
with pytest.raises(MandateRoutingError):
route_by_bundle(_mandate(Approach(id="a", label="A")), ())
# ---------------------------------------------------------------------------------------------
# 3. Assignment: what the exploration puts in the field, and what it refuses to put there.
# ---------------------------------------------------------------------------------------------
_CONTRACT = ExplorationContract(
max_rounds=6,
max_tokens=100_000,
max_stall_count=2,
max_reset_count=1,
max_plan_revisions=0,
enable_plan_review=False,
)
_PROMPT = "Find the cheapest saving available."
def _ledger_json(*, satisfied: bool, speaker: str) -> str:
return json.dumps(
{
"is_request_satisfied": {"reason": "r", "answer": satisfied},
"is_in_loop": {"reason": "r", "answer": False},
"is_progress_being_made": {"reason": "r", "answer": True},
"next_speaker": {"reason": "r", "answer": speaker},
"instruction_or_question": {"reason": "r", "answer": "Shape one hypothesis."},
}
)
def _manager_script(ledgers: list[str]) -> Callable[[str, str], str]:
"""Route a manager prompt blob to its scripted reply. The ORDER of these branches is
load-bearing and was measured in økt 56 (§ F, A6): the selector receives the CONCATENATION of
every message in the call, so a later-stage prompt still carries the earlier stage's text."""
def _select(blob: str, _role: str) -> str:
if "provide the final answer" in blob:
return "FINAL: exploration done."
if "pure JSON format" in blob:
return ledgers.pop(0) if ledgers else _ledger_json(satisfied=True, speaker="navigator")
if "went wrong on this last run" in blob:
return "PLAN-UPDATE: revised plan."
if "rewrite the following fact sheet" in blob:
return "FACTS-UPDATE: revised facts."
if "bullet-point plan" in blob:
return "PLAN: - ask the hypothesiser"
if "pre-survey" in blob:
return "FACTS: the bundle is anchored."
return "{}"
return _select
def _factory(
*, ledgers: list[str], hypothesiser: list[str], sink: list[str] | None = None
) -> Callable[[str], BaseChatClient]:
"""One fresh ``ScriptedChatClient`` per role. ``sink`` records every prompt that reached a
client, which is how "refused BEFORE the first model call" becomes measurable rather than
merely asserted at the exit code."""
def factory(role: str) -> BaseChatClient:
if role == explore.MANAGER_ROLE:
return ScriptedChatClient(reply_selector=_manager_script(ledgers), role=role, sink=sink)
if role == explore.HYPOTHESISER_ROLE:
replies = list(hypothesiser)
def _hyp(_blob: str, _role: str) -> str:
return replies.pop(0) if replies else "nothing further."
return ScriptedChatClient(reply_selector=_hyp, role=role, sink=sink)
return ScriptedChatClient("NAVIGATOR: index read.", role=role, sink=sink)
return factory
def _hypothesis_line(label: str, rationale: str, bundle_id: str | None = None) -> str:
payload: dict[str, Any] = {"label": label, "rationale": rationale}
if bundle_id is not None:
payload["bundle_id"] = bundle_id
return f"{explore.HYPOTHESIS_MARKER} " + json.dumps(payload)
def _two_round_ledgers() -> list[str]:
return [
_ledger_json(satisfied=False, speaker=explore.HYPOTHESISER_ROLE),
_ledger_json(satisfied=True, speaker=explore.HYPOTHESISER_ROLE),
]
@pytest.mark.asyncio
async def test_a_marked_hypothesis_may_name_its_base_and_the_mandate_carries_it() -> None:
"""T8: the marker's ``bundle_id`` reaches ``Approach.bundle_id``.
Asserted DIRECTLY on the returned mandate rather than through anything downstream: with a
single base configured a router that dropped the field would route identically, so a
behavioural assertion here could not tell an implementation that assigns from one that does
not. Two bases are configured for the same reason.
"""
result = await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL), str(_VEGLYS)),
client_factory=_factory(
ledgers=_two_round_ledgers(),
hypothesiser=[_hypothesis_line("LED retrofit", "old fixtures", "veglys-fv-soer")],
),
)
assert [a.bundle_id for a in result.mandate.approaches] == ["veglys-fv-soer"]
@pytest.mark.asyncio
async def test_with_one_base_a_marker_that_names_none_still_yields_an_assigned_approach() -> None:
"""T9: one base configured, marker silent — the minted approach carries that base's id.
Again asserted on the FIELD. This is the arm that would go quietly green under an
implementation that never assigns anything, which is precisely why T8 configures two bases and
why neither test reads the field through the router.
"""
result = await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL),),
client_factory=_factory(
ledgers=_two_round_ledgers(),
hypothesiser=[_hypothesis_line("LED retrofit", "old fixtures")],
),
)
assert [a.bundle_id for a in result.mandate.approaches] == ["tunnel-hauglia"]
@pytest.mark.asyncio
async def test_with_several_bases_a_marker_that_names_none_is_refused() -> None:
"""T10: the discriminator for T9 — two bases and a silent marker is a hard error.
A marked line is a claim the loop committed to; one that cannot be routed is a claim it could
not finish making. That is the ``write_concept_file`` rule (validation, never repair) and NOT
the tolerant RAW-inbox rule, because this is the product of the run.
"""
with pytest.raises(HypothesisParseError):
await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL), str(_VEGLYS)),
client_factory=_factory(
ledgers=_two_round_ledgers(),
hypothesiser=[_hypothesis_line("LED retrofit", "old fixtures")],
),
)
@pytest.mark.asyncio
async def test_a_marker_naming_an_unconfigured_base_is_refused() -> None:
"""T11: a ``bundle_id`` matching no configured base refuses — ``_resolve_bundle``'s own rule.
The navigator's tools already refuse an unknown base id rather than resolving it by order; a
hypothesis that names one must not be treated more leniently than a read of one.
"""
with pytest.raises(ExplorationError):
await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL), str(_VEGLYS)),
client_factory=_factory(
ledgers=_two_round_ledgers(),
hypothesiser=[_hypothesis_line("LED", "old fixtures", "no-such-base")],
),
)
@pytest.mark.asyncio
async def test_a_seed_naming_an_unknown_base_is_refused_before_the_first_model_call() -> None:
"""T12: an unroutable SEED refuses with ZERO model calls made.
The assertion is on the sink, not on the exception, and that is the økt-57 outbox-hoist
precedent: at the exception alone, a refusal AFTER the exploration has spent its whole budget
looks identical to one before. The expert's mandate cannot be dispatched either way — what is
at stake is whether they pay to find out.
"""
sink: list[str] = []
with pytest.raises(ExplorationError):
await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL), str(_VEGLYS)),
seed_approaches=(Approach(id="s1", label="Seed", bundle_id="no-such-base"),),
client_factory=_factory(ledgers=_two_round_ledgers(), hypothesiser=[], sink=sink),
)
assert sink == []
@pytest.mark.asyncio
async def test_an_unassigned_seed_with_several_bases_is_refused_before_the_first_call() -> None:
"""T13: a seed naming no base, with more than one configured, refuses — also before spending.
Same defect as T10 one door earlier (§ C.6 door 1 rather than the loop's own findings). The
expert configured several bases; which one their hypothesis belongs to is a thing only they
know, and inventing it would put their name on a direction they did not commission.
"""
sink: list[str] = []
with pytest.raises(ExplorationError):
await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL), str(_VEGLYS)),
seed_approaches=(Approach(id="s1", label="Seed"),),
client_factory=_factory(ledgers=_two_round_ledgers(), hypothesiser=[], sink=sink),
)
assert sink == []
@pytest.mark.asyncio
async def test_a_seed_is_never_rewritten_only_validated() -> None:
"""T14: with one base, an unassigned seed comes back EXACTLY as it was written.
§ C.6 door 1 is a preservation rule: the seed is the expert's own words, and the mandate hands
``description`` to the proposer verbatim. Filling in ``bundle_id`` on their behalf would be
repair of an input, which this repo refuses even when the repair is obviously right — the
single-base default belongs to the ROUTER, at consumption, where it is unambiguous by
construction.
"""
seed = Approach(id="s1", label="Night setback", description="the expert's own words")
result = await explore.explore(
_PROMPT,
contract=_CONTRACT,
bundle_dirs=(str(_TUNNEL),),
seed_approaches=(seed,),
client_factory=_factory(ledgers=_two_round_ledgers(), hypothesiser=[]),
)
assert result.mandate.approaches[0] == seed
assert result.mandate.approaches[0].bundle_id == ""
# ---------------------------------------------------------------------------------------------
# 4. The dispatch. One run_project per base — the existing function, composed, never widened.
# ---------------------------------------------------------------------------------------------
def _recorder(
calls: list[dict[str, Any]], *, meter: PortfolioMeter | None = None, charge: int = 0
) -> Callable[..., Any]:
"""A stand-in for ``run_project`` that records exactly the kwargs it was called with.
Deliberately paired with ``test_the_dispatch_composes_with_the_real_run_project`` below: a
recorder that swallows ``**kwargs`` proves the dispatch's ROUTING but would happily accept an
argument ``run_project`` does not take — the Fase 4e defect, one layer up.
``charge`` credits the global ledger the way a real run does (its ``TokenMeter`` is bound to the
same ``PortfolioMeter``). Without it the remainder never moves, and an admission check made once
per base could not be told apart from one made once per pass.
"""
async def _fake(project_id: str, profile: Any = "local", **kwargs: Any) -> Any:
calls.append({"project_id": project_id, "profile": profile, **kwargs})
if meter is not None and charge:
meter.record(charge)
return _stub_result()
return _fake
def _stub_result() -> Any:
class _Stub:
coverage: tuple[Any, ...] = ()
provenance = None
return _Stub()
@pytest.mark.asyncio
async def test_the_dispatch_runs_one_pipeline_per_base_with_that_bases_approaches(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""T15: two bases -> two ``run_project`` calls, each bound to ITS base and ITS approaches.
This is § C.7's "pipelinen kjøres per bundle" made falsifiable. A dispatch that collapsed the
partition would still return a result — one run, one outcome — and nothing else in the suite
would notice.
"""
calls: list[dict[str, Any]] = []
monkeypatch.setattr(run_module, "run_project", _recorder(calls))
mandate = _mandate(
Approach(id="a", label="A", bundle_id="tunnel-hauglia"),
Approach(id="b", label="B", bundle_id="veglys-fv-soer"),
)
await run_mandate_across_bundles(
mandate,
(str(_TUNNEL), str(_VEGLYS)),
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
)
assert len(calls) == 2
assert [c["bundle_dir"] for c in calls] == [str(_TUNNEL), str(_VEGLYS)]
assert [[ap.id for ap in c["mandate"].approaches] for c in calls] == [["a"], ["b"]]
@pytest.mark.asyncio
async def test_each_bases_project_id_comes_from_that_base_not_from_the_caller(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""T16: the ``project_id`` per call is read from THAT base's own IR projection.
The dispatch takes no ``project_id`` argument at all, and that is the design point rather than
an omission: ``_project_from_bundle`` already fail-fasts when a bundle's ``project_id`` is not
the requested one, so a caller-supplied constant could only ever be right for one of N bases.
Turning that existing fail-fast into the routing key removes the guess entirely.
"""
calls: list[dict[str, Any]] = []
monkeypatch.setattr(run_module, "run_project", _recorder(calls))
mandate = _mandate(
Approach(id="a", label="A", bundle_id="tunnel-hauglia"),
Approach(id="b", label="B", bundle_id="veglys-fv-soer"),
)
await run_mandate_across_bundles(
mandate,
(str(_TUNNEL), str(_VEGLYS)),
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
)
assert [c["project_id"] for c in calls] == ["TUNNEL-HAUGLIA", "VEGLYS-FV-SOER"]
@pytest.mark.asyncio
async def test_one_base_is_dispatched_exactly_as_a_single_run(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""T17: the control — a single base makes exactly ONE call, carrying the whole mandate.
Without this arm T15 could pass on an implementation that fanned out over configured bases
regardless of what the mandate said, spending a run on every base a caller merely made
available.
"""
calls: list[dict[str, Any]] = []
monkeypatch.setattr(run_module, "run_project", _recorder(calls))
await run_mandate_across_bundles(
_mandate(Approach(id="a", label="A"), Approach(id="b", label="B")),
(str(_TUNNEL),),
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
)
assert len(calls) == 1
assert [ap.id for ap in calls[0]["mandate"].approaches] == ["a", "b"]
@pytest.mark.asyncio
async def test_a_base_that_cannot_be_funded_is_never_started_and_its_approaches_are_reported(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""T18: the S3.4 admission tooth — an unfundable base is NEVER STARTED, and it SAYS SO.
Two halves, and both are load-bearing. Never-started is the point: a base that is merely
aborted mid-run has already cost real model calls. And reporting is the ``not_evaluated``
rule — an approach the pass never reached must appear as unreached, because an omitted row is
indistinguishable from an approach nobody commissioned.
"""
calls: list[dict[str, Any]] = []
meter = PortfolioMeter(PortfolioBudget(max_total_tokens=1_000, max_tokens_per_run=500))
monkeypatch.setattr(run_module, "run_project", _recorder(calls, meter=meter, charge=600))
result = await run_mandate_across_bundles(
_mandate(
Approach(id="a", label="A", bundle_id="tunnel-hauglia"),
Approach(id="b", label="B", bundle_id="veglys-fv-soer"),
),
(str(_TUNNEL), str(_VEGLYS)),
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
portfolio_meter=meter,
)
# Base 1 is funded (1000 left, 500 required) and spends 600; base 2 then has 400 left against a
# 500 reserve. The SECOND base is the one that must never start.
assert [c["bundle_dir"] for c in calls] == [str(_TUNNEL)]
assert result.stopped_early is True
assert result.budget_stop is not None
assert {row.id for row in result.unreached} == {"b"}
assert all(row.status == "not_evaluated" for row in result.unreached)
@pytest.mark.asyncio
async def test_a_pass_that_can_fund_nothing_at_all_is_refused_at_startup() -> None:
"""T19: a global remainder below one run's reserve raises ``BudgetRefused`` before anything
loads — ``run_portfolio``'s startup refusal, same primitive, same reason: a pass that has room
for zero runs is a caller error, not a result.
"""
meter = PortfolioMeter(
PortfolioBudget(max_total_tokens=1_000, max_tokens_per_run=500), spent=1_000
)
with pytest.raises(BudgetRefused):
await run_mandate_across_bundles(
_mandate(Approach(id="a", label="A")),
(str(_TUNNEL),),
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
portfolio_meter=meter,
)
def _reply_for(code: str, quantity: float, unit_cost: float, claimed: int) -> str:
return json.dumps(
{
"measure": "Redusert omfang",
"affected_items": [{"code": code, "quantity": quantity, "unit_cost": unit_cost}],
"claimed_saving_nok": claimed,
}
)
@pytest.mark.asyncio
async def test_the_dispatch_composes_with_the_real_run_project(tmp_path: Path) -> None:
"""T20: every kwarg the dispatch passes is a REAL ``run_project`` parameter.
The Fase 4e proof, one layer up: every routing test above uses a stand-in that swallows
``**kwargs``, so the dispatch could name an argument ``run_project`` does not take — or pass
one twice — and not a single one of them would notice, while a live call raised ``TypeError``.
Driven against two real bases end to end, offline, through the scripted client seam.
"""
bases = []
for src in (_BYGG, _TUNNEL):
dst = tmp_path / src.name
shutil.copytree(src, dst)
bases.append(str(dst))
def factory(role: str) -> BaseChatClient:
return ScriptedChatClient(
reply_selector=lambda _blob, _role: _reply_for("ENERGI-TOTAL-EL", 300000, 1.0, 30_000),
role=role,
)
result = await run_mandate_across_bundles(
_mandate(
Approach(id="a", label="A", bundle_id="bygg-energi-mikro"),
Approach(id="b", label="B", bundle_id="tunnel-hauglia"),
),
tuple(bases),
"local",
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
client_factory=factory,
max_rounds=1,
)
assert [r.bundle_id for r in result.runs] == ["bygg-energi-mikro", "tunnel-hauglia"]
assert [r.project_id for r in result.runs] == ["BYGG-KONTOR-NORD", "TUNNEL-HAUGLIA"]
assert all(isinstance(r.result, RunResult) for r in result.runs)
# Each run answered for ITS OWN commissioned approach, and for nobody else's.
assert [{row.id for row in r.result.coverage} for r in result.runs] == [
{"a", "own-proposal"},
{"b", "own-proposal"},
]
@pytest.mark.asyncio
async def test_one_store_is_threaded_across_every_base(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""T21: the SAME ``VerdictStore`` instance reaches every base's run.
The cross-base learning claim, and the same one ``run_portfolio`` makes across projects: a
verdict captured while evaluating base k must be able to reach base k+1's hypothesis. A fresh
store per base would leave the loop looking wired while carrying nothing between the runs.
**Asserted on IDENTITY, and that is a correction the mutation forced** (this repo's vacuous-gate
class, eighth occurrence). The first version compared with ``==``, and ``VerdictStore`` is a
pydantic model with VALUE equality — so a mutation handing every base its own
``VerdictStore(verdicts=[])`` left the whole suite green: three distinct empty stores are all
equal to one another. Sharing an instance is the actual claim, so ``is`` is the actual test.
"""
calls: list[dict[str, Any]] = []
monkeypatch.setattr(run_module, "run_project", _recorder(calls))
store = VerdictStore(verdicts=[])
await run_mandate_across_bundles(
_mandate(
Approach(id="a", label="A", bundle_id="tunnel-hauglia"),
Approach(id="b", label="B", bundle_id="veglys-fv-soer"),
),
(str(_TUNNEL), str(_VEGLYS)),
verdict_input=_VERDICT_INPUT,
store=store,
)
assert len(calls) == 2
assert all(c["store"] is store for c in calls)