open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(explore): U4+U13 del 3 - multi-base er en PARTISJON, ikke en videre signatur (ORDRE 20260825T080753Z) [skip-docs]

Browse source 
Ordrens premiss («endrer run_project sin signatur») er FALSIFISERT foer bygging,
og ordren ba selv om nettopp den sjekken. run_project kan ikke ta mer enn en
bundle_dir: paa bundle-stien avleder den FIRE enkeltverdier fra DEN basen -
prosjektet (_project_from_bundle fail-faster naar basens egen project_id ikke er
den forespurte), validatorens stage-0-baseline, agentenes lesekontekst og
ExpeL-noekkelen - og returnerer ETT stemplet RunResult. En andre katalog paa den
signaturen ville tvunget et stille velg-en for alle fire. Planens egen setning
sier det samme lest naert: «pipelinen kjoeres per bundle som i dag
(run_portfolio-formen)» = N kall, ikke ett kall med N.

Levert form, tre soemmer:
- mandate.Approach.bundle_id (default "" - hvert mandat skrevet foer i dag er
  fortsatt gyldig og dispatchbart uendret)
- mandate.route_by_bundle - ren partisjon, fail-fast paa et mandat som ikke kan
  utfoeres som skrevet (load_mandate-regelen). En base som ingen approach navngir
  kjoeres ikke; med NOEYAKTIG en base absorberer den alt uten navn, som ikke er en
  gjetning men det eneste mulige svaret.
- run.run_mandate_across_bundles - dispatchen. INGEN project_id-parameter:
  hver base sitt prosjekt leses fra DEN basens egen IR-projeksjon, altsaa den
  verdien _project_from_bundle allerede fail-faster mot. En delt VerdictStore
  traades paa tvers (kryss-base-laering, run_portfolio-formen), og med
  portfolio_meter gjelder de to S3.4-tennene som HAR mening her: oppstartsnekt
  (BudgetRefused) og aldri-startet + budget_stop + not_evaluated-rader.

INGEN eksisterende kaller endrer signatur - CLI, hosting og simulation sender
fortsatt en base hver, og kan fortsatt gjoere det.

explore(): hver myntet approach baerer bundle_id; markoeren kan navngi basen;
en umerket markoer med flere baser NEKTES (HypothesisParseError), med en base
resolveres den. Froe-approaches VALIDERES men skrives ALDRI om (§ C.6 doer 1 er
en bevaringsregel) - og valideringen skjer FOER foerste modellkall, samme
oekt-57-hoist-grunn: ved unntaket alene ser en nekt etter forbruket identisk ut.
explore()-docstringens gamle aerlighets-grense («venter paa at run_project tar
mer enn en bundle_dir») er RETTET - den ville vaert usann fra i dag.

Load-bearing MAALT, 12 mutasjoner alle roede mot HELE suiten, groenn kontroll
1020/5 (fra 999/5 + 21 nye tester); golden demo-transcript.stdout BYTE-UENDRET
(ea8c534773acdbe41ae68f2c55724d69aaf8be4f): detach myntet bundle_id (2) ·
stille gjennomfall ved >1 base (1) · ukjent id resolvert etter rekkefoelge (1) ·
froe-sjekk etter forbruket (2) · ruteren gjetter foerste base (1) · uroutbar
approach droppet (1) · dispatchen kollapser til en base (5) · project_id fra
foerste base (2) · detach aldri-startet-tannen (1) · unreached urapportert (1) ·
fersk store per base (1) · detach oppstartsnekten (2).

EN MUTASJON FALSIFISERTE TESTEN FOERST (repoets vakuoes-gate-klasse, aattende
gang): store-testen sammenlignet med ==, og VerdictStore er en pydantic-modell
med VERDI-likhet - tre ulike tomme stores er alle like, saa «fersk store per
base» lot hele suiten staa groenn. Delt INSTANS er paastanden, saa testen
asserterer naa paa `is`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3YHobQC3WzYVoxSgZus4d
This commit is contained in:
Kjell Tore Guttormsen 2026-08-25 13:35:39 +02:00
commit 18af86e422
6 changed files with 1051 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

183
src/portfolio_optimiser/run.py
View file

@ -70,8 +70,10 @@ from portfolio_optimiser.mandate import (
Approach,
ApproachOutcome,
Mandate,
MandateRoutingError,
announce,
load_mandate,
route_by_bundle,
settle,
)
from portfolio_optimiser.mcp_tools import (
@ -1343,6 +1345,187 @@ async def run_portfolio(
return base
@dataclass(frozen=True)
class BundleRun:
"""One knowledge base's run inside a multi-base dispatch (§ C.7).
``bundle_id`` and ``project_id`` are carried BESIDE the result rather than read back off it,
because they are what the dispatch ROUTED on: the id is how the mandate named the base, and the
project is what that base's own IR projection says it is. A reader pairing a coverage row back
to a base should not have to re-derive either.
"""
bundle_id: str
bundle_dir: str
project_id: str
result: RunResult
@dataclass(frozen=True)
class MultiBaseResult:
"""What one multi-base dispatch produced: one run per base the commission named (§ C.7).
A DISTINCT type from ``PortfolioResult``, and deliberately so. ``PortfolioResult`` keys on
"one ``RunResult`` per PROJECT in input order" and its ``runs``/``failures`` partition *the
projects that were actually submitted*; this keys on the KNOWLEDGE BASE the mandate routed each
approach to. Two bases may legitimately describe one project, which the project-keyed shape
cannot express at all — ``run_portfolio`` reads each base off ``projects[pid].bundle_dir``, so
a pid admits exactly one base there. Reusing the type would fuse two axes.
``unreached`` is the ``not_evaluated`` rule at the dispatch layer: when the global cap stops the
pass, every approach in a base that was never started is reported as unreached rather than
omitted. An omitted row is indistinguishable from an approach nobody commissioned, which is the
silence the coverage report exists to remove.
"""
runs: tuple[BundleRun, ...]
store: VerdictStore
stopped_early: bool = False
budget_stop: BudgetStop | None = None
unreached: tuple[ApproachOutcome, ...] = ()
async def run_mandate_across_bundles(
mandate: Mandate,
bundle_dirs: Sequence[str],
profile: Profile | str = Profile.LOCAL,
*,
verdict_input: dict[str, str],
store: VerdictStore | None = None,
verdict_dir: str | None = None,
dimension: Dimension | None = None,
client_factory: Callable[[str], BaseChatClient] | None = None,
max_rounds: int = _DEFAULT_MAX_ROUNDS,
max_tokens: int = _DEFAULT_MAX_TOKENS,
top_k: int = 3,
portfolio_meter: PortfolioMeter | None = None,
) -> MultiBaseResult:
"""Evaluate ONE commission across SEVERAL knowledge bases — the multi-base dispatch (§ C.7).
``mandate.route_by_bundle`` partitions the commission by ``Approach.bundle_id``, and this runs
the EXISTING ``run_project`` once per base with that base's own sub-mandate. Composition, never
a widening: ``run_project`` derives the project, the validator's stage-0 baseline, the agents'
read context and the ExpeL query key from THE bundle it is handed, and returns one stamped
``RunResult`` — a second ``bundle_dir`` on that signature would force a silent pick-one for all
four. § C.7 words it the same way: *pipelinen kjøres per bundle som i dag*.
**There is no ``project_id`` argument, and that is the design rather than an omission.** Each
base's project is read from THAT base's own IR projection (``okf.load_ir_projection``), which is
the same value ``_project_from_bundle`` already fail-fasts against — so a caller-supplied
constant could only ever be right for one base out of N. Turning the existing fail-fast into the
routing key removes the guess entirely.
ONE ``VerdictStore`` is threaded across every base, exactly as ``run_portfolio`` threads one
across every project: a verdict captured while evaluating base k must be able to reach base
k+1's hypothesis, and a fresh store per base would leave the loop looking wired while carrying
nothing between the runs.
**The budget has the two S3.4 teeth that apply here, and no more.** With a ``portfolio_meter``:
a remainder that cannot fund one run refuses at STARTUP (``BudgetRefused`` — a pass with room
for zero runs is a caller error, not a result), and a base that cannot be funded is NEVER
STARTED, the pass stopping with ``budget_stop`` and every unreached approach reported. Never
started is the whole point: a base merely aborted mid-run has already cost real model calls.
The wave-reservation tooth has no counterpart here — this dispatch is SEQUENTIAL, so there is
no wave of runs funded off one pre-wave remainder to over-commit.
Honesty limits, stated rather than implied. (1) A base that RAISES propagates; the
collect-and-continue policy belongs to ``run_portfolio``, where the caller submitted a batch of
independent projects, whereas here the caller asked for ONE commission to be evaluated.
(2) Without a ``portfolio_meter`` the pass's ceiling is the number of routed bases times
``max_tokens``, each run bounded on its own — the global ledger is opt-in, and this does not
re-implement it (``_run_meter`` is the one copy of the binding rule). (3) The outbox is NOT
wired: N runs need N ``run_id``s, and minting them here would default a key this repo requires
a caller to supply, for byte-determinism. A caller who needs artefacts per base calls
``run_project`` itself with the sub-mandates ``route_by_bundle`` hands back.
:raises MandateRoutingError: the commission cannot be routed against ``bundle_dirs``.
:raises BudgetRefused: a global remainder that cannot fund a single run.
"""
by_id: dict[str, str] = {}
for raw in bundle_dirs:
bundle_id = Path(raw).name
if bundle_id in by_id:
# The same refusal ``explore._bundle_index`` makes, for the same reason: the id is how
# the mandate names a base, so two bases answering to one name would let an approach be
# evaluated against A while the report says B (the S3.2 key-collision class).
raise MandateRoutingError(
f"two knowledge bases share the id {bundle_id!r} ({by_id[bundle_id]!r} and "
f"{raw!r}); an approach names a base by that id, so it must be unique"
)
by_id[bundle_id] = raw
routed = route_by_bundle(mandate, tuple(by_id))
if portfolio_meter is not None and not portfolio_meter.can_fund_run():
raise BudgetRefused(portfolio_meter.remaining(), portfolio_meter.required_per_run)
store = store if store is not None else VerdictStore(verdicts=[])
runs: list[BundleRun] = []
unreached: list[ApproachOutcome] = []
budget_stop: BudgetStop | None = None
for index, (bundle_id, sub_mandate) in enumerate(routed):
if portfolio_meter is not None and not portfolio_meter.can_fund_run():
budget_stop = BudgetStop(
limit_tokens=portfolio_meter.budget.max_total_tokens,
spent_tokens=portfolio_meter.spent,
remaining_tokens=portfolio_meter.remaining(),
required_tokens=portfolio_meter.required_per_run,
)
unreached.extend(
ApproachOutcome(
id=approach.id,
label=approach.label,
status="not_evaluated",
detail=(f"budget exhausted before knowledge base {pending_id!r} was run"),
)
for pending_id, pending in routed[index:]
for approach in pending.approaches
)
break
bundle_dir = by_id[bundle_id]
# ONE reading of the base's own project id, used both to ADDRESS the run and to LABEL it.
# A second lookup for the label would be the kø-(p) duplicate free to drift from the value
# the run was actually dispatched with.
project_id = str(okf.load_ir_projection(bundle_dir)["project_id"])
result = cast(
RunResult,
await run_project(
project_id,
profile,
docs_dir=bundle_dir,
bundle_dir=bundle_dir,
verdict_input=verdict_input,
verdict_dir=verdict_dir,
dimension=dimension,
store=store,
client_factory=client_factory,
max_rounds=max_rounds,
max_tokens=max_tokens,
top_k=top_k,
mandate=sub_mandate,
meter=_run_meter(None, portfolio_meter, max_rounds),
),
)
runs.append(
BundleRun(
bundle_id=bundle_id,
bundle_dir=bundle_dir,
project_id=project_id,
result=result,
)
)
return MultiBaseResult(
runs=tuple(runs),
store=store,
stopped_early=budget_stop is not None,
budget_stop=budget_stop,
unreached=tuple(unreached),
)
# The roles ``debate``/``generate`` ask the factory for. Fixed here so a malformed replies file is
# caught at the door instead of mid-run.
_SCRIPTED_ROLES = ("proposer", "checker")