feat(run): every commissioned approach is evaluated, and every one is reported

Carrying an approach into the prompt only half-answers krav 1. The expert asked
for their approaches to be CONCRETELY EVALUATED, which means each must reach a
verdict and each verdict must be visible. `run_project(mandate=...)` evaluates
every commissioned approach in turn — the run's own proposal last, when allowed —
each under the SAME meter. No new loop: the caps already in force are the bound.

`RunResult.coverage` is the settlement, one row per approach: validated (with the
figure), rejected (with the validator's reason verbatim), or not_evaluated (with
why). `not_evaluated` is the row that earns the type its keep — an approach the
run never reached must be reported as unreached, because an omitted row is
indistinguishable from an approach nobody ordered. That silence is the defect
class krav 1 is asking us to remove.

Budget exhaustion mid-list is reported, not swallowed. But if the FIRST approach
exhausts it there is nothing honest to return, so BudgetExceeded propagates
exactly as before — a run that produced nothing must still fail loudly.

RunResult stays single-outcome (portfolio aggregation, outbox artefacts and HITL
keying all rest on that). The choice is deterministic: highest validated saving,
ties by mandate order — never whichever ran last.

Load-bearing MEASURED against the whole 702-test suite. FIVE mutations red:
evaluate only the first approach (5 red) · drop the rejected rows (3) · ignore
allow_own_proposals (1) · select produced[-1] (1) · select produced[0] (1).

The sixth measurement is why this commit exists in this shape: the ordering
mutation FIRST STAYED GREEN. The test had placed the bigger approach last, where
"highest saving" and "whichever ran last" give the same answer, so an
order-dependent implementation passed it. A scenario that cannot separate two
implementations proves nothing about either — the test now pins BOTH orderings,
and each mutation direction fails one of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULCqjLF61rehj5cZmdUoR3

src/portfolio_optimiser/run.py

@@ -27,7 +27,7 @@ from __future__ import annotations
 
 import asyncio
 import json
-from collections.abc import Callable, Iterable, Sequence
+from collections.abc import Awaitable, Callable, Iterable, Sequence
 from dataclasses import dataclass, replace
 from pathlib import Path
 from typing import Any, Literal, cast
@@ -38,6 +38,7 @@ from pydantic import ValidationError
 from portfolio_optimiser.backends import Profile, get_backend, resolve_model
 from portfolio_optimiser.budget import (
     Budget,
+    BudgetExceeded,
     BudgetMiddleware,
     BudgetRefused,
     PortfolioMeter,
@@ -54,6 +55,12 @@ from portfolio_optimiser.datasource import (
 from portfolio_optimiser.dimension import Dimension, admits, load_dimension
 from portfolio_optimiser.generate import generate_via_llm
 from portfolio_optimiser.ir import SavingsProposal
+from portfolio_optimiser.mandate import (
+    OWN_PROPOSAL_ID,
+    Approach,
+    ApproachOutcome,
+    Mandate,
+)
 from portfolio_optimiser.provenance import ProvenanceStamp
 from portfolio_optimiser.reference_domain import Project, load_reference_projects
 from portfolio_optimiser.validator import Rejection, ValidatedProposal, baseline_from_project
@@ -100,6 +107,10 @@ class RunResult:
     store: VerdictStore
     debate_output: str
     checker_verdict: str = "absent"
+    #: One row per commissioned approach (Trekk A4). EMPTY when the run had no mandate — an empty
+    #: report is honest there, because nothing was ordered. It defaults so every existing
+    #: constructor call and every frozen aggregate over ``RunResult`` is unaffected.
+    coverage: tuple[ApproachOutcome, ...] = ()
 
 
 @dataclass(frozen=True)
@@ -199,6 +210,79 @@ class PortfolioResult:
     budget_stop: BudgetStop | None = None
 
 
+def _coverage_row(
+    row_id: str, label: str, outcome: ValidatedProposal | Rejection
+) -> ApproachOutcome:
+    """One coverage row from one evaluated approach. A rejection carries the validator's reason
+    verbatim — a bare status would tell the expert their approach failed without telling them why,
+    which is the part they can actually act on."""
+    if isinstance(outcome, ValidatedProposal):
+        return ApproachOutcome(
+            id=row_id,
+            label=label,
+            status="validated",
+            saving_nok=outcome.proposal.claimed_saving_nok,
+        )
+    return ApproachOutcome(id=row_id, label=label, status="rejected", detail=outcome.reason)
+
+
+def _select_outcome(
+    produced: list[tuple[int, ValidatedProposal | Rejection]],
+) -> ValidatedProposal | Rejection:
+    """Pick the ONE outcome a ``RunResult`` carries out of everything the mandate produced.
+
+    ``RunResult`` stays single-outcome on purpose: portfolio aggregation, the outbox artefacts and
+    the HITL verdict keying all rest on there being exactly one. The choice is deterministic —
+    highest validated saving, ties broken by mandate order — so it can never depend on which
+    approach happened to run last. When nothing validated, the FIRST rejection stands, which keeps
+    a fully-rejected mandate reporting a typed ``Rejection`` rather than a fabricated success.
+    """
+    validated = [(i, o) for i, o in produced if isinstance(o, ValidatedProposal)]
+    if validated:
+        return min(validated, key=lambda t: (-t[1].proposal.claimed_saving_nok, t[0]))[1]
+    return produced[0][1]
+
+
+async def _evaluate_mandate(
+    mandate: Mandate,
+    evaluate: Callable[[Approach | None], Awaitable[ValidatedProposal | Rejection]],
+) -> tuple[ValidatedProposal | Rejection, tuple[ApproachOutcome, ...]]:
+    """Evaluate every commissioned approach, then the run's own proposal when allowed, and report
+    what became of each (Trekk A3/A4).
+
+    Budget exhaustion mid-list is REPORTED, not swallowed: the approaches that were never reached
+    become ``not_evaluated`` rows. But if the very first approach exhausts the budget there is
+    nothing honest to return, so ``BudgetExceeded`` propagates exactly as it did before — a run
+    that produced nothing must still fail loudly rather than hand back an empty report.
+    """
+    plan: list[tuple[str, str, Approach | None]] = [(a.id, a.label, a) for a in mandate.approaches]
+    if mandate.allow_own_proposals:
+        plan.append((OWN_PROPOSAL_ID, "the system's own proposal", None))
+
+    rows: list[ApproachOutcome] = []
+    produced: list[tuple[int, ValidatedProposal | Rejection]] = []
+    for index, (row_id, label, approach) in enumerate(plan):
+        try:
+            outcome = await evaluate(approach)
+        except BudgetExceeded:
+            if not produced:
+                raise
+            rows.extend(
+                ApproachOutcome(
+                    id=rid,
+                    label=lbl,
+                    status="not_evaluated",
+                    detail="budget exhausted before this approach was evaluated",
+                )
+                for rid, lbl, _ in plan[index:]
+            )
+            break
+        produced.append((index, outcome))
+        rows.append(_coverage_row(row_id, label, outcome))
+
+    return _select_outcome(produced), tuple(rows)
+
+
 def _authored_texts(result: Any, name: str) -> list[str]:
     """The texts of ``get_outputs()`` entries authored by participant ``name`` (proposer/checker),
     in surfaced order. MAF surfaces ``author_name`` on each output's ``messages`` — NOT on the
@@ -317,6 +401,7 @@ async def run_project(
     live_dry_run: bool = False,
     semantic_retrieval: bool = False,
     embedder: Embedder | None = None,
+    mandate: Mandate | None = None,
 ) -> RunResult | DryRunReport:
     """Run the vertical slice for ONE project. ``client_factory`` is the test-injection seam
     (defaults to the real backend). ``verdict_input`` carries the expert decision/rationale
@@ -487,11 +572,22 @@ async def run_project(
         ).format_fewshot()
         gen_context = f"{fewshot}\n\n{gen_context}"
 
-    # 5. Structured candidate -> blocking validation on the NUMBERS; token bound = the meter.
+    # 5. Structured candidate(s) -> blocking validation on the NUMBERS; token bound = the meter.
+    # Without a mandate this is the single pre-Trekk-A call, unchanged. With one, every
+    # commissioned approach is evaluated in turn (and the run's own proposal last, when allowed),
+    # each under the SAME meter — no new unbounded loop; the caps already in force are the bound.
     proposer_client = factory("proposer")
-    validator_outcome = await generate_via_llm(
-        proposer_client, project, gen_context, meter, baseline=baseline
-    )
+
+    async def _evaluate(approach: Approach | None) -> ValidatedProposal | Rejection:
+        return await generate_via_llm(
+            proposer_client, project, gen_context, meter, baseline=baseline, approach=approach
+        )
+
+    coverage: tuple[ApproachOutcome, ...] = ()
+    if mandate is None:
+        validator_outcome = await _evaluate(None)
+    else:
+        validator_outcome, coverage = await _evaluate_mandate(mandate, _evaluate)
     proposal = validator_outcome.proposal
 
     # 6. First-class provenance stamp (authoritative; independent of MAF Annotation).
@@ -579,6 +675,7 @@ async def run_project(
         store=store,
         debate_output=debate_output,
         checker_verdict=checker_decision,
+        coverage=coverage,
     )