portfolio-optimiser/.claude/projects/2026-06-26-fase3-portfolio-fanout/plan.md

Fase 3 — Sequential fan-out orchestrator over N portfolio projects

Plan quality: B+ (88/100) — APPROVE_WITH_NOTES

Generated by trekplan v5.6.1 on 2026-06-26 — plan_version: 1.7 Profile: premium · phase_signal plan = effort:high, model:opus Brief: .claude/projects/2026-06-26-fase3-portfolio-fanout/brief.md

Context

The framework today proves the method on ONE project (run_project, Fase 2 vertical slice). The core promise — a generic portfolio optimiser — is unproven until the framework runs over N independent projects with zero core-code change per project. Fase 3 generalises: a sequential run_portfolio orchestrator fans out over a configured portfolio, isolates each project's execution state (budget meter, debate, retrieval context) so there is no state-bleed, while threading one shared VerdictStore so the ExpeL learning loop composes across the portfolio (a verdict on project A informs a similar proposal on project C). Proving a brand-new synthetic project drops in via config only is the concrete falsification of the genericity claim (D4/D5) and unlocks Fase 4 (open-source) + D7 (Claude-SDK sibling). Motivation traces to the brief's Intent verbatim.

Architecture Diagram

graph TD
    subgraph "Fase 3 — fan-out (NEW + changed)"
        CFG["reference_projects.json<br/>+ docs_dir + verdict_input (NEW fields)"] --> LD["load_reference_projects<br/>(extended loader)"]
        LD --> RP["run_portfolio (NEW)<br/>sequential for-loop"]
        RP -->|"per project, await"| RJ["run_project (REUSE as-is<br/>+ additive meter= seam)"]
        RP -->|"ONE instance, threaded"| VS["VerdictStore<br/>(SHARED — ExpeL accumulates)"]
        RJ -->|"fresh per call"| TM["TokenMeter<br/>(ISOLATED per project)"]
        RJ -->|"retrieve before add"| VS
        RP --> PR["PortfolioResult (NEW)<br/>runs + store + thin aggregate"]
    end

Codebase Analysis

• Tech stack: Python ≥3.10, Microsoft Agent Framework (agent-framework-core 1.9.0), Pydantic IR, PuLP/CBC validator, uv, pytest (asyncio_mode=auto), ruff, mypy.
• Key patterns: layered/hexagonal composition root; every seam explicit (client_factory, store, ChatBackend protocol). Result types are @dataclass(frozen=True); the one mutable accumulator (VerdictStore) is a non-frozen @dataclass by design. from __future__ import annotations + full type hints + keyword-only (*,) options + absolute imports in every module. Tests mirror src module names; each opens with a docstring + Pattern: pointer; lettered SC names (test_a_…); guard-tests grep src for anti-patterns (test_budget.py:88-99). JSON config carries a _note provenance/disclaimer first key.
• The seam to compose — run_project (src/portfolio_optimiser/run.py:114-127): signature run_project(project_id, profile=LOCAL, *, docs_dir, verdict_input, store=None, client_factory=None, max_rounds=3, max_tokens=100_000, top_k=3, enable_layer1_hitl=False, notify=None) -> RunResult. All execution state is built fresh per call — TokenMeter (run.py:151), fresh_workflow (run.py:155), retrieval context (run.py:141) — so sequential reuse is inherently isolated. The ONE shared seam is store=: retrieve (run.py:189) happens before store.add (run.py:193), so project k+1 sees project k's verdict. RunResult (run.py:57-68, frozen): outcome (ValidatedProposal | Rejection, both expose .proposal), provenance (token_usage = meter.tokens, run.py:179), verdict, retrieved, store, debate_output.
• Learning store (verdicts.py): VerdictStore.add is idempotent on a content-hash id minted over (affected_codes, measure_type, claimed_saving_nok) — description excluded (_mint_id, verdicts.py:78-90). retrieve ranks by structural similarity (codes-Jaccard 0.60 + measure-match 0.25 + magnitude-bucket 0.15) with no similarity floor (verdicts.py:110-119). _features_of (run.py:98-104) maps proposal → features.
• Project loader (reference_domain.py:53-75): reads bundled data/reference_projects.json via importlib.resources.files. Project (reference_domain.py:38-51, frozen) has id/name/description/currency/cost_items — no docs_dir, no verdict_input. docs_dir is supplied per-call to run_project today (CLI --docs-dir); there is no project→docs mapping anywhere. This is the gap Step 2 closes config-side.
• Reusable code: _project_by_id (run.py:91), load_reference_projects, SyntheticUsageChatClient + make_client_factory/fresh_store/docs_dir fixtures (tests/conftest.py:27-114), the two-call accumulation precedent (test_vertical_slice_e2e.py:69-90), the decoy-vs-true-match precedent (test_verdicts.py), the load-bearing fence (test_h, test_vertical_slice_e2e.py:172-199), the gated-live skip (test_foundry_profile_live.py:19-24), the src-grep guard (test_budget.py:88-99).
• No external tech. MAF concurrency was spiked in Fase 1 and deliberately out of scope (sequential chosen). No research-scout needed.
• Baseline (verified this session): uv run pytest -q → 103 passed, 3 skipped (the 3 skips are env-gated live arms). mypy src + ruff check . clean.

Implementation Plan

Steps are TDD (test-first; the project has tests). Steps 1–2 are the authorized one-time core changes (the marginal per-project cost SC1 measures afterwards). Steps 3+ are additive. Forbidden across every step: src/portfolio_optimiser/validator.py and ir.py (the D7-portable pure core — never imported/edited by the orchestrator).

Step 1: Add additive meter= injection seam to run_project

• Files: src/portfolio_optimiser/run.py, tests/test_vertical_slice_e2e.py
• Changes: Add a keyword-only param meter: TokenMeter | None = None to run_project (after notify, run.py:126). Change the meter construction at run.py:151 from meter = TokenMeter(Budget(...)) to meter = meter if meter is not None else TokenMeter(Budget(max_tokens=max_tokens, max_rounds=max(max_rounds * 4, 4))). Behaviour is preserved when meter is None (the default) — every existing call is unaffected. This is the injection point SC3 needs to encode the isolation detach as an automatic guard (there is no meter seam today; without it the detach degrades to a manual reviewer-revert, the Fase 2 failure mode the brief forbids). Why a seam and not Step 5's seamless per-project-baseline fallback: the baseline assertion proves isolation holds now, but provides no automatic guard that reddens if a future edit makes run_portfolio share a meter. SC3 explicitly prefers the encoded detach ("inject en delt meter i orkestratoren, assert at isolasjons-testen da blir rød") — that requires an injection point. The param is additive, optional, and behaviour-preserving (None = today's code), consistent with the brief's authorization of one-time core changes for the Fase 3 build (cf. docs_dir).
• Reuses: TokenMeter/Budget (budget.py:40,57), already imported at run.py:35.
• Test first:
  • File: tests/test_vertical_slice_e2e.py (existing) — add test_i_injected_meter_is_used
  • Verifies: a TokenMeter pre-charged to a known value and passed as meter= is the meter used by the run — assert result.provenance.token_usage == pre_charged + run_delta (i.e. strictly greater than the same run with a fresh meter), proving the injected instance flows to provenance.token_usage (run.py:179). Detaches cleanly: revert the run.py:151 change and the injected meter is ignored → assertion fails.
  • Pattern: tests/test_vertical_slice_e2e.py:28-41 (test_a, run_project call shape)
• Verify: Run test_i_injected_meter_is_used BEFORE the run.py:151 change → expected: red (injected meter ignored). After the change: uv run pytest tests/test_vertical_slice_e2e.py -q → expected: all pass incl. the new test (Iron-Law red→green gate).
• On failure: revert — git checkout -- src/portfolio_optimiser/run.py tests/test_vertical_slice_e2e.py
• Checkpoint: git commit -m "feat(fase3): additive meter= seam on run_project (SC3 detach hook)"
• Manifest:

  manifest:
    expected_paths:
      - src/portfolio_optimiser/run.py
      - tests/test_vertical_slice_e2e.py
    min_file_count: 2
    commit_message_pattern: "^feat\\(fase3\\): additive meter= seam on run_project \\(SC3 detach hook\\)$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
      - src/portfolio_optimiser/ir.py
    must_contain:
      - path: src/portfolio_optimiser/run.py
        pattern: "meter: TokenMeter \\| None = None"
      - path: tests/test_vertical_slice_e2e.py
        pattern: "test_i_injected_meter_is_used"
  

Step 2: Add config-driven docs_dir + verdict_input to the project loader

• Files: src/portfolio_optimiser/reference_domain.py, src/portfolio_optimiser/data/reference_projects.json, src/portfolio_optimiser/data/docs/FV42-GSV-E1/notes.txt (new), src/portfolio_optimiser/data/docs/RV13-RAS-TP/notes.txt (new), src/portfolio_optimiser/data/docs/BRU-LAKS-REHAB/notes.txt (new), tests/test_reference_domain.py
• Changes:
  1. JSON: add to each of the 3 existing projects a "docs_dir" (relative, e.g. "docs/FV42-GSV-E1") and a "verdict_input" object {"decision","rationale"}. Mark the verdict_input values SYNTHETIC/AI-authored in the file-level _note (operator is not a domain expert — [[user-not-domain-expert]]; production replaces the reference domain with a real data source + real HITL — see Step 8 doc).
  2. Project dataclass (reference_domain.py:38-51): add fields docs_dir: str and verdict_input: dict[str, str].
  3. Loader (reference_domain.py:53-75): read p["docs_dir"] and resolve it to an absolute filesystem path against the package data root — str(files("portfolio_optimiser").joinpath("data", p["docs_dir"])) — and read p["verdict_input"] into the dataclass. Missing keys raise KeyError (fail-fast, matches existing loader contract).
  4. Create one small synthetic notes.txt per project (AI-authored, flagged) whose text contains the cost-saving query terms the orchestrator searches for (retrieve_chunks("cost saving measure", …), run.py:141) — e.g. a line naming a cost-saving measure on a cost code — so retrieve_chunks returns ≥1 chunk and run_project does not raise ValueError (run.py:143-144). Mirror the existing docs_dir fixture content style (conftest.py:110-113).
• Reuses: importlib.resources.files pattern already at reference_domain.py:55; the JSON _note provenance discipline (reference_projects.json:2, model_map.json:2).
• Test first:
  • File: tests/test_reference_domain.py (existing)
  • Verifies: every loaded Project has a non-empty docs_dir that points to an existing directory, and a verdict_input dict containing exactly decision + rationale keys.
  • Pattern: tests/test_reference_domain.py:8-38 (loader shape assertions)
• Verify: Run the extended loader assertions BEFORE the reference_domain.py change → expected: red (Project has no docs_dir/verdict_input). After: uv run pytest tests/test_reference_domain.py -q → expected: all pass (Iron-Law red→green gate).
• On failure: revert — git checkout -- src/portfolio_optimiser/reference_domain.py src/portfolio_optimiser/data/reference_projects.json tests/test_reference_domain.py and git clean -fd src/portfolio_optimiser/data/docs/
• Checkpoint: git commit -m "feat(fase3): config-driven docs_dir + verdict_input on Project loader"
• Manifest:

  manifest:
    expected_paths:
      - src/portfolio_optimiser/reference_domain.py
      - src/portfolio_optimiser/data/reference_projects.json
      - src/portfolio_optimiser/data/docs/FV42-GSV-E1/notes.txt
      - src/portfolio_optimiser/data/docs/RV13-RAS-TP/notes.txt
      - src/portfolio_optimiser/data/docs/BRU-LAKS-REHAB/notes.txt
      - tests/test_reference_domain.py
    min_file_count: 6
    commit_message_pattern: "^feat\\(fase3\\): config-driven docs_dir \\+ verdict_input on Project loader$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
      - src/portfolio_optimiser/ir.py
      - src/portfolio_optimiser/run.py
    must_contain:
      - path: src/portfolio_optimiser/reference_domain.py
        pattern: "docs_dir: str"
      - path: src/portfolio_optimiser/reference_domain.py
        pattern: "verdict_input: dict\\[str, str\\]"
      - path: src/portfolio_optimiser/data/reference_projects.json
        pattern: "verdict_input"
  

Step 3: Create PortfolioResult + run_portfolio orchestrator (SC2 fan-out)

• Files: src/portfolio_optimiser/run.py, src/portfolio_optimiser/__init__.py, tests/conftest.py, tests/test_portfolio.py (new)
• Changes:

  1. run.py: add @dataclass(frozen=True) class PortfolioResult mirroring RunResult style (run.py:57-68): fields runs: tuple[RunResult, ...], store: VerdictStore, validated_count: int, rejected_count: int, sum_claimed_saving_nok: float, sum_token_usage: int. Class docstring describing each field.

  2. run.py: add async def run_portfolio(project_ids: Sequence[str] | None = None, profile: Profile | str = Profile.LOCAL, *, store: VerdictStore | None = None, client_factory=None, max_rounds=3, max_tokens=100_000, top_k=3, meter_factory: Callable[[], TokenMeter] | None = None) -> PortfolioResult. Body: build {p.id: p for p in load_reference_projects()}; resolve ids (None → all loaded); create ONE store if not given; sequential for pid in ids: await run_project(pid, profile, docs_dir=project.docs_dir, verdict_input=project.verdict_input, store=store, client_factory=client_factory, max_rounds=..., max_tokens=..., top_k=..., meter=meter_factory() if meter_factory else None); collect RunResults; return _aggregate(runs, store). Unknown id → ValueError (reuse _project_by_id semantics). Import Sequence from collections.abc.

  3. run.py: add private _aggregate(runs, store) -> PortfolioResult — validated = [r for r in runs if isinstance(r.outcome, ValidatedProposal)]; sum_claimed = sum(r.outcome.proposal.claimed_saving_nok for r in validated); sum_tokens = sum(r.provenance.token_usage for r in runs).

  4. __init__.py: export PortfolioResult, run_portfolio (add to imports + __all__).

  5. tests/conftest.py: add make_portfolio_client_factory(replies: dict[str, str]) fixture → a project-aware SyntheticUsageChatClient subclass that selects its reply by scanning the incoming prompt for a known project_id substring (the prompt embeds project.id at run.py:162 and generate.py:48), falling back to a default. Keeps run_portfolio's single client_factory production-shaped while letting tests vary the proposal per project.

  6. Define the shared REPLIES fixture constant (used by Steps 3, 4, 7) in tests/test_portfolio.py. The synthetic reply IS the proposal — generate._parse_ir (generate.py:60-64) builds affected_items (each with its own quantity/unit_cost) straight from this JSON, and the validator's P90 = 0.30 × Σ(quantity·unit_cost) only when assumptions is empty (degenerate Monte Carlo, validator.py:108-113). All three replies therefore OMIT assumptions and carry explicit magnitudes so each claimed_saving_nok ≤ P90. Exact constants (verified):

     project_id	affected_items (code, qty, unit_cost)	Σ item totals	P90 (0.30·Σ)	claimed	validates?
     FV42-GSV-E1	(05.2, 4300, 215), (03.1, 1800, 310)	1,482,500	444,750	200,000	yes
     RV13-RAS-TP (decoy)	(88.2, 180, 4200)	756,000	226,800	130,000	yes
     BRU-LAKS-REHAB	(05.2, 4300, 215), (07.4, 2400, 690)	2,580,500	774,150	210,000	yes

     measure strings are byte-identical "Reduce scope" for FV42 + BRU (measure-match is exact string equality, verdicts.py:68) and "Material substitution" for the decoy. This makes validated_count==3, rejected_count==0, sum_claimed_saving_nok==540000 derivable IN this step (no forward dependency on Step 4), and sets up the Step-4 overlap (FV42 and BRU share code 05.2 + measure + magnitude bucket → similarity 0.60).

• Reuses: run_project (run.py:114, composed as-is), RunResult/ValidatedProposal/ Rejection, load_reference_projects, SyntheticUsageChatClient (conftest.py:27), the __init__.py:3,7 export pattern.
• Test first:
  • File: tests/test_portfolio.py (new) — test_a_fanout_returns_one_runresult_per_project
  • Verifies: run_portfolio(["FV42-GSV-E1","RV13-RAS-TP","BRU-LAKS-REHAB"], "local", store=fresh_store, client_factory=make_portfolio_client_factory(REPLIES)) → len(runs)==3, each isinstance(r, RunResult); aggregate pinned to the Step-3 fixture constants: validated_count==3, rejected_count==0, sum_claimed_saving_nok==540000; and sum_token_usage equals the explicit element-wise sum runs[0]+runs[1]+runs[2] token_usage (a real wiring check, not the field's own definition) with each run's token_usage > 0. Add test_a2_unknown_project_id_raises: with pytest.raises(ValueError): await run_portfolio(["NOPE-DOES-NOT-EXIST"], ...) (the unknown-id error path, run.py _project_by_id semantics).
  • Pattern: tests/test_vertical_slice_e2e.py:28-41 + module docstring/Pattern: convention
• Verify: Run the new tests BEFORE the run.py change → expected: red (ImportError / run_portfolio undefined) — the Iron-Law failing-test-first gate. Then after the change: uv run pytest tests/test_portfolio.py -q → expected: pass
• On failure: revert — git checkout -- src/portfolio_optimiser/run.py src/portfolio_optimiser/__init__.py tests/conftest.py and git clean -fd tests/test_portfolio.py
• Checkpoint: git commit -m "feat(fase3): run_portfolio sequential orchestrator + PortfolioResult aggregate"
• Manifest:

  manifest:
    expected_paths:
      - src/portfolio_optimiser/run.py
      - src/portfolio_optimiser/__init__.py
      - tests/conftest.py
      - tests/test_portfolio.py
    min_file_count: 4
    commit_message_pattern: "^feat\\(fase3\\): run_portfolio sequential orchestrator \\+ PortfolioResult aggregate$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
      - src/portfolio_optimiser/ir.py
    must_contain:
      - path: src/portfolio_optimiser/run.py
        pattern: "async def run_portfolio"
      - path: src/portfolio_optimiser/run.py
        pattern: "class PortfolioResult"
      - path: src/portfolio_optimiser/__init__.py
        pattern: "run_portfolio"
      - path: tests/test_portfolio.py
        pattern: "test_a_fanout_returns_one_runresult_per_project"
  

Step 4: Shared VerdictStore accumulates + load-bearing retrieval (SC4)

• Files: tests/test_portfolio.py
• Changes: Add test_b_shared_store_accumulates_and_surfaces_prior_verdict. Fan out over ["FV42-GSV-E1","RV13-RAS-TP","BRU-LAKS-REHAB"] reusing the Step-3 REPLIES (already specified with explicit magnitudes + empty assumptions — no re-specification here). The three proposals are deliberately distinct-but-overlapping and all validate, minting 3 distinct ids. similarity(BRU, FV42) = 0.60, decomposed as: codes-Jaccard({05.2,07.4}, {03.1,05.2}) = 1/3 → 0.60 × 0.333 = 0.20, measure-match ("Reduce scope" == "Reduce scope", exact equality) → 0.25, magnitude-bucket (both in [1e5,5e5)) → 0.15; total 0.60. similarity(BRU, decoy) = codes 0 + measure 0 + magnitude 0.15 = 0.15. So BRU's retrieval ranks FV42's verdict above the decoy.
• Reuses: the two-call precedent (test_vertical_slice_e2e.py:69-90); the decoy/true-match ranking idea (test_verdicts.py); _mint_id/similarity semantics (verdicts.py:65-90).
• Test first:
  • File: tests/test_portfolio.py
  • Verifies (load-bearing, not mere non-emptiness): after fan-out len(result.store.verdicts) == 3; the 3 verdict ids are pairwise distinct; result.runs[2].retrieved[0].id == result.runs[0].verdict.id (BRU surfaces FV42, the structural match, ranked above the decoy). If retrieval ranking breaks or proposals collide to one id, this fails — it asserts the specific match, not store-non-empty.
  • Pattern: tests/test_verdicts.py (true-match-beats-decoy assertion style)
• Verify: uv run pytest tests/test_portfolio.py::test_b_shared_store_accumulates_and_surfaces_prior_verdict -q → expected: pass
• On failure: revert — git checkout -- tests/test_portfolio.py
• Checkpoint: git commit -m "test(fase3): SC4 shared-store accumulation + load-bearing cross-project retrieval"
• Manifest:

  manifest:
    expected_paths:
      - tests/test_portfolio.py
    min_file_count: 1
    commit_message_pattern: "^test\\(fase3\\): SC4 shared-store accumulation \\+ load-bearing cross-project retrieval$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
      - src/portfolio_optimiser/verdicts.py
    must_contain:
      - path: tests/test_portfolio.py
        pattern: "test_b_shared_store_accumulates_and_surfaces_prior_verdict"
  

Step 5: Load-bearing state-isolation guard (SC3, meter detach)

• Files: tests/test_portfolio.py
• Changes: Add test_c_execution_state_isolation_is_load_bearing, a cap-independent automatic detach guard built on the Step-1 meter= seam + Step-3 meter_factory. The factory f here emits a uniform but valid JSON SavingsProposal (reuse REPLIES["FV42-GSV-E1"], the same string for both projects) — NOT a bare default like "ok": generate_via_llm loops re-fetching until _parse_ir succeeds, charging the meter each pass (generate.py:106-115), so an unparseable reply would run to BudgetExceeded instead of completing. Observable is per-project RunResult.provenance.token_usage (= meter.tokens, run.py:180):
  1. Baseline: standalone = await run_project("RV13-RAS-TP", ..., client_factory=f).
  2. Isolated (default): iso = await run_portfolio(["FV42-GSV-E1","RV13-RAS-TP"], ..., client_factory=f) (uniform reply) → assert iso.runs[1].provenance.token_usage == standalone.provenance.token_usage — project 1's usage is its OWN only, independent of project 0. This == is the load-bearing assertion: if run_portfolio shared a meter by default, runs[1] would be cumulative and this breaks.
  3. Shared (injected): shared = TokenMeter(Budget(...)); sh = await run_portfolio([...], ..., client_factory=f, meter_factory=lambda: shared) → assert sh.runs[1].provenance.token_usage > sh.runs[0].provenance.token_usage — sharing accumulates, proving the detach is real and the == arm would redden under sharing. Both arms run every CI → the detach is encoded automatically (no manual revert).
• Reuses: the load-bearing fence technique (test_h, test_vertical_slice_e2e.py:172-199); TokenMeter/Budget.
• Test first:
  • File: tests/test_portfolio.py
  • Verifies: the three assertions above (baseline equality under isolation; strict growth under sharing).
  • Pattern: tests/test_vertical_slice_e2e.py:172-199 (load-bearing detach)
• Verify: uv run pytest tests/test_portfolio.py::test_c_execution_state_isolation_is_load_bearing -q → expected: pass
• On failure: retry — if the isolated == does not hold because the two projects issue unequal chat-call counts, switch the isolation observable to a per-project standalone baseline for BOTH projects (iso.runs[i] == standalone_i); if still flaky, fall back to the cap-based variant (size max_tokens so 1 project < cap < 2 shared → pytest.raises(BudgetExceeded) only under sharing). Then revert on persistent failure: git checkout -- tests/test_portfolio.py
• Checkpoint: git commit -m "test(fase3): SC3 load-bearing meter-isolation detach guard"
• Manifest:

  manifest:
    expected_paths:
      - tests/test_portfolio.py
    min_file_count: 1
    commit_message_pattern: "^test\\(fase3\\): SC3 load-bearing meter-isolation detach guard$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
    must_contain:
      - path: tests/test_portfolio.py
        pattern: "test_c_execution_state_isolation_is_load_bearing"
  

Step 6: Both profiles exercised offline + gated Azure arm (SC7)

• Files: tests/test_portfolio.py, tests/test_portfolio_live.py (new)
• Changes:
  1. test_portfolio.py: add test_d_both_profiles_run_offline, parametrized @pytest.mark.parametrize("profile", ["local", "azure"]), running run_portfolio with the synthetic client_factory and asserting a PortfolioResult with len(runs)==N of RunResult for each profile (the contract path is actually executed offline, not just the backend instantiated). Plus a load-bearing teeth assertion that the profile reaches a profile-dependent seam offline: resolve_model(profile, "proposer") resolves to that profile's configured model from the bundled model_map.json and resolve_model("local", "proposer") != resolve_model("azure", "proposer") (deleting the azure model-map entry reddens it). Honest limitation documented: under an injected client_factory, profile is otherwise inert inside run_project (run.py:152,173 — model="fake-model", the F9 standing item); the orchestrator end-to-end run proves the path executes under both, and the resolve_model check is the only profile-dependent seam provable without egress.
  2. test_portfolio_live.py (new): a @pytest.mark.skipif-gated real-Azure portfolio run mirroring test_foundry_profile_live.py:14-24 — stays skipped offline (SC9 unchanged).
• Reuses: resolve_model (backends.py:54), the gated-live skip pattern (test_foundry_profile_live.py:19-24).
• Test first:
  • File: tests/test_portfolio.py + tests/test_portfolio_live.py
  • Verifies: both profiles produce a PortfolioResult offline; resolve_model distinguishes the profiles; the live arm is collected-but-skipped.
  • Pattern: tests/test_foundry_profile_live.py:14-24
• Verify: uv run pytest tests/test_portfolio.py -k both_profiles tests/test_portfolio_live.py -q → expected: offline params pass, live arm skipped
• On failure: revert — git checkout -- tests/test_portfolio.py and git clean -fd tests/test_portfolio_live.py
• Checkpoint: git commit -m "test(fase3): SC7 both profiles offline + gated Azure portfolio arm"
• Manifest:

  manifest:
    expected_paths:
      - tests/test_portfolio.py
      - tests/test_portfolio_live.py
    min_file_count: 2
    commit_message_pattern: "^test\\(fase3\\): SC7 both profiles offline \\+ gated Azure portfolio arm$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
    must_contain:
      - path: tests/test_portfolio.py
        pattern: "both_profiles"
      - path: tests/test_portfolio_live.py
        pattern: "skipif"
  

Step 7: New project via config only + no-code-mapping guard (SC1)

• Files: src/portfolio_optimiser/data/reference_projects.json, src/portfolio_optimiser/data/docs/SKOLE-VVS-OPPGR/notes.txt (new), tests/test_portfolio.py
• Changes:
  1. Append a 4th synthetic project (id e.g. SKOLE-VVS-OPPGR) to reference_projects.json with the full key set including docs_dir + verdict_input — AND its bundled docs dir. All domain data (codes/units/prices, the verdict decision/rationale) is AI-authored and flagged SYNTHETIC/ "ikke verifisert" in the _note discipline ([[user-not-domain-expert]]). No src/portfolio_optimiser/*.py change — this is the SC1 demonstration that adding a project is config + docs only.
  2. test_portfolio.py: test_e_new_project_flows_through_via_config_only — run run_portfolio(["SKOLE-VVS-OPPGR"], ...) and assert it produces a RunResult whose outcome.proposal.project_id == "SKOLE-VVS-OPPGR", proving config-only onboarding.
  3. test_portfolio.py: test_f_no_hardcoded_project_ids_in_src — a src-grep guard asserting no reference-project id literal (FV42-GSV-E1|RV13-RAS-TP|BRU-LAKS-REHAB|SKOLE-VVS-OPPGR) appears anywhere under src/portfolio_optimiser/ (ids live only in the data JSON). If a future project is added via a code-side id→path/verdict mapping, this guard reddens — load-bearing for SC1's "config-only" claim.
• Reuses: the _note provenance discipline; the src-grep guard idiom (test_budget.py:88-99).
• Test first:
  • File: tests/test_portfolio.py
  • Verifies: the new project runs end-to-end via config; no hardcoded ids in src.
  • Pattern: tests/test_budget.py:88-99 (src anti-pattern grep guard)
• Verify: uv run pytest tests/test_portfolio.py -k "new_project or no_hardcoded" -q → expected: pass; then git diff --name-only HEAD~1 shows only *.json, data/docs/**, tests/** (no src/portfolio_optimiser/*.py)
• On failure: revert — git checkout -- src/portfolio_optimiser/data/reference_projects.json tests/test_portfolio.py and git clean -fd src/portfolio_optimiser/data/docs/SKOLE-VVS-OPPGR
• Checkpoint: git commit -m "test(fase3): SC1 new project via config-only + no-hardcoded-id src guard"
• Manifest:

  manifest:
    expected_paths:
      - src/portfolio_optimiser/data/reference_projects.json
      - src/portfolio_optimiser/data/docs/SKOLE-VVS-OPPGR/notes.txt
      - tests/test_portfolio.py
    min_file_count: 3
    commit_message_pattern: "^test\\(fase3\\): SC1 new project via config-only \\+ no-hardcoded-id src guard$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/run.py
      - src/portfolio_optimiser/reference_domain.py
      - src/portfolio_optimiser/validator.py
      - src/portfolio_optimiser/ir.py
    must_contain:
      - path: tests/test_portfolio.py
        pattern: "test_f_no_hardcoded_project_ids_in_src"
      - path: src/portfolio_optimiser/data/reference_projects.json
        pattern: "SKOLE-VVS-OPPGR"
  

Step 8: Extension documentation (SC6)

• Files: docs/extending.md (new), tests/test_portfolio.py
• Changes:
  1. docs/extending.md — sections "Legg til eget prosjekt", "Legg til egen datakilde", "Legg til egen modell-map", each pointing at the concrete config seams: reference_projects.json (the per-project docs_dir + verdict_input fields, the _note SYNTHETIC discipline), the bundled data/docs/<id>/ docs source, and model_map.json (role→deployment). State plainly that the bundled reference domain + its verdict_input values are SYNTHETIC fixtures and that a real deployer replaces the data source and supplies Layer-2 verdicts via real HITL (not static config) — keeps the architecture honest.
  2. test_portfolio.py: test_g_extension_doc_exists_and_references_seams — assert docs/extending.md exists and contains reference_projects.json, docs_dir, verdict_input, and model_map.json.
• Reuses: the file-existence + content guard idiom.
• Test first:
  • File: tests/test_portfolio.py
  • Verifies: doc exists and names the config seams.
  • Pattern: tests/test_budget.py:88-99 (file-content assertion)
• Verify: uv run pytest tests/test_portfolio.py -k extension_doc -q → expected: pass
• On failure: revert — git checkout -- tests/test_portfolio.py and git clean -fd docs/extending.md
• Checkpoint: git commit -m "docs(fase3): extension-point guide (add project / data source / model-map) + SC6 test"
• Manifest:

  manifest:
    expected_paths:
      - docs/extending.md
      - tests/test_portfolio.py
    min_file_count: 2
    commit_message_pattern: "^docs\\(fase3\\): extension-point guide \\(add project / data source / model-map\\) \\+ SC6 test$"
    bash_syntax_check: []
    forbidden_paths:
      - src/portfolio_optimiser/validator.py
    must_contain:
      - path: docs/extending.md
        pattern: "reference_projects.json"
      - path: tests/test_portfolio.py
        pattern: "test_g_extension_doc_exists_and_references_seams"
  

Alternatives Considered

Approach	Pros	Cons	Why rejected
Per-project client_factory arg on run_portfolio (test seam)	Explicit; no prompt-scanning	Adds test-only API surface; production always uses one factory; diverges from run_project's single-factory seam	Project-aware synthetic client keeps run_portfolio production-shaped (one client_factory), pushing per-project reply logic into the test double only
Per-project isolated VerdictStore	Simpler isolation story	Kills cross-project ExpeL learning — the whole point of Fase 3	Explicit Non-Goal; the brief deliberately shares one store
asyncio.gather concurrent fan-out	Faster	Races the shared store's retrieve-before-add ordering; interleaves debates; needs the meter/store made concurrency-safe	Explicit Non-Goal (sequential only; concurrency is a documented extension point, 90%-principle)
Hardcoded {project_id: docs_dir} map in orchestrator code	No JSON change	Adding project N+1 edits src/*.py → breaks SC1	Config-driven docs_dir on the loader is the authorized one-time change that makes N+1 config-only
Make profile load-bearing by changing run.py:173 (model="fake-model" → resolve_model)	Would make SC7 fully load-bearing end-to-end	Touches a Fase 2 seam (F9) — scope creep beyond the brief	Out of scope; SC7 teeth come from a direct resolve_model assertion instead, F9 left as the standing item

Test Strategy

• Framework: pytest + pytest-asyncio (asyncio_mode=auto — bare async def test_*).
• Existing patterns: synthetic offline client (conftest.py:27), shared-store two-call accumulation (test_vertical_slice_e2e.py:69-90), load-bearing fence (test_h), gated-live skip (test_foundry_profile_live.py), src-grep guard (test_budget.py:88-99), decoy-vs-match ranking (test_verdicts.py).
• New tests in this plan: 1 in test_vertical_slice_e2e.py (meter seam) + ~7 in tests/test_portfolio.py (SC2,4,3,7,1×2,6) + 1 in test_portfolio_live.py (gated) + loader assertions in test_reference_domain.py.

Tests to write

Type	File	Verifies	Model test
Unit	tests/test_vertical_slice_e2e.py	injected meter= is used (Step 1)	test_a (:28)
Loader	tests/test_reference_domain.py	docs_dir+verdict_input resolved per project	test_reference_domain.py:8
Integration	tests/test_portfolio.py	SC2 fan-out + aggregate sums	test_vertical_slice_e2e.py:28
Integration	tests/test_portfolio.py	SC4 accumulation + load-bearing retrieval	test_verdicts.py
Load-bearing	tests/test_portfolio.py	SC3 meter-isolation detach (auto guard)	test_h (:172)
Parametrized	tests/test_portfolio.py	SC7 both profiles offline + resolve_model teeth	test_foundry_profile_live.py
Integration+Guard	tests/test_portfolio.py	SC1 config-only project + no-hardcoded-id src grep	test_budget.py:88
Guard	tests/test_portfolio.py	SC6 extension doc exists + references seams	test_budget.py:88
Gated live	tests/test_portfolio_live.py	real-Azure portfolio arm (skipped offline)	test_foundry_profile_live.py:19

Risks and Mitigations

Priority	Risk	Location	Impact	Mitigation
Critical	SC4 collapses to len==1 if projects mint identical ids	verdicts.py:121-124,78-90	green-but-dead	Project-aware factory yields 3 structurally distinct proposals; assert 3 distinct ids, not just len==3 (Step 4)
Critical	SC4 "surfaces prior" passes trivially — retrieve has no similarity floor	verdicts.py:110-119	zero-overlap projects pass	Assert runs[2].retrieved[0].id == runs[0].verdict.id (specific match beats the RV13 decoy), not non-emptiness (Step 4)
High	SC3 has no meter-injection point today	run.py:151	detach degrades to manual revert	Add additive meter= seam (Step 1) + meter_factory (Step 3); both detach arms run every CI
High	SC3 == arm flaky if the two projects issue unequal chat-call counts	run.py:151-169	false red/green	Cap-independent baseline: iso.runs[1] == standalone(project_1); documented fallbacks in Step 5 On-failure (per-project baseline, or cap-based BudgetExceeded)
High	SC7 profile inert under injected client (model="fake-model")	run.py:152,173 (F9)	parametrize proves nothing	Add direct resolve_model(profile,…) teeth + document the limitation honestly; do not silently claim full profile coverage (Step 6)
High	verdict_input hidden non-config dependency breaks SC1	run.py:119	adding N+1 edits core	Make verdict_input a per-project config field resolved by the loader (Step 2), same home as docs_dir
Medium	Bundled-docs path resolution via importlib.resources is source/editable-install dependent	reference_domain.py:55	a future zip-wheel install can't open() a Traversable dir	Acceptable for the dev+test context (90%); note as Assumption; if packaged, switch to importlib.resources.as_file
Medium	Partial-failure mid-fan-out aborts the sequential loop with a half-accumulated store	run.py:130-131,193	ambiguous portfolio result	v1 policy = fail-fast (a raising project aborts; consistent with the offline synthetic suite where all validate); collect-and-continue is a documented future extension
Low	Magnitude-bucket boundary flips drop the magnitude similarity component	verdicts.py:29,53-57	fixture overlap not as intended	All three SC4 savings (130k/200k/210k) sit inside bucket [1e5,5e5) interior — verified numerically (Step 4)

Assumptions

#	Assumption	Why unverifiable	Impact if wrong
1	The two/three offline projects issue equal synthetic chat-call counts, so the SC3 == isolation arm holds	Depends on fresh_workflow round dynamics under the synthetic client; not statically determinable	SC3 isolation arm needs the documented fallback (per-project baseline or cap-based variant) — Step 5 already carries it
2	importlib.resources.files(...).joinpath("data", docs_dir) yields an os.PathLike real path under the uv editable/source install	files() returns a Traversable; concrete behavior differs for zip-imported packages	A zip-wheel install would need as_file; harmless in the current dev/test context
3	Putting synthetic verdict_input (decision/rationale) in reference_projects.json is the right config home (vs a separate portfolio-config)	Brief leaves it open	If a separate portfolio-config is later preferred, the loader field moves; SC1 still holds as long as it stays config-driven
4	Static config verdict_input faithfully stands in for Layer-2 HITL in the offline synthetic framework	It is a fixture convenience, not the production HITL path	Documented in Step 8 doc as synthetic — production supplies verdicts via real HITL, not config

Verification

Per-step manifests are checked automatically by trekexecute. These are the end-to-end gates.

• uv run pytest -q → expected: (103 + new) passed, 3 skipped — the 3 existing live skips preserved, zero new failures (SC5; baseline re-read at execute-start, not hardcoded).
• uv run mypy src → expected: clean (SC5).
• uv run ruff check . → expected: clean; uv run ruff format . → no diff (SC5).
• uv run pytest tests/test_portfolio.py -q → all SC2/SC3/SC4/SC7/SC1/SC6 tests pass.
• SC1 git-diff proof — for the Step-7 commit: git show --name-only --format= HEAD (the SC1 commit) lists ONLY *.json, data/docs/**, and tests/** — no src/portfolio_optimiser/*.py.
• SC3 detach is genuinely load-bearing — temporarily make run_portfolio share one meter across projects; test_c_execution_state_isolation_is_load_bearing MUST go red; revert.
• SC4 is genuinely load-bearing — temporarily give two projects identical proposals; the distinct-id assertion MUST go red; revert.

Estimated Scope

• Files to modify: 7 — run.py, reference_domain.py, reference_projects.json, __init__.py, conftest.py, test_reference_domain.py, test_vertical_slice_e2e.py.
• Files to create: ~9 — tests/test_portfolio.py, tests/test_portfolio_live.py, docs/extending.md, and 4 bundled data/docs/<id>/notes.txt (+ the SC1 new project's docs).
• Complexity: medium — additive orchestration over a proven seam; the difficulty is in the load-bearing test design (SC3/SC4), which the exploration de-risked with concrete fixtures.

Execution Strategy

8 steps. Dependencies make Steps 1–3 a foundation wave; Steps 4–8 are additive and largely independent (they extend tests/test_portfolio.py + sibling files). A single sequential /trekexecute session is appropriate (one author, clean tree); the grouping below is the dependency map, not a mandate to parallelise.

Session 1: Core seams

• Steps: 1, 2
• Wave: 1
• Depends on: none
• Scope fence:
  • Touch: run.py (meter seam only), reference_domain.py, reference_projects.json, data/docs/**, test_vertical_slice_e2e.py, test_reference_domain.py
  • Never touch: validator.py, ir.py, verdicts.py

Session 2: Orchestrator

• Steps: 3
• Wave: 2
• Depends on: Session 1 (loader provides docs_dir/verdict_input)
• Scope fence:
  • Touch: run.py (PortfolioResult + run_portfolio), __init__.py, conftest.py, test_portfolio.py
  • Never touch: validator.py, ir.py

Session 3: Success-criteria tests + docs

• Steps: 4, 5, 6, 7, 8
• Wave: 3
• Depends on: Session 2 (run_portfolio exists)
• Scope fence:
  • Touch: tests/test_portfolio.py, tests/test_portfolio_live.py, docs/extending.md, reference_projects.json (SC1 new project), data/docs/SKOLE-VVS-OPPGR/**
  • Never touch: src/portfolio_optimiser/*.py (SC1 demands config-only) except none

Execution Order

• Wave 1: Session 1
• Wave 2: Session 2 (after Wave 1)
• Wave 3: Session 3 (after Wave 2)

Grouping rules applied

• Steps sharing files → same session (Steps 4–8 all extend test_portfolio.py).
• Core changes isolated to Waves 1–2 so Wave 3 can assert config-only (SC1).
• Sessions ordered strictly by dependency.

Plan Quality Score

Dimension	Weight	Score	Notes
Structural integrity	0.15	92	Steps dependency-ordered; foundation seams before consumers
Step quality	0.20	90	Each step TDD, single focused change, concrete fixtures + commands
Coverage completeness	0.20	92	All 7 SCs mapped to steps; brief Open Questions (docs_dir, verdict_input, SC4 fixture, aggregate) resolved
Specification quality	0.15	88	Exact file:line, computed P90/similarity math, no placeholders
Risk & pre-mortem	0.15	90	The 5 green-but-dead traps surfaced by exploration each have a mitigation + a documented fallback
Headless readiness	0.10	85	On-failure + Checkpoint per step; SC3 carries an explicit fallback path
Manifest quality	0.05	80	All steps have checkable manifests with forbidden_paths fencing the pure core
Weighted total	1.00	88	Grade: B+

Adversarial review:

• Plan critic: APPROVE_WITH_NOTES — 0 blockers, 2 major, 9 minor (score 81/B). Both majors (SC4 fixture magnitudes unspecified + Step 3↔4 forward coupling) addressed by moving the concrete REPLIES table into Step 3. Minors folded in (red-phase gates, JSON uniform reply, error-path test, similarity wording, notes.txt relevance, file count, tautology).
• Scope guardian: ALIGNED — 0 creep, 0 gaps. All 7 SCs covered; the 3 flagged items (meter=, resolve_model teeth, config verdict_input) each judged in-scope; 4 over-reaches correctly rejected in Alternatives.

Revisions

Added by adversarial review (Phase 9).

#	Finding	Severity	Resolution
1	SC4/SC2 proposal magnitudes (quantity·unit_cost → P90) never specified; identity assumes empty assumptions	major	Step 3 now carries the explicit REPLIES table (item magnitudes, Σ totals, P90, claimed) + states empty assumptions; all three validate by construction
2	Step 3 540000/validated_count==3 depended on a fixture deferred to Step 4 (forward coupling)	major	REPLIES constant defined in Step 3; Step 4 now reuses it (no re-specification)
3	Step 5 "uniform reply" could be a bare non-JSON string → runs to BudgetExceeded	minor	Step 5 now reuses REPLIES["FV42-GSV-E1"] (valid JSON) for f
4	Step 3 sum_token_usage assertion was tautological (field's own definition)	minor	Re-stated as an explicit element-wise sum wiring check + each run >0
5	meter=/meter_factory production surface not justified vs the seamless fallback	minor	Step 1 now states why the encoded detach (SC3-preferred) needs the injection point
6	Steps 1–3 lacked an explicit TDD red-phase gate (Iron Law)	minor	Steps 1–3 Verify now require the test to be red before the production change
7	notes.txt relevance to retrieve_chunks("cost saving measure") not tied	minor	Step 2 now requires the notes text to contain the cost-saving query terms
8	run_portfolio unknown-id / error paths untested	minor	Step 3 adds test_a2_unknown_project_id_raises (pytest.raises(ValueError))
9	similarity(BRU,FV42) parenthetical overstated the codes contribution; measure equality is exact	minor	Step 4 now shows the full 0.20+0.25+0.15 decomposition + byte-identical measure requirement
10	Estimated Scope "6 files to modify" but listed 7	minor	Corrected to 7
11	SC7 azure offline arm near-inert under injected client	minor (acknowledged)	Already documented honestly in Step 6 + Risks; resolve_model teeth retained; no change

Adversarial Pass 2 (gemini-bridge, v5.1.1 high-effort)

Status: attempted, unavailable — no independent findings. The high-effort plan phase (phase_signal: plan effort=high) prescribes an additional independent gemini-bridge review of the post-revision plan. The pass was launched but the Gemini Deep Research API returned a deterministic failure: its legacy Interactions API schema was deprecated (May 2026) and the local gemini-mcp client SDK predates the replacement. This is an operator-side tooling issue, not a plan defect, and not retryable as-is.

Decision: proceed without the third cross-check. The plan already cleared two independent adversarial reviewers (plan-critic APPROVE_WITH_NOTES, scope-guardian ALIGNED) with all substantive findings consumed; the gemini pass was additive triangulation, not a gate. Operator action if desired: upgrade gemini-mcp per the May-2026 breaking-change notice, then this pass can be re-run for an independent second opinion before /trekexecute.