14-step plan composing the four Fase 1 spikes into a src/ vertical slice (debate -> blocking validator -> two-layer HITL + provenance -> ExpeL learning), deterministic-core-first, on real chat clients in both profiles. Grounded in the 3 research briefs + installed-1.9.0-source introspection (7 exploration agents). Adversarial review: plan-critic REVISE (3 blockers/7 major/4 minor) -> all addressed; scope-guardian ALIGNED (0 creep, 9/9 criteria mapped, 6/6 Non-Goals honored). Key revisions: in-process retriever-as-tool MVP path (mcp dep conditional/GA-only); two-layer HITL capture + stable Verdict.id minting; extend_instructions retired by a REAL SessionContext test; TextSpan ownership + wave re-ordering; budget None-as-hard-fail with synthetic-usage test double; self_repair token-bound in the generate loop (validator.py frozen); global stop-on-failure rule. gemini-bridge Pass 2 unavailable (MCP SDK broke). plan-validator strict 0 errors. brief research_status pending to complete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fif1r1En5W542HbZV88yMH
53 KiB
Fase 2 — MVP Vertical Slice: one synthetic project end-to-end on MAF 1.9.0
Plan quality: A− (88/100) — APPROVE_WITH_NOTES (revised after adversarial review)
Generated by trekplan v2.0 on 2026-06-24 —
plan_version: 1.7
Context
Fase 1 proved the four riskiest assumptions in isolation with throwaway spikes (maker-checker convergence, fan-out state-bleed, the hybrid blocking validator, ExpeL retrieval) and produced a verified MAF 1.9.0 capability map. Fase 2 builds the first thing that is actually a system: ONE synthetic "anleggskostnad" project flowing through the whole method — candidate measures debated (Group Chat maker-checker), a deterministic validator deciding the value, a domain expert judging via HITL, and the judgment fed back so the next run is smarter — on real (not faked) chat clients, in both profiles. The danger being bought down is integration risk: each spike worked alone; the open question is whether they compose into one fail-fast, budget-bounded, provenance-stamped, learning loop. This slice is the MVP backbone every later phase builds on (Fase 3 fan-out, Fase 4 open-source, the D7 Claude-SDK sibling), so getting the seams right matters more than breadth (brief §Intent).
Three research briefs (installed-1.9.0-source-verified) correct the brief's earlier leanings and are binding on this plan:
- HITL (research 01): capture the durable learned verdict out-of-band in the
VerdictStore; use the GA in-run gate (
GroupChatBuilder.with_request_info) only for optional synchronous Layer-1 review; defer checkpointing off the MVP path. - Data access (research 02): the official
@modelcontextprotocol/server-filesystemcannot carry citations → the data source is a framework-agnostic in-process retriever returning{file, locator, snippet, score}(the D7-portable seam), exposed to the agents as a GA tool/ContextProvider; provenance is first-class Pydantic data, not MAF'sAnnotation(open streaming-drop bug #4316). - Local client (research 03): use
OpenAIChatCompletionClient(base_url=…)non-streaming (NOTOpenAIChatClient/Responses);UsageDetailsis populated None-safely non-streaming; the blocking validator becomes the retry/repair reliability mechanism for weak local models.
Architecture Diagram
graph TD
subgraph "src/portfolio_optimiser/ — Fase 2 vertical slice"
RUN[run.py orchestrator<br/>single-command entry] --> CONTRACTS[contracts.py<br/>fail-fast loaders]
RUN --> WF[workflow.py<br/>fresh_workflow + GroupChat maker-checker]
RUN --> CAP[verdicts.py<br/>capture_verdict Layer-2 + VerdictStore + ExpeL]
WF --> BK[backends.py<br/>LocalBackend OpenAIChatCompletionClient / FoundryChatClient + model-map]
WF --> BUD[budget.py<br/>TokenMeter + ChatMiddleware off UsageDetails]
WF --> TOOL[retrieval.py exposed as GA tool/ContextProvider]
TOOL --> RET[retrieval.py<br/>in-process retriever core + path-security - D7 seam]
WF --> GEN[generate.py<br/>LLM->IR + validator-as-retry]
GEN --> VAL[validator.py + ir.py<br/>blocking: IR->CBC->Monte-Carlo]
VAL --> PROV[provenance.py<br/>first-class Pydantic stamp]
PROV --> CAP
CAP -.next run.-> WF
end
REF[reference_domain.py + data/*.json<br/>Fase 0, settled] --> RUN
Codebase Analysis
- Tech stack: Python ≥3.10,
uv, hatchling,ruff(line-length 100),mypy,pytest+pytest-asyncio(asyncio_mode="auto"). Installed (ground-truth-verified):agent-framework-core1.9.0,-openai1.8.2,-foundry1.8.2,-orchestrations1.0.0; underlyingopenaiSDK 2.43.0;pulp3.3.2;pydantic2.13.4. CBC binary present (.venv/.../pulp/solverdir/cbc/osx/i64/cbc).mcpSDK is NOT installed (governs Step 7's build-vs-defer decision). - Key patterns: backend profiles = Strategy+Factory (
backends.py:26-84,ChatBackend@runtime_checkableProtocol +Profileenum +get_backend(), fail-fastValueError); validator = staged pipeline returning a type-discriminatedValidatedProposal | Rejection(frozen dataclasses); debate =GroupChatBuilderround-robin with deterministic termination; learning = retrieval +ContextProviderinjection. Tests:tests/(production) +tests/spikes/(mirror); gated live arms via@pytest.mark.skipif(_NO_ENDPOINT); determinism asserted over fixed inputs. - Relevant files (verified):
src/portfolio_optimiser/backends.py(D2 seam, bothcreate_chat_clientraiseNotImplementedError:50-61,64-76),reference_domain.py(Project/CostItemfrozen +load_reference_projects()),data/reference_projects.json(3 projects, project[0]=FV42-GSV-E1). Real injection seam for tests:SessionContext(exported,_sessions.py:154;extend_instructions(source_id, instructions)two-arg:253). - Reusable code (promote from spikes — REUSE near-verbatim unless noted):
spikes/c_validator.py:58-205(Pydantic IR + validate_proposal + CBC + Monte-Carlo seed_MC_SEED=20260624+self_repairattempts-bounded +proposal_for;generate_via_llm:207is the stub);spikes/d_verdictstore.py:29-227(ProposalFeatures/Verdict/similarity/VerdictStore/ExpeLContextProvider/seed_store — bug:115one-argextend_instructions, masked by single-arg faketest_d_verdictstore.py:99);spikes/b_footguns.py:100-106(fresh_workflow());spikes/a_groupchat.py:48-63,130-163(termination + maker-checker builder);spikes/_harness.py:41-94(Budget/TokenMeter — drop_word_tokens:97-99). - External tech (researched): see Research Sources.
- Recent git activity:
src/written in Fase 0 (b57aa83), cold/settled. Spike cluster most-recently churned by the round-2 fan-out rebuild (a2dff21). Conventions: Conventional Commits,feat(fase2):,[skip-docs]on doc commits.
Research Sources
| Technology | Source | Key Findings | Confidence |
|---|---|---|---|
| MAF native HITL | research/01 | ctx.request_info/@response_handler/run(responses=); GroupChatBuilder.with_request_info(agents=[checker]); durable checkpoint-resume fragile → out-of-band verdict, defer checkpointing |
high |
| Local-folder MCP + citations | research/02 | official filesystem server can't cite → in-process retriever (D7 seam) + provenance as own Pydantic (route around #4316); SessionContext.extend_instructions(source_id,…) confirmed |
high |
| Local chat client + UsageDetails | research/03 | OpenAIChatCompletionClient(base_url) non-streaming; UsageDetails None-safe (_chat_completion_client.py:705,757); add_usage_details(); qwen3:4b; Intel-CPU = plumbing only |
high |
| Solver | Spike C + installed | pulp 3.3.2, bundled CBC works Intel mac; PULP_CBC_CMD deprecation; PuLP 4.0 → pulp[cbc]/COIN_CMD |
high |
Implementation Plan
Deterministic-core-first ordering (MAF-free modules before MAF-coupled wiring). TDD
throughout. Dependency note: TextSpan is owned by retrieval.py (Step 5) and
imported by provenance.py (Step 6) — so Step 6 follows Step 5. Headless halt
rule (applies to every step): on any failure, perform the step's On-failure action
then HALT the session — never run a dependent step on a failed predecessor.
Step 1: Promote orchestrations + pulp to core; resolve the mcp dependency decision
- Files:
pyproject.toml - Changes: Move
agent-framework-orchestrations>=1.0.0andpulp>=2.8fromdevinto[project.dependencies](now MVP-runtime). Update the pulp comment to note installed 3.3.2 + the PuLP-4.0pulp[cbc]/COIN_CMDmigration. mcp decision: attemptuv add mcp— if it resolves a GA (non-pre) release, add it to core deps (enables the optional thin MCP-server wrapper in Step 7); if it resolves only as a pre-release, DO NOT add it (GA-pin discipline) and record that the MVP data source is the in-process retriever-as-tool (research 02 fallback), with the MCP-server wrapper deferred as a documented fast-follow. Re-lock. (Per brief Constraints.) - Reuses: existing dependency block
pyproject.toml:7-26. - Test first:
- File:
tests/test_imports_core.py(new) - Verifies:
agent_framework_orchestrations+pulpimport;PULP_CBC_CMD().available()is True. - Pattern:
tests/spikes/test_imports.py
- File:
- Verify:
uv lock && uv sync && uv run pytest tests/test_imports_core.py→ expected:1 passed - On failure: retry once pinning exact installed versions; if still failing, escalate — then HALT.
- Checkpoint:
git commit -m "build(fase2): promote orchestrations + pulp to core deps" - Manifest:
manifest: expected_paths: - pyproject.toml - tests/test_imports_core.py min_file_count: 2 commit_message_pattern: "^build\\(fase2\\): promote orchestrations \\+ pulp to core deps$" bash_syntax_check: [] forbidden_paths: - src/portfolio_optimiser/backends.py must_contain: - path: pyproject.toml pattern: "agent-framework-orchestrations"
Step 2: Promote the Pydantic IR + blocking validator to src/
- Files:
src/portfolio_optimiser/ir.py(new),src/portfolio_optimiser/validator.py(new) - Changes: Move
AffectedItem/SavingsProposal(+ validators) intoir.py. Movevalidate_proposal,ValidatedProposal,Rejection,_solve_max_feasible,_monte_carlo(keep_MC_SEED=20260624),self_repair(attempts-bounded — promoted verbatim; token-budget bounding is added by the generate loop in Step 10, NOT here),proposal_for,CbcUnavailable,_quiet_pulpintovalidator.py. Pure modules — NOagent_frameworkimport (D7-portable core). ImportProjectfromreference_domain. (new files) - Reuses:
spikes/c_validator.py:44-205;reference_domain.Project. - Test first:
- File:
tests/test_validator.py(new) - Verifies: valid IR →
ValidatedProposal(p10<=p50<=p90); out-of-range →Rejection(reason, no percentiles); same fixed input twice → identical(p10,p50,p90); Pydantic blocks negative qty / claim>affected-total. - Pattern:
tests/spikes/test_c_validator.py:24-78(determinism over FIXED inputs, never an LLM)
- File:
- Verify:
uv run pytest tests/test_validator.py→ expected:5 passed - On failure: revert
git checkout -- src/portfolio_optimiser/ir.py src/portfolio_optimiser/validator.py tests/test_validator.py; HALT. - Checkpoint:
git commit -m "feat(fase2): promote blocking validator + IR to src" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/ir.py - src/portfolio_optimiser/validator.py - tests/test_validator.py min_file_count: 3 commit_message_pattern: "^feat\\(fase2\\): promote blocking validator \\+ IR to src$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/validator.py pattern: "class Rejection" - path: src/portfolio_optimiser/validator.py pattern: "_MC_SEED"
Step 3: Promote VerdictStore + ExpeL + verdict capture (two-arg fix; real-SessionContext test)
- Files:
src/portfolio_optimiser/verdicts.py(new) - Changes: Move
ProposalFeatures,Verdict,similarity,VerdictStore(in-memory, deterministic top-k retrieve),ExpeLContextProvider(ContextProvider),seed_store(). Fix the Fase 1 bug:before_runmust callcontext.extend_instructions(self.source_id, [self.format_fewshot()])— TWO args (_sessions.py:253). AddVerdict.idminting = stable content hashsha256(canonical(ProposalFeatures))[:16](so a structurally identical proposal maps to a stable id) andcapture_verdict(features, decision, rationale) -> Verdict(the Layer-2 out-of-band constructor). VerdictStore is in-memory only for the MVP — no JSON/file persistence (durable persistence deferred to Fase 3; the brief's learning-loop criterion is satisfied within a session viaseed_store()+ in-memory state). (new file) - Reuses:
spikes/d_verdictstore.py:29-227(with the:115two-arg correction);agent_framework.ContextProvider,agent_framework.SessionContext. - Test first:
- File:
tests/test_verdicts.py(new) - Verifies: structural match beats text decoys (
hits[0].id=="TRUE"); retrieval deterministic;k<=0raises;capture_verdictmints a stable id (same features → same id);before_runpopulates a REALSessionContextvia two-argextend_instructions(source_id, instructions)— constructfrom agent_framework import SessionContext, callawait provider.before_run(...)with it, assert the retrieved verdict text landed inctx.instructions(NOT a single-arg fake — this exercises the genuine GA signature and retires the Critical risk). - Pattern:
tests/spikes/test_d_verdictstore.py:67-89(retrieval) + realSessionContext
- File:
- Verify:
uv run pytest tests/test_verdicts.py→ expected:5 passed - On failure: revert
git checkout -- src/portfolio_optimiser/verdicts.py tests/test_verdicts.py; HALT. - Checkpoint:
git commit -m "feat(fase2): VerdictStore + ExpeL + capture_verdict (two-arg, real SessionContext test)" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/verdicts.py - tests/test_verdicts.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): VerdictStore \\+ ExpeL \\+ capture_verdict \\(two-arg, real SessionContext test\\)$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/verdicts.py pattern: "extend_instructions\\(self.source_id" - path: src/portfolio_optimiser/verdicts.py pattern: "def capture_verdict"
Step 4: Token meter + budget ChatMiddleware off real UsageDetails
- Files:
src/portfolio_optimiser/budget.py(new) - Changes: Promote
Budget/BudgetExceeded/TokenMeter(fail-fast positive caps, structuredBudgetExceeded.kind/limit/observed) fed from realUsageDetails: aChatMiddlewarereadsresponse.usage_details["total_token_count"](None-safe viaadd_usage_details) and short-circuits when the cap is crossed.usage_details is Nonehandling: astrict_usage: boolflag (default True in dev/prod) makes aNoneusage a HARD FAIL (a usage regression must never silently disable the cap, research 03 Rec 3); test doubles that legitimately supply synthetic usage do not trip it. NOlen(...split())proxy. (new file) - Reuses:
spikes/_harness.py:41-94(drop_word_tokens);agent_framework.UsageDetails/add_usage_details(_types.py:417),agent_framework.ChatMiddleware. - Test first:
- File:
tests/test_budget.py(new) - Verifies (hand-built
UsageDetails, no LLM): meter readstotal_token_count+ accumulates; cap crossed →BudgetExceeded; non-positive cap rejected; withstrict_usage=True,usage_details=Noneraises. Meta-guard:grep -rE "\.split\(\)" src/finds zero token proxies. - Pattern:
tests/spikes/test_harness.py:27-60
- File:
- Verify:
uv run pytest tests/test_budget.py→ expected:5 passed - On failure: revert
git checkout -- src/portfolio_optimiser/budget.py tests/test_budget.py; HALT. - Checkpoint:
git commit -m "feat(fase2): real-UsageDetails token meter + budget middleware" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/budget.py - tests/test_budget.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): real-UsageDetails token meter \\+ budget middleware$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/budget.py pattern: "usage_details"
Step 5: Framework-agnostic local-folder retriever core (owns TextSpan + path-security)
- Files:
src/portfolio_optimiser/retrieval.py(new) - Changes: Define
TextSpan(start_index,end_index— the single owner; imported by provenance in Step 6),RetrievedChunk(file,locator: TextSpan,snippet,score), andretrieve(query, docs_dir, top_k) -> list[RetrievedChunk]. Chunk-at-ingest with EXACT char-span locators (citations exact by construction). Keyword/substring scoring (no heavy vector deps — D5/D6). Path-security (this module reads the docs folder): canonicalise +os.path.realpath+ boundary-check every accessed path againstdocs_dir, fail closed on prefix-collision and symlink-escape (the EscapeRoute CVE class, research 02 Dim 5); native file APIs only. NOagent_framework/mcpimports — D7-portable seam. (new file) - Reuses: stdlib only;
TextSpanaligns withprovenance.Citation.locator(Step 6). - Test first:
- File:
tests/test_retrieval.py(new) - Verifies:
retrievereturns chunks whoselocatorchar-span exactly slicessnippet; deterministic for fixed query+dir; top_k respected; a../traversal and a symlink pointing outsidedocs_dirare both rejected (fail-closed); a sibling dir sharing a name prefix is rejected. - Pattern:
tests/test_reference_domain.py:8-38+tests/test_backends.py:30-32(raises)
- File:
- Verify:
uv run pytest tests/test_retrieval.py→ expected:7 passed - On failure: revert
git checkout -- src/portfolio_optimiser/retrieval.py tests/test_retrieval.py; HALT. - Checkpoint:
git commit -m "feat(fase2): local-folder retriever core with path-security" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/retrieval.py - tests/test_retrieval.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): local-folder retriever core with path-security$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/retrieval.py pattern: "class RetrievedChunk" - path: src/portfolio_optimiser/retrieval.py pattern: "realpath"
Step 6: First-class Pydantic provenance stamp
- Files:
src/portfolio_optimiser/provenance.py(new) - Changes: Define
Citation(file,locator: TextSpanimported fromretrieval,snippet) andProvenanceStamp(citations: list[Citation]min_length=1,model,role,validator_decision: Literal["validated","rejected"],token_usage: int). Authoritative provenance — independent of MAFAnnotation(#4316). Provideto_annotations()mapping to MAFAnnotation(type="citation", …)for display only. (new file; importsTextSpanfrom Step 5) - Reuses:
retrieval.TextSpan(Step 5);agent_framework._types.Annotation/TextSpanRegion(display adapter);pydantic.BaseModel. - Test first:
- File:
tests/test_provenance.py(new) - Verifies: 0 citations →
ValidationError; valid stamp exposes ≥1 citation + model/role + decision + token usage;to_annotations()yieldstype=="citation"dicts. - Pattern:
tests/spikes/test_c_validator.py:66-78
- File:
- Verify:
uv run pytest tests/test_provenance.py→ expected:3 passed - On failure: revert
git checkout -- src/portfolio_optimiser/provenance.py tests/test_provenance.py; HALT. - Checkpoint:
git commit -m "feat(fase2): first-class Pydantic provenance stamp" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/provenance.py - tests/test_provenance.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): first-class Pydantic provenance stamp$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/provenance.py pattern: "class ProvenanceStamp"
Step 7: Expose the retriever to the agents as a citation-bearing data source
- Files:
src/portfolio_optimiser/datasource.py(new) - Changes: Wire
retrieval.retrieve()into the debate as a citation-bearing data source. MVP (GA-safe) path: a GA@tool/FunctionTool(or aContextProvider) that callsretrieve()and returns chunks whose{file, locator, snippet}the orchestrator maps intoprovenance.Citation— no new dependency, D7-portable. Conditional MCP wrapper: IF Step 1 addedmcpas a GA dependency, ALSO expose a thin custom stdio MCP server (FastMCP) returning the same chunks asstructuredContent, consumed viaMCPStdioTool(parse_tool_results=…), to honor the CLAUDE.md "data access via MCP" convention; ifmcpwas not GA-addable, the MCP wrapper is a documented fast-follow and the MVP uses the in-process tool. Either way the path-security lives inretrieval.py(Step 5). (new file) - Reuses:
retrieval.retrieve(Step 5);provenance.Citation(Step 6);agent_framework@tool/FunctionTool; (conditional)MCPStdioTool+mcp. - Test first:
- File:
tests/test_datasource.py(new) - Verifies: the tool/ContextProvider returns citation-ready chunks for a query over the synthetic docs dir; chunks carry exact locators usable to build a
Citation; (ifmcppresent) the MCP wrapper returns the samestructuredContentshape. - Pattern:
tests/test_retrieval.py(Step 5) +tests/test_backends.py
- File:
- Verify:
uv run pytest tests/test_datasource.py→ expected:all passed - On failure: revert
git checkout -- src/portfolio_optimiser/datasource.py tests/test_datasource.py; HALT. - Checkpoint:
git commit -m "feat(fase2): expose retriever as citation-bearing data source" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/datasource.py - tests/test_datasource.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): expose retriever as citation-bearing data source$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/datasource.py pattern: "retrieve"
Step 8: Wire the two backend profiles + role→model map + env template
- Files:
src/portfolio_optimiser/backends.py,src/portfolio_optimiser/data/model_map.json(new),.env.template(new),tests/test_backends.py - Changes: Implement
LocalBackend.create_chat_client→OpenAIChatCompletionClient(base_url=<PORTFOLIO_LOCAL_BASE_URL>, api_key=<PORTFOLIO_LOCAL_API_KEY default "ollama">, model_id=<resolved>)configured non-streaming (research 03 — NOTOpenAIChatClient). ImplementAzureFoundryBackend.create_chat_client→FoundryChatClient(deployment names from env). Add adata/model_map.json(the role→model/deployment map artifact, the SAME file Step 11's contract validates) loaded into a small resolver feedingcreate_chat_client(model=…). Add.env.templatedocumentingPORTFOLIO_LOCAL_BASE_URL,PORTFOLIO_LOCAL_API_KEY, Foundryproject_endpoint/credential + the research-03 no-egress notes (Ollama bind127.0.0.1, pin ≥0.17.1,OLLAMA_DEBUGunset, model-pull = explicit egress). Keepget_backend()fail-fast. Invert the skeleton test. - Reuses:
backends.py:26-84;spikes/_harness.py:191-215(env pattern, switch client class);agent_framework_openai.OpenAIChatCompletionClient,agent_framework_foundry.FoundryChatClient. - Test first:
- File:
tests/test_backends.py(existing — modify) - Verifies: both backends satisfy
ChatBackend; unknown profile raisesValueError; the model_map resolves a role→model id;create_chat_clientreturns a client object (no network) — replacetest_create_chat_client_is_skeleton(:36). - Pattern:
tests/test_backends.py:14-39
- File:
- Verify:
uv run pytest tests/test_backends.py→ expected:all passed - On failure: revert
git checkout -- src/portfolio_optimiser/backends.py src/portfolio_optimiser/data/model_map.json .env.template tests/test_backends.py; HALT. - Checkpoint:
git commit -m "feat(fase2): wire local + Foundry profiles + model-map + env template" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/backends.py - src/portfolio_optimiser/data/model_map.json - .env.template - tests/test_backends.py min_file_count: 4 commit_message_pattern: "^feat\\(fase2\\): wire local \\+ Foundry profiles \\+ model-map \\+ env template$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/backends.py pattern: "OpenAIChatCompletionClient"
Step 9: fresh_workflow isolation factory + maker-checker GroupChat (Layer-1 HITL gate)
- Files:
src/portfolio_optimiser/workflow.py(new) - Changes: Promote
fresh_workflow()as the core factory — FRESHGroupChatBuilder+ FRESH chat clients per project run (zero cross-run bleed, B7). Maker-checker debate: proposer + checker, deterministic termination (make_termination),with_max_rounds(n)AND the external budget-middleware guard (B4:max_round_count=Nonedoes NOT self-terminate). Layer-1 HITL = optionally enablewith_request_info(agents=[checker])(in-process synchronous review, no checkpoint). Non-streaming. (new file) - Reuses:
spikes/b_footguns.py:100-106;spikes/a_groupchat.py:48-63,130-163;budget.ChatMiddleware(Step 4);backends(Step 8);agent_framework_orchestrations.GroupChatBuilder. - Test first:
- File:
tests/test_workflow.py(new) - Verifies (FakeChatClient): fresh workflow per run → each run a clean thread; a participant sees ONLY its own project (assert on received-message CONTENT, not a call counter — the Fase 1 round-2 fix); a never-converging debate with
with_max_rounds(3)halts at exactly 3 rounds (pin the observed count, retiring the 1-turn=1-round off-by-one assumption — not just<=). - Pattern:
tests/spikes/test_b_footguns.py:40-45+tests/spikes/test_harness.py:103-126
- File:
- Verify:
uv run pytest tests/test_workflow.py→ expected:all passed - On failure: revert
git checkout -- src/portfolio_optimiser/workflow.py tests/test_workflow.py; HALT. - Checkpoint:
git commit -m "feat(fase2): fresh_workflow factory + maker-checker GroupChat" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/workflow.py - tests/test_workflow.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): fresh_workflow factory \\+ maker-checker GroupChat$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/workflow.py pattern: "with_max_rounds"
Step 10: LLM→IR generation wired to validator-as-retry (token budget in the generate loop)
- Files:
src/portfolio_optimiser/generate.py(new) - Changes: Implement
generate_via_llm(chat_client, project, context, meter) -> SavingsProposal(replaces the stubc_validator.py:207): a NON-STREAMING chat call requesting structured output, parsed into the IR. OnValidationError(small models emit text-leaked/wrong-typed tool calls — research 03 Dim 3), invokevalidator.self_repair— bounded byself_repair's ownmax_attemptsAND by the budgetmeterchecked here in the generate loop between attempts (NO edit tovalidator.py— the validator stays the verbatim-promoted module from Step 2; the token bound lives in this loop). (new file) - Reuses:
validator.self_repair/ir.SavingsProposal(Step 2);budgetmeter (Step 4); a chat client frombackends(Step 8). - Test first:
- File:
tests/test_generate.py(new) - Verifies (FakeChatClient scripted): well-formed reply →
SavingsProposal; malformed/text-leaked reply → boundedself_repair(retried, not silently accepted); exhausting attempts OR crossing the meter cap → typed failure (never a malformed proposal); validator.py is unmodified (no diff). - Pattern:
tests/spikes/test_harness.py:66-76+test_c_validator.py
- File:
- Verify:
uv run pytest tests/test_generate.py→ expected:3 passed - On failure: revert
git checkout -- src/portfolio_optimiser/generate.py tests/test_generate.py; HALT. - Checkpoint:
git commit -m "feat(fase2): LLM->IR generation with validator-as-retry" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/generate.py - tests/test_generate.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): LLM->IR generation with validator-as-retry$" bash_syntax_check: [] forbidden_paths: - src/portfolio_optimiser/validator.py must_contain: - path: src/portfolio_optimiser/generate.py pattern: "self_repair"
Step 11: Fail-fast contract loaders
- Files:
src/portfolio_optimiser/contracts.py(new) - Changes: Pydantic models +
load_contracts(...)for the four contracts: data-source config, model-map (validatesdata/model_map.jsonfrom Step 8), termination contract, feedback schema. Validate ALL at startup and raise before any chat-client is constructed or called (brief NFR fail-fast). Pydantic gives JSON-Schema-grade validation (CLAUDE.md convention). (new file) - Reuses:
pydantic.BaseModel;data/model_map.json(Step 8);backends.Profile. - Test first:
- File:
tests/test_contracts.py(new) - Verifies: malformed data-source / model-map / termination / feedback →
ValidationErrorat load; AND no chat-client call was made (inject a FakeChatClient spy, assertcall_count==0). - Pattern:
tests/test_backends.py:30-32+ FakeChatClientcall_count(spikes/_harness.py:138)
- File:
- Verify:
uv run pytest tests/test_contracts.py→ expected:all passed - On failure: revert
git checkout -- src/portfolio_optimiser/contracts.py tests/test_contracts.py; HALT. - Checkpoint:
git commit -m "feat(fase2): fail-fast contract loaders" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/contracts.py - tests/test_contracts.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): fail-fast contract loaders$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/contracts.py pattern: "def load_contracts"
Step 12: Orchestrator + single-command entry (two-layer HITL wiring)
- Files:
src/portfolio_optimiser/run.py(new),src/portfolio_optimiser/__init__.py - Changes:
run_project(project_id, profile, verdict_input=None)composes the slice:load_contracts(fail-fast) → load project + retrieve cited chunks via the Step-7 data source →fresh_workflowmaker-checker debate (budget middleware + round cap; Layer-1 = the optional in-runwith_request_infogate) →generate_via_llmcandidate → blockingvalidate_proposal→ValidatedProposal | Rejection→ attachProvenanceStamp→ Layer-2 (out-of-band):capture_verdict(features, decision, rationale)wheredecisioncomes fromverdict_input(function arg / CLI prompt / fixture in tests; the B11 notification is a stub in Fase 2) →VerdictStorepersist → (next run)ExpeLContextProviderretrieval. Document the two layers explicitly in a module docstring. Console entry for the single-command run. Export public API from__init__.py. (new file + modify__init__.py) - Reuses: every Step 2–11 module.
- Test first:
- File:
tests/test_run_smoke.py(new) - Verifies:
run_projectimportable; runs end-to-end on an injected FakeChatClient (with a fixtureverdict_input) and returns aValidatedProposalwith aProvenanceStamp. - Pattern:
tests/test_smoke.py
- File:
- Verify:
uv run python -c "from portfolio_optimiser.run import run_project"→ no error;uv run pytest tests/test_run_smoke.py→1 passed - On failure: revert
git checkout -- src/portfolio_optimiser/run.py src/portfolio_optimiser/__init__.py tests/test_run_smoke.py; HALT. - Checkpoint:
git commit -m "feat(fase2): vertical-slice orchestrator + two-layer HITL wiring" - Manifest:
manifest: expected_paths: - src/portfolio_optimiser/run.py - tests/test_run_smoke.py min_file_count: 2 commit_message_pattern: "^feat\\(fase2\\): vertical-slice orchestrator \\+ two-layer HITL wiring$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: src/portfolio_optimiser/run.py pattern: "def run_project"
Step 13: End-to-end integration test (success criteria, on FakeChatClient with synthetic usage)
- Files:
tests/test_vertical_slice_e2e.py(new),tests/conftest.py(new) - Changes: A deterministic CI integration test (scripted FakeChatClient — no real LLM). The test double emits a synthetic
UsageDetails(e.g.{"total_token_count": N}) so the budget middleware (Step 4,strict_usage=True) does NOT hard-fail and the provenance stamp'stoken_usageis a positive number sourced fromUsageDetails(this is the in-CI accounting check; the REAL-provider populated-usage assertion lives in Step 14). Add a sharedtests/conftest.pyproviding the scripted FakeChatClient + synthetic-usage factory +seed_storefixtures. Criteria covered: (a) one valid proposal e2e → exactly oneValidatedProposalwith populated provenance (≥1 citation, model/role, validator decision, token usage>0 from synthetic UsageDetails); (b) out-of-range →Rejectionwith reason; (c) Layer-2 verdict captured viacapture_verdict→ written to VerdictStore (record exists); (d) second run on a structurally similar proposal retrieves the prior verdict viaExpeLContextProvider(assertretrieved.id == persisted.id, exercising two-argextend_instructions); (e) tiny budget/round cap → halts via middleware/round-cap (assert cap not exceeded); (f) malformed contract → raises before any chat call. (new files) - Reuses:
run.run_project;tests/conftest.pyFakeChatClient + synthetic-usage;seed_store(). - Test first: this IS the test step.
- File:
tests/test_vertical_slice_e2e.py(new) - Verifies: all six assertions above.
- Pattern:
tests/spikes/test_harness.py:103-126+test_c_validator.py+test_d_verdictstore.py
- File:
- Verify:
uv run pytest && uv run ruff check . && uv run mypy src→ expected: all pass, ruff clean, mypy clean - On failure: escalate — an e2e failure means a composition seam is wrong; diagnose against the failing criterion; HALT.
- Checkpoint:
git commit -m "test(fase2): end-to-end vertical-slice integration suite" - Manifest:
manifest: expected_paths: - tests/test_vertical_slice_e2e.py - tests/conftest.py min_file_count: 2 commit_message_pattern: "^test\\(fase2\\): end-to-end vertical-slice integration suite$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: tests/test_vertical_slice_e2e.py pattern: "ValidatedProposal"
Step 14: Gated live-profile checks (real UsageDetails + no-egress)
- Files:
tests/test_local_profile_live.py(new),tests/test_foundry_profile_live.py(new) - Changes: Separately-gated tests (
@pytest.mark.skipif, NOT default CI). Local: a REALOpenAIChatCompletionClient(base_url)non-streaming call returns a response AND the meter is populated from realUsageDetails— positive, load-bearingtotal_token_count > 0(this is the criterion FakeChatClient cannot prove). Assert the call goes only to the configured loopbackbase_url(no-egress: no outbound toapi.openai.com). Foundry: a trivial agent responds onFoundryChatClient(cheapest model, hard cap — D6). (new files) - Reuses:
backends(Step 8);live_local_client_or_skipskip pattern (spikes/_harness.py:191-215). - Test first: these ARE gated tests.
- File:
tests/test_local_profile_live.py,tests/test_foundry_profile_live.py(new) - Verifies: skip cleanly without env; when env present, a successful response +
total_token_count > 0from each path; local call targets only the loopback endpoint. - Pattern:
tests/spikes/test_a_groupchat.py:14-16,65
- File:
- Verify:
uv run pytest tests/test_local_profile_live.py tests/test_foundry_profile_live.py→ expected:2 skipped(no endpoint in CI) — orpassedwith env set - On failure: skip — no endpoint is the expected CI outcome; a non-skip failure with env set escalates; HALT.
- Checkpoint:
git commit -m "test(fase2): gated live local + Foundry profile checks" - Manifest:
manifest: expected_paths: - tests/test_local_profile_live.py - tests/test_foundry_profile_live.py min_file_count: 2 commit_message_pattern: "^test\\(fase2\\): gated live local \\+ Foundry profile checks$" bash_syntax_check: [] forbidden_paths: [] must_contain: - path: tests/test_local_profile_live.py pattern: "skipif"
Alternatives Considered
| Approach | Pros | Cons | Why rejected |
|---|---|---|---|
Reuse @modelcontextprotocol/server-filesystem |
No custom code; official | Cannot produce citations; npx/Node dep; CVE/supply-chain surface (research 02) | Citation NFR is load-bearing; in-process retriever produces exact locators |
| Standalone thin MCP server on the MVP critical path | Honors "data access via MCP" convention fully | Requires mcp (not installed; possibly --pre → violates GA-pin); extra process |
In-process retriever-as-tool for MVP (GA-safe); MCP wrapper conditional on mcp GA (Step 7) |
Native OllamaChatClient for local |
MS-recommended; sidesteps /v1 edges |
--pre beta + new ollama dep — violates GA-pin |
OpenAIChatCompletionClient non-streaming; native is spike-gated fallback |
| MAF checkpoint-resume for durable verdict | Native durable pause | Fragile (open bugs into 1.9.0; pickle vs Pydantic IR) + D7 lock-in | Out-of-band VerdictStore; defer checkpointing |
MAF Annotation propagation for provenance |
Native | Python streaming-drop bug #4316 → silent empty citations | First-class own Pydantic provenance |
| JSON-persistent VerdictStore in MVP | Cross-process durability | Not required by any Fase 2 criterion (in-session learning suffices) | In-memory for MVP; durable persistence deferred to Fase 3 |
Test Strategy
- Framework:
pytest+pytest-asyncio(asyncio_mode="auto").pythonpath=["src","."],testpaths=["tests"]. New sharedtests/conftest.py(Step 13) provides the scripted FakeChatClient + synthetic-UsageDetailsfactory +seed_storefixtures. - Existing patterns: deterministic validator/Monte-Carlo over FIXED inputs (
test_c_validator.py:59-63, seed20260624); FakeChatClient driving real GA builders with round caps (test_harness.py:103-126); gated live arm viaskipif(test_a_groupchat.py:14-16); real-SessionContextexercise for the two-argextend_instructions(Step 3). - Determinism discipline (research 03 Dim 5): determinism asserted ONLY over the validator + Monte-Carlo with fixed inputs; NO test asserts LLM output. The propose (non-deterministic) → validate (deterministic) boundary is explicit.
- Usage-accounting split: in-CI accounting uses the FakeChatClient's synthetic
UsageDetails(Step 13); the REAL-providertotal_token_count > 0assertion is gated/live (Step 14) — this is the only place "real UsageDetails" is provable.
Tests to write
| Type | File | Verifies | Model test |
|---|---|---|---|
| Unit | tests/test_validator.py |
validate/reject/determinism/Pydantic-block | tests/spikes/test_c_validator.py |
| Unit | tests/test_verdicts.py |
retrieval + REAL-SessionContext two-arg extend_instructions + id minting | tests/spikes/test_d_verdictstore.py |
| Unit | tests/test_budget.py |
UsageDetails meter, strict None hard-fail, cap, no-proxy guard | tests/spikes/test_harness.py |
| Unit | tests/test_retrieval.py |
exact-locator chunks + EscapeRoute path rejection | tests/test_reference_domain.py |
| Unit | tests/test_provenance.py |
≥1 citation, fields, annotation adapter | tests/spikes/test_c_validator.py |
| Unit | tests/test_datasource.py |
citation-ready chunks via tool/(opt) MCP | tests/test_retrieval.py |
| Unit | tests/test_backends.py |
profiles wired, model-map, no-NotImplementedError | tests/test_backends.py (existing) |
| Integration | tests/test_workflow.py |
fresh-instance isolation (content), exact round cap | tests/spikes/test_b_footguns.py |
| Unit | tests/test_generate.py |
LLM→IR + bounded self_repair (attempts+budget); validator.py unmodified | tests/spikes/test_harness.py |
| Unit | tests/test_contracts.py |
fail-fast before any chat call | tests/test_backends.py |
| Integration | tests/test_vertical_slice_e2e.py |
all 6 e2e criteria (synthetic usage) | tests/spikes/test_harness.py |
| Live (gated) | tests/test_local_profile_live.py |
real client + real UsageDetails>0 + loopback-only | tests/spikes/test_a_groupchat.py |
| Live (gated) | tests/test_foundry_profile_live.py |
trivial Foundry response | tests/spikes/test_a_groupchat.py |
Risks and Mitigations
| Priority | Risk | Location | Impact | Mitigation |
|---|---|---|---|---|
| Critical | extend_instructions single-arg bug masked by a tautological fake |
spikes/d_verdictstore.py:115, test_d_verdictstore.py:99 |
Learning-loop close crashes against a real SessionContext |
Step 3: two-arg call + a test using a REAL SessionContext (not a fake), retiring the risk in CI |
| Critical | Budget enforcement word-count theatre / usage=None silently disables cap |
spikes/_harness.py:97-99 |
Hard-cap NFR fake; unbounded run | Step 4: middleware off UsageDetails; strict_usage None=hard fail; non-streaming; src grep-guard |
| Critical | Round cap not self-terminating | Fase 1 B4 | Unbounded debate on a real client | Step 9: explicit with_max_rounds + external budget guard; test pins EXACT round count |
| High | usage=None on FakeChatClient breaks e2e under strict middleware |
Steps 4/13 | e2e cannot run | Step 13: test double emits synthetic UsageDetails; real-usage assertion moved to gated Step 14 |
| High | mcp not installed / possibly --pre |
Steps 1/7 | Step 7 import fails; GA-pin violated | Step 1 conditional add; Step 7 MVP uses in-process tool (no dep); MCP wrapper only if mcp GA |
| High | Wrong OpenAI client (Responses vs Chat Completions) | spikes/_harness.py:210, backends.py:65 |
Local profile fails / UsageDetails not populated | Step 8: OpenAIChatCompletionClient non-streaming |
| High | Small-model tool-calling unreliability | research 03 Dim 3 | Malformed candidates crash/pass | Step 10: validator-as-retry (attempts+budget); qwen3:4b; num_ctx≥32k |
| High | Provenance via MAF annotation (#4316) | research 02 Dim 2 | Citations silently drop on streaming | Step 6: own Pydantic provenance; non-streaming |
| High | Retriever owns path/symlink security (EscapeRoute) | Step 5 | Out-of-folder read → silent exfil | Step 5: canonicalise+realpath+boundary fail-closed; TDD bypass patterns |
| Medium | self_repair token-budget vs frozen validator | Steps 2/10 | Scope-fence violation | Step 10: budget checked in generate loop; validator.py in forbidden_paths |
| Medium | HITL verdict on checkpoint-resume | research 01 | Silent loss / pickle rejection | Out-of-band capture_verdict; checkpointing deferred |
| Medium | Determinism asserted through the LLM | research 03 Dim 5 | Flaky tests | Pin validator+MC inputs only |
| Medium | Intel-CPU latency makes a real debate minutes-long | research 03 Dim 4 | Live test hangs | Tiny runs (1 maker+1 checker, ≤3 rounds, hard caps); gate live; representative runs on Foundry |
| Medium | fresh_workflow reuse → B7 bleed returns |
spikes/b_footguns.py:100 |
Silent cross-project contamination | Step 9: fresh workflow + clients per run; B7 content-level test |
| Medium | Headless cascade on dependency-chained failure | all steps | Downstream steps run on a failed predecessor | Global halt rule + per-step "HALT" in On-failure |
| Low | #1772 double system prompt (OpenAIChatClient+Ollama+middleware) |
research 03 Dim 3 | Wasted tokens | Verify vs 1.9.0 in client spike-gate; native client avoids it |
| Low | no-op uv.lock diff on re-lock |
Step 1 | Manifest min_file_count misfire | Step 1 manifest gates only pyproject.toml + test (min_file_count 2) |
Assumptions
| # | Assumption | Why unverifiable | Impact if wrong |
|---|---|---|---|
| 1 | Research is COMPLETE (3 valid briefs on disk) | — (verified) | Would re-run research |
| 2 | Follow research over brief prose (HITL split, in-process retriever, OpenAIChatCompletionClient non-streaming) | research post-dates brief prose | Planning against superseded premises |
| 3 | Operator supplies a working local OpenAI-compatible endpoint + model (Ollama qwen3:4b on 127.0.0.1:11434/v1/) |
machine-specific (brief [OPEN]) | Local full-run + "both profiles" criterion can't execute; local-only run still satisfies the other 8 |
| 4 | Operator supplies Foundry deployment names | tenant-specific (brief [OPEN]) | Foundry minimal check skips |
| 5 | mcp GA availability decides Step 7's MCP wrapper (else in-process tool only) |
not installed; resolved in Step 1 | If --pre-only, the "data access via MCP" convention is met by a fast-follow, not the MVP |
| 6 | Synthetic Fase 0 domain reused as the single project | brief [ASSUMPTION] | — |
| 7 | Solver stays PuLP/CBC (installed 3.3.2, CBC present); PuLP 4.0 → pulp[cbc]/COIN_CMD |
— (verified) | Future PuLP bump breaks bundled CBC |
Verification
End-to-end integration checks (per-step manifests verify each step):
uv run pytest→ all pass (live arms skip without env)uv run ruff check .→ cleanuv run mypy src→ cleanuv run python -c "from portfolio_optimiser.run import run_project"then a FakeChatClient e2e → exactly oneValidatedProposalwith a populatedProvenanceStamp(≥1 citation, model/role, validator decision, token usage>0)- Out-of-range candidate →
Rejection(type-asserted, reason) - Second run on a structurally similar proposal →
retrieved.id == persisted.id - Malformed contract → raises at startup; no chat-client call made
- Tiny budget/round cap → halts; loop did not exceed cap (exact round count pinned)
grep -rE "\.split\(\)" src/→ no word-count token proxy- No silent egress: gated live local call targets only the loopback
base_url(Step 14); telemetry env vars unset - Gated live (when env set): real
UsageDetails.total_token_count > 0on the local profile; trivial Foundry response
Estimated Scope
- Files to modify: 2 existing (
pyproject.toml,backends.py,__init__.py,tests/test_backends.py— 4 touch points) - Files to create: ~25 (11
src/modules incl.datasource.py+data/model_map.json+.env.template+ ~12 test files incl.conftest.py) - Complexity: high (integration of four subsystems on real clients, two profiles)
Execution Strategy
14 steps > 5 → grouped into sessions/waves. Dependencies are explicit; sessions in the same wave touch disjoint files AND have no cross-session import edge. Global headless rule: on any step failure, perform the On-failure action then HALT — never run a dependent step on a failed predecessor.
Session 1: Dependencies
- Steps: 1
- Wave: 1
- Depends on: none
- Scope fence: Touch
pyproject.toml,uv.lock,tests/test_imports_core.py; Never touchsrc/**
Session 2: Deterministic core (no MAF), dependency-ordered
- Steps: 2, 5, 6, 3, 4 (note: 5 retrieval before 6 provenance — TextSpan ownership; all internally independent except 6→5)
- Wave: 2
- Depends on: Session 1
- Scope fence: Touch
src/portfolio_optimiser/{ir,validator,retrieval,provenance,verdicts,budget}.py+ tests; Never touchbackends.py,workflow.py,datasource.py
Session 3: Data source + backend wiring
- Steps: 7, 8
- Wave: 3 (Step 7 depends on Steps 5+6; Step 8 independent of Session 2 internals)
- Depends on: Session 2 (Step 7 → retrieval/provenance)
- Scope fence: Touch
src/portfolio_optimiser/{datasource,backends}.py,data/model_map.json,.env.template+ tests; Never touch the core modules' internals (import only)
Session 4: Composition
- Steps: 9, 10, 11, 12
- Wave: 4
- Depends on: Sessions 2, 3
- Scope fence: Touch
src/portfolio_optimiser/{workflow,generate,contracts,run,__init__}.py+ tests; Never touch core/datasource/backends internals (incl.validator.py— Step 10 forbidden_path)
Session 5: Integration + live
- Steps: 13, 14
- Wave: 5
- Depends on: Session 4
- Scope fence: Touch
tests/test_vertical_slice_e2e.py,tests/conftest.py,tests/test_local_profile_live.py,tests/test_foundry_profile_live.py; Never touchsrc/**
Execution Order
- Wave 1: Session 1
- Wave 2: Session 2 (internal order 2 → 5 → 6 → 3 → 4)
- Wave 3: Session 3 (after Wave 2)
- Wave 4: Session 4 (after Wave 3)
- Wave 5: Session 5 (after Wave 4)
Grouping rules applied
- Steps sharing files / import edges → same session, sequenced; 3–5 steps/session; sessions ordered by dependency. No two sessions in the same wave share an import edge (the Step 7→6 edge moved Step 7 to Wave 3).
Plan Quality Score
| Dimension | Weight | Score | Notes |
|---|---|---|---|
| Structural integrity | 0.15 | 90 | dependency-ordered; TextSpan ownership fixed; Step 7 moved to its own wave; halt rule added |
| Step quality | 0.20 | 88 | each step 1–2 files, TDD, concrete reuse; verdict capture + id minting now explicit |
| Coverage completeness | 0.20 | 92 | all 9 criteria → steps + verification; two-layer HITL mapped; no-egress verified |
| Specification quality | 0.15 | 86 | model-map artifact + env template defined; some new-module internals left to impl |
| Risk & pre-mortem | 0.15 | 92 | 17 risks incl. the 3 premise corrections; extend_instructions retired by a real-session test |
| Headless readiness | 0.10 | 88 | On-failure + HALT + Checkpoint + Manifest per step |
| Manifest quality | 0.05 | 86 | checkable manifests; strengthened retrieval/datasource predicates; lock-step gating fixed |
| Weighted total | 1.00 | 89 | Grade: A− |
Adversarial review:
- Plan critic: REVISE on the first draft — 3 blockers (FakeChatClient usage contradiction, missing
mcpdep, unspecified two-layer HITL capture), 7 major, 4 minor. All addressed in Revisions below. - Scope guardian: ALIGNED — 0 creep, all 9 criteria mapped, all 6 Non-Goals honored; 1 minor (two-layer HITL naming) addressed.
Revisions
| # | Finding | Severity | Resolution |
|---|---|---|---|
| 1 | FakeChatClient e2e can't satisfy usage-populated vs None-hard-fail |
blocker | Step 13 test double emits synthetic UsageDetails; strict_usage flag (Step 4); real-usage assertion moved to gated Step 14 |
| 2 | mcp/FastMCP dependency never added |
blocker | Step 1 conditional mcp add (GA only); Step 7 MVP uses in-process retriever-as-tool (no dep); MCP wrapper conditional/fast-follow |
| 3 | Two-layer HITL verdict-capture unspecified; verdict-id minting undefined | blocker | Step 3 adds capture_verdict + stable Verdict.id hash; Step 12 wires Layer-1 (in-run gate) + Layer-2 (out-of-band capture) + B11 notification stub, mapped explicitly |
| 4 | extend_instructions not retired by a real-session test |
major | Step 3 test uses a REAL agent_framework.SessionContext (verified importable), not a fake |
| 5 | Session 2∥3 not independent (Step 7→6) + unowned TextSpan |
major | TextSpan owned by retrieval.py (Step 5), imported by provenance (Step 6); Step 7 moved to Wave 3; Execution Strategy reworked |
| 6 | env vars / .env.template / no-silent-egress unverified |
major | Step 8 adds .env.template + no-egress notes; Verification adds loopback-only + telemetry-unset checks (Step 14) |
| 7 | No circuit-breaker/halt language | major | Global halt rule + per-step "HALT" appended to every On-failure |
| 8 | Step 3 JSON persistence = scope creep | major | Removed; VerdictStore in-memory for MVP; durable persistence deferred to Fase 3 |
| 9 | self_repair token-budget edits frozen validator.py | major | Step 10 enforces the token bound in the generate loop; validator.py in Step 10 forbidden_paths |
| 10 | Step 1 manifest min_file_count:3 may misfire on no-op lock |
major | Manifest gates only pyproject.toml + test (min_file_count 2) |
| 11 | Estimated Scope counts inconsistent | minor | Corrected |
| 12 | model-map config artifact undefined | minor | data/model_map.json defined in Step 8, validated by Step 11 |
| 13 | Round-cap assertion doesn't catch off-by-one | minor | Step 9 pins the EXACT observed round count |
| 14 | Weak realpath manifest predicate |
minor | Path-security moved into retrieval.py (Step 5) with realpath + behavioural EscapeRoute tests |
Adversarial Pass 2 (gemini-bridge, v5.1.1 high-effort)
Status: UNAVAILABLE. The high-effort plan signal mandates an independent
gemini-bridge review of the post-revision plan. The gemini-mcp server has
returned an identical 400 BadRequestError on every call this project (its
client SDK predates Google's May-2026 Interactions API change). A fifth
invocation would fail deterministically, so it was not attempted (cost
discipline D6). Independent second-opinion triangulation on the plan is
therefore absent — the plan rests on plan-critic + scope-guardian (Phase 9) +
installed-source-grounded exploration. To restore this pass, upgrade the
gemini-mcp client SDK to ≥ 2.0.0 and re-run /trekplan (or a standalone
gemini review) on plan.md.