docs(research): MAF 1.9.0 capability map — feature-utilization for Fase 2 [skip-docs]

Two independent grounded passes (installed-source introspection + official MS
Learn via MCP) produce a per-need adopt/keep decision table for using MAF
features well in Fase 2, instead of reinventing them.

Headline: Microsoft's Workflows "State Isolation" page documents verbatim the
exact footgun Spike B(b) found today — a reused Workflow accumulates agent
threads across runs; the fix is a fresh-instance-per-run factory. Our
fresh_workflow() IS the official pattern.

Key verdicts: ADOPT real UsageDetails token counts + a budget ChatMiddleware +
native builder round caps + GA @tool/MCP + observability; KEEP the hand-rolled
structural VerdictStore and inline validator (MAF memory/eval are the wrong
shape); ROLL a tiny role->deployment map (declarative is preview/not installed).
Corrections recorded: CLAUDE.md "Magentic experimental" stands at doc-level (no
code gate); Spike D extend_instructions is two-arg (source_id, instructions).

Skills answer: method-as-Skill yes (MAF consumes SKILL.md natively, experimental);
MAF-docs-mirror Skill no (rots vs live MCP); the digest lives in this map.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fif1r1En5W542HbZV88yMH

docs/research/2026-06-24-maf-capability-map.md

@@ -0,0 +1,159 @@
+# MAF capability map — feature-utilization for portfolio-optimiser
+
+> **Purpose.** A living map of what Microsoft Agent Framework (MAF) `agent-framework-core`
+> **1.9.0** actually offers, what this repo currently hand-rolls, and a per-need decision
+> on whether to adopt the MAF feature or keep our own. It exists so Fase 2 wires MAF into
+> core *using its features well* instead of reinventing them. Front-end input to the Fase 2
+> `/trekbrief`.
+>
+> **Provenance & trust.** Grounded by two independent passes on 2026-06-24:
+> (1) source-introspection of the *installed* `agent_framework` 1.9.0 (+ `agent_framework_orchestrations` 1.0.0)
+> with `file:line` citations, and (2) official Microsoft Learn docs via the `microsoft-learn` MCP.
+> Each verdict is tagged **[verified-source]** (read in installed bytes), **[verified-docs]**
+> (official Learn), or **[hypothesis]** (needs a confirming spike). Version caveat: Learn pages
+> are not version-pinned (the .NET API ref shows `v1.0.0-rc2`); where source and docs disagree,
+> **the installed 1.9.0 source wins** for our code. Triggered the search-first + verification
+> duties: every symbol below was read or cited, not recalled.
+
+## 0. Headline: Microsoft documents the exact footgun Spike B(b) found
+
+Today's `/trekreview` BLOCKER was that Spike B(b) "confirmed" fan-out state isolation with a
+tautological call-counter. The rebuilt experiment showed a reused `ConcurrentBuilder` `Workflow`
+accumulates the conversation thread across `.run()` calls (project N contaminates N+1). **MAF's
+own Workflows "State Isolation" page states this verbatim** [verified-docs]:
+
+> "Agent threads are persisted across workflow runs… content generated by the agent will be
+> available in subsequent runs of the same workflow instance… it can also lead to unintended
+> state sharing if the same workflow instance is reused for different tasks. To ensure each task
+> has isolated agent state, wrap agent and workflow creation inside a helper method so each call
+> produces new agent instances with their own threads. **It is not recommended to reuse a single
+> workflow instance for multiple tasks or requests.**"
+> — learn.microsoft.com/agent-framework/workflows/state
+
+So our `fresh_workflow()` factory **is the documented Microsoft pattern**, and the de-risk
+assumption is confirmed both empirically (Spike B) and against official guidance. The lesson is
+the thesis of this document: **knowing MAF's semantics is load-bearing — a missing piece of that
+knowledge produced a false-confirm that a counter-test would have shipped.**
+
+Mechanism, traced in installed source [verified-source]: cross-run memory flows exclusively
+through `AgentSession.state` (`_sessions.py:772`). With a session and no explicit context
+provider, `Agent.run` auto-injects an `InMemoryHistoryProvider` (`_agents.py:1200-1209`);
+`save_messages` appends into `state["messages"]` (`_sessions.py:889-890`); the next run reloads
+the full history via `SessionContext.get_messages(include_input=True)` (`_sessions.py:315-348`).
+A reused workflow threads one `AgentSession` into every run → accumulation. A fresh instance per
+run gets a clean session.
+
+## 1. Feature-utilization decision table (the core deliverable)
+
+| Need | We hand-roll now | MAF 1.9.0 feature | Status | Verdict | Fase 2 action |
+|---|---|---|---|---|---|
+| **Token *measurement*** | `len(text.split())` proxy (`spikes/_harness.py`) | `UsageDetails` on `ChatResponse`/`AgentResponse` (`_types.py:393-414`), real provider counts | GA [verified-source] | **ADOPT** | Read `response.usage_details["total_token_count"]`; delete word-count. Handle `None` (not all providers report). |
+| **Token *cap* / budget stop** | `TokenMeter.charge` raises `BudgetExceeded` | Stateful `ChatMiddleware` reading `context.result.usage_details`, short-circuits (`_middleware.py:592`); officially the "Guardrails & termination" use case | GA [verified-source+docs] | **ADOPT** | Implement budget as a `ChatMiddleware`; share one meter across agents. No native *orchestration*-level token cap exists. |
+| **Round / iteration cap** | `Budget.max_rounds` + `tick_round()` | `GroupChatBuilder.with_max_rounds(n)` (`_group_chat.py:832`); Magentic `max_round_count` ctor kwarg | GA (orchestrations 1.0.0) [verified-source] | **ADOPT native; drop counter** | Use builder round caps for GroupChat/Magentic. Concurrent is single-shot (no rounds). Confirm "1 turn = 1 round" matches intent. |
+| **Fan-out state isolation** | `fresh_workflow()` factory (Spike B) | Documented "factory / helper per run" pattern | core [verified-docs] | **KEEP — it is the official pattern** | Promote `fresh_workflow()` into a core fan-out factory; never reuse a built workflow across projects. |
+| **Stop / resume around budget** | — | Workflow checkpointing: `WorkflowBuilder(checkpoint_storage=…)` + `run(checkpoint_id=…)` (`_workflow.py:707-731`) | GA [verified-source] | **ADOPT (later)** | Optional for resumable budget-stops; superstep-granular (not mid-LLM-call); resume needs identical graph. Defer past MVP unless needed. |
+| **Cost control on long debates** | — | `_compaction.py` (`TokenBudgetComposedStrategy`, `ContextWindowCompactionStrategy`) | GA [verified-source] | **COMPLEMENT** | Reduces per-call tokens; does NOT cap spend or stop. Pair with the meter only if debates grow long (D5: not MVP). |
+| **VerdictStore retrieval (ExpeL)** | Structural Jaccard over typed cost-codes + measure-type + magnitude bucket (`spikes/d_verdictstore.py`) | `MemoryStore`/`MemoryContextProvider` (keyword/substring); `mem0` (embeddings) | experimental / Preview, **wrong shape** [verified-source+docs] | **KEEP ours** | MAF retrieval is bag-of-words or embedding; cannot do structural-typed matching. Our store stays. |
+| **ExpeL injection seam** | `ExpeLContextProvider(ContextProvider)` | `ContextProvider` base (`_sessions.py:351`) — `source_id` **required** | GA [verified-source] | **KEEP seam, FIX call** | Real contract. Correct the call: `extend_instructions(source_id, instructions)` — two args, `source_id` first (`_sessions.py:253-262`), not the single-list form in the spike. |
+| **Deterministic blocking validator** | Hybrid IR+solver+MC returning `ValidatedProposal`\|`Rejection` (`spikes/c_validator.py`) | `_evaluation.py` (`Evaluator`/`LocalEvaluator`) | experimental, **wrong shape** [verified-source] | **KEEP ours** | MAF eval is offline batch returning a *quality score*; our validator is an inline gate returning a *domain object the system branches on*. Not interchangeable. |
+| **Maker-checker regression scoring** | — | `LocalEvaluator` + `@evaluator` (`_evaluation.py`) | experimental [verified-source] | **OPTIONAL (CI only)** | "Did the checker catch the planted flaw?" fits an offline CI harness. Not for the live loop. Nice-to-have, not Fase 2 core. |
+| **Model map (role→deployment)** | Planned config | `declarative/` (`AgentFactory.create_agent_from_yaml`) | Preview, **not installed** [verified-source+docs] | **ROLL OWN** | No first-class role→model registry in MAF; declarative is per-agent YAML + prerelease. Build a tiny role→deployment dict/YAML → chat-client ctor. |
+| **Wrap validator / data fns as tools** | Planned | `@tool` / `FunctionTool` with `schema=` (`_tools.py`) | GA [verified-source] | **ADOPT** | GA tool surface with explicit JSON-Schema — fits "JSON-Schema-validated, fail-fast". |
+| **Data access via MCP** | Planned | `MCPStdioTool` (local) / `MCPStreamableHTTPTool` (remote+SSE) / `MCPWebsocketTool` (`_mcp.py`) | GA core [verified-source+docs] | **ADOPT** | `MCPStdioTool` for local-profile data servers; `MCPStreamableHTTPTool` for remote. Pass creds per-run via header provider, never persisted. |
+| **Method as Agent Skill** | Planned (CLAUDE.md) | `SkillsProvider.from_paths(...)` consumes standard `SKILL.md` folders (`_skills.py`) | experimental [verified-source] | **ADOPT (pin API)** | MAF natively consumes agentskills.io `SKILL.md` + `references/`+`scripts/`. Surface is experimental → pin version, watch for breaks. |
+
+## 2. Verified MAF semantics worth codifying (the digest)
+
+These are the behaviors whose absence caused today's BLOCKER — capture them so the next session
+doesn't relearn them the hard way:
+
+- **Agents are stateless per run; state lives in the session.** `agent.run(...)` with no session
+  carries no history; pass `session=agent.create_session()` to accumulate. Sessions are bound to
+  the agent that created them (only same-provider agents can share). [verified-docs]
+- **Reused workflow ⇒ shared thread ⇒ contamination.** Build fresh per task via a factory. The
+  *only* cross-run channel is `AgentSession.state`. [verified-source+docs]
+- **`ContextProvider` requires `source_id`** (positional) for attribution; providers filter each
+  other's contributions by it (`_sessions.py:362-368`). A history provider instance is shared
+  across sessions — never store per-task state on the provider; store it in the session.
+  [verified-source+docs]
+- **Orchestration round semantics:** GroupChat round = one orchestrator selection/dispatch cycle;
+  `with_max_rounds` enforced in `_handle_response` (not the initial handler). If you subclass
+  `BaseGroupChatOrchestrator`, the builder's `max_rounds`/`termination_condition` are **ignored** —
+  you must set them yourself. [verified-source+docs]
+- **Magentic caps are constructor kwargs**, not `with_*` methods: `max_round_count`,
+  `max_stall_count` (default 3), `max_reset_count`. There is NO `with_max_round_count`. [verified-source]
+- **Token usage is on every response** as `UsageDetails` and as the OTel metric
+  `gen_ai.client.token.usage`; **there is no cost/dollar metric** — derive cost = tokens × your
+  per-model pricing downstream. [verified-source+docs]
+- **Middleware short-circuit = the official termination/guardrail mechanism** (return without
+  calling `next`); function-middleware blocking needs a `FunctionInvokingChatClient`. [verified-docs]
+
+## 3. Corrections to existing repo claims (verification duty)
+
+- **CLAUDE.md "Magentic is experimental (NOT the default)" — stands, with a nuance.** The installed
+  `agent_framework_orchestrations` 1.0.0 carries **no `@experimental` code gate** [verified-source],
+  but Microsoft's *docs* explicitly mark agent-orchestration features experimental, Magentic most so
+  ("untested outside the original Magentic-One design") [verified-docs]. So the invariant is correct
+  at the documentation/maturity level; it is just not enforced by a decorator. Keep the invariant;
+  understand it is a doc-level stance, not a code signal.
+- **STATE.md "agent-framework-core 1.9.0" — confirmed** [verified-source] (`importlib.metadata`).
+  CLAUDE.md was corrected 1.8.0→1.9.0 in Fase 1.
+- **Spike D `extend_instructions([...])` (single list) is the wrong signature.** Installed GA is
+  `extend_instructions(source_id, instructions)` — two args, `source_id` first (`_sessions.py:253-262`)
+  [verified-source]. Spike D's tests pass because they exercise `retrieve()` ranking, not the
+  injection path; fix the call when promoting the ExpeL seam to core in Fase 2.
+- **API-naming to confirm in 1.9.0:** docs use `AgentSession`/`ContextProvider`; older docs say
+  `AgentThread` and note `BaseContextProvider`/`BaseHistoryProvider` as deprecated aliases. The
+  installed source uses `SessionContext`/`ContextProvider`/`AgentSession`/`InMemoryHistoryProvider`
+  [verified-source] — trust the source for our code.
+
+## 4. The Skills question (explicit answer to "should we skillify MAF knowledge?")
+
+**No to a MAF-documentation-mirror Skill; yes to two narrow, real uses.**
+
+- **Why not a docs-mirror suite:** MAF docs are live via the `microsoft-learn` MCP and the
+  installed source. Mirroring them into local Skills duplicates a moving target (1.9→1.10→…),
+  rots, and violates DRY + the verification duty (a stale local copy can drift from truth). It is
+  the docs-mirror anti-pattern for Skills and gold-plating against D5/D6.
+- **Yes #1 — the *method* as a Skill (already in CLAUDE.md).** Our portfolio-optimization method
+  (debate → validate → learn) is correctly an Agent Skill, and MAF can **natively consume** a
+  standard `SKILL.md` folder via `SkillsProvider.from_paths(...)` [verified-source]. This is a
+  capability, not a reference dump. Caveat: the Skills surface is experimental — pin and watch.
+- **Yes #2 — a *digest*, which is THIS document.** §2 (verified semantics) + §3 (corrections) are
+  exactly the curated "MAF patterns + footguns" an agent needs while authoring. Their durable home
+  is this living capability-map (and, when the method-Skill lands, its `references/`), **not** a
+  separate skill suite. A standalone "MAF-expert Skill" only pays off if many future sessions need
+  *invokable* MAF fluency on demand — defer that decision until Fase 2 shows the need; the raw
+  material (this doc) will already exist.
+
+**Net:** the capability-map is the no-regret first artifact; it feeds both the Fase 2 design and
+any later skillification, and it answers "skillify?" with "the method yes, the reference no."
+
+## 5. Fase 2 implications (what changes in the brief)
+
+1. **Replace** hand-rolled token measurement with `UsageDetails`; **re-implement** the budget cap
+   as a shared `ChatMiddleware`; **use native** builder round caps. The hand-rolled `Budget`/
+   `TokenMeter` shrink to a thin shared-meter object the middleware drives.
+2. **Promote** `fresh_workflow()` to a core fan-out factory — it is the documented isolation pattern.
+3. **Keep** the hand-rolled VerdictStore (structural retrieval) and validator (inline gate); MAF's
+   memory and eval are the wrong shape. Promote the `ExpeLContextProvider` seam with the two-arg fix.
+4. **Adopt** GA `@tool` + `MCPStdioTool`/`MCPStreamableHTTPTool` for tools/data access.
+5. **Roll** a tiny role→deployment config map (declarative is preview + not installed).
+6. **Wire** observability (`gen_ai.client.token.usage`) for cost tracking; derive cost from pricing.
+7. **Production-risk flags (D2/D6):** mem0/redis memory, Cosmos/Redis history, and declarative are
+   all **Preview** — do not put them on the MVP critical path; the GA core (sessions, middleware,
+   observability, tools, MCP, checkpointing) is enough for the vertical slice.
+
+## 6. D7 sibling note
+
+The Claude Agent SDK sibling (eget repo, deler kun spec+golden-suite) faces the same
+feature-utilization discipline against a different substrate. This map's *structure* (need →
+feature → adopt/keep verdict) is the reusable analog; the *answers* differ. The shared golden-suite
+must not bake in MAF-specific assumptions (e.g., MAF's session-accumulation semantics).
+
+---
+
+*Sources: installed `agent_framework` 1.9.0 source (file:line cited inline); Microsoft Learn —
+workflows/state, orchestrations/{group-chat,magentic}, agents/{middleware,observability,evaluation,
+conversations/storage}, integrations/chat-history-memory-provider, agents/declarative,
+agents/tools/local-mcp-tools. Two independent research passes, 2026-06-24.*